Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Deserialization & Type system limitation #1997

Open
andrewdavidmackenzie opened this issue Aug 20, 2023 · 3 comments
Open

Deserialization & Type system limitation #1997

andrewdavidmackenzie opened this issue Aug 20, 2023 · 3 comments

Comments

@andrewdavidmackenzie
Copy link
Owner

andrewdavidmackenzie commented Aug 20, 2023

Example happens in Mandelbrot.

Generate_pixels in fact generates a row of pixels as array/array/number
E.g. [[0,0], [1, 0], [2, 0] … [x, 0]]

This is fed into pixel to point sub-flow that accepts array/number
E.g. [0,0]

Semantically this should be deserialized, and many jobs possibly launched to handle the queue of inputs…

But subroutes are used on that array/number input, so the connection is formed from the source to the destination, not using the type declared on the sub-flow input.

The problem is that deserialization is currently left to run-time, and not taken into account at compile time, so the subroute is applied incorrectly.,

@andrewdavidmackenzie
Copy link
Owner Author

Test files to reproduce it are attached.
test.toml generates array/array/number and sends it to the input of sub_flow.toml.
E.g. [[0,10], [1, 9]… [10,0]]

That sub flow has the input types declared as array/number, expecting it to be deserialized.
E.g.
[0,10]
[1,9]

[10,0]

It then takes the first element using “/0”, what out wants to do is take the first number of each tuple of coordinates
E.g.
0
1

10

So it can add each tuple

But as it is not deerialized, “/0” picks the first element of array/array/number
E.g.
[0,10]

Not the single number if was expecting.

@andrewdavidmackenzie
Copy link
Owner Author

Proposal
Re-write Connection to build up a chain of operations and subroutes, before collapsing, so it is aware of the deserialization that will be done at runtime.

It indicates in the connection what deserialization will be needed at run-time, and the runtime only implements it as instructed when directed to do so?

Need to keep general deserialization at runtime to cover generic type cases, where only the runtime knows the types involved?

See if can get existing code running using this change.

Then add a test like above that fails.

Then implement the deserialization in runtime as si trusted by the compiled and manifest and see if can get test passing.

Then fix any errors introduced

Then go back and look at Mandelbrot, see if we can get many jobs queued up for the prows of pixels.

@andrewdavidmackenzie
Copy link
Owner Author

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant