Support for node streams?

[everett1992]

Does this library support, or plan to support, node's stream module? The makeReader function only accepts buffers. That requires reading the whole ion document into memory before beginning to parse.

[zslayton]

ion-js needs to support both node and browser runtimes. Is there a common stream API we could code against for both?

[everett1992]

Looking at the library I don't think it would be easy to target node's stream api because it is async and callback based, while next and stepin are sync.

I see an experimental web streams api, but it's not the same as nodes. https://developer.mozilla.org/en-US/docs/Web/API/Streams_API

I'm going to try reading the stream in a worker_thread, pushing data into a SharedArrayBuffer, and using Atomics.wait to block the main thread.

[everett1992]

After some initial success I've hit a roadblock
https://stackoverflow.com/questions/66145735/is-there-a-blocking-arraybuffer-that-is-filled-by-a-worker-thread

[jimmywarting]

i would say iterators are the common factor, and it's easy to create a node stream from it

stream.Readable.from(iterator)

think there was some proposal also about ReadableStream.from(iterable) too (for web)

[desaikd]

@jimmywarting Can you please elaborate maybe with an example of using an iterator for reader API? Did you mean to provide an iterator for DOM values (It does mean materializing Ion data to DOM values)?