feat: Add IPC writer scaffolding #564
Conversation
There was a problem hiding this comment.
Thank you!
The CI failure is I think because of the missing line here:
arrow-nanoarrow/ci/scripts/bundle.py
Lines 205 to 206 in d6368d0
There was a problem hiding this comment.
Sorry for missing a few of these in the initial round!
I think we want to append to the output buffer rather than resize it (more flexible for the caller, who can choose to resize it to zero before they call or not depending what they're about to do/what they just did)!
| return NANOARROW_OK; | ||
| } | ||
|
|
||
| void* data = flatcc_builder_get_direct_buffer(&private->builder, NULL); |
There was a problem hiding this comment.
My reading of https://github.com/dvidelabs/flatcc/blob/fe24a8012d689d4f7ca9e1ccdfeb012c74ca96bb/include/flatcc/flatcc_builder.h#L1802-L1805 is that flatcc_builder_get_direct_buffer() it might fail for large output. It's a bummer the copy function doesn't work (I was trying to replicate https://github.com/dvidelabs/flatcc/blob/fe24a8012d689d4f7ca9e1ccdfeb012c74ca96bb/src/runtime/builder.c#L1972-L1988 but using our own allocation).
Feel free to test/solve later!
There was a problem hiding this comment.
The returns-null case is checked and we return ENOMEM if we can't allocate the direct buffer. We can try again with copy_buffer later
There was a problem hiding this comment.
I don't think it's an allocation failure, I think the builder uses multiple pages of non-contiguous memory for large output. +1 to debugging later!
paleolimbot
changed the title
Add
ArrowIpcEncoder, init/reset, and tests. Extracted from #555 (review)