Change API to use move semantics or custom buffer type

[elcritch]

Making datastore multi-threaded would pose a problem with transferring data ownership between threads. It seems we could either use move, or switch from seq[byte] to a custom buffer. I'm not sure which is better.

This looks related to:

• Refactor the Datastore API to be generic #11
• Refactor get so that it takes a var parameter to avoid copying of data #12

[elcritch]

My concern here would be that with refc GC and preventing data copies of large blocks. If my understanding is correct there's not a shared heap, so just using move to transfer the data to another thread wouldn't work. It'd crash or corrupt things. Though I'm going to do some quick tests/research as I'm not too familiar with refc in these matters.

We can keep the the seq[bytes] alive on the async thread. That'd be a bit more work for items like put where the data would be sent, but we'll need a hold onto it and only free it once the data-thread has sent back an ok.

Actually, writing this out, I realize that in this case the get operation might be harder. Since we don't seem to know the size of seq[bytes] upfront, we'd need to dynamically allocate on the data-thread. Moving the data over to the main-thread would require doing a copy to the target seq[bytes]. Maybe shallowCopy could be use somehow?

[elcritch]

@dryajov and myself discusses this issue some more, and the get and query operations do pose a bit more of an issue to the simple model of lending data to a thread to do work. That may work well for computations but does pose challenges for non-copying data mechanisms.

We checked out a few variants on GC_ref and GC_unref which do fail when transferring GC types, as expected. There is an option to use protect / dispose but it appears undocumented and only mentioned briefly on the Nim forums:

• protect/dispose - which are the multithreading counterparts of GC_ref/GC_unref. The idea is that one can allocate an object in one thread, protect the pointer, send the pointer over to another thread and have that thread dispose it when done. protect returns a ForeignCell object, which has a pointer to the owning GC, dispose expects the cell to free it/decrease its RC.

This approach, while potentially usable strikes me as finicky and requiring getting a lot of GC details correct. It's also not future compatible with ARC/ORC either.