Avoid reallocating a new array when reading it in my CustomProcedure

[danielvan1]

Hi,

I'm trying to avoid reallocating a new array for each time I am reading it in my CustomProcedure. I'm working with a loop similar to the one in Sum.cs:

Copy code
while ((key = GetNextArg(input, ref offset)).Length > 0) {
    if (garnetApi.GET(ref key, ref ArgSlice array) == GarnetStatus.OK) {
        // Process the array
    }
}

From my understanding, the call garnetApi.GET(ref key, ref ArgSlice array) allocates a new array each time it's invoked. This interpretation comes from looking at the method in MainStoreOps.cs.

I believe that to avoid array reallocation, I should use this method in MainStoreOps.cs.

1. However, I'm unsure about the meaning and usage of the SpanByte input parameter in this method.
2. How do you create the objects SpanByte input and SpanByteAndMemory output in the method of the CustomProcedure
3. Is this the correct approach to prevent reallocation, or is there a better way to handle this scenario?

Any guidance would be greatly appreciated!

Thanks.

[yrajas]

Hi

Thanks for trying this feature.

Using the GET api that takes in the ArgSlice argument is the recommended way to perform GET operations from a custom procedure/transaction as shown in the Sum.cs example. The allocation is required to ensure the input buffer is setup in the right way for the backend processing.

[yrajas]

Thanks for pointing this out. Looking into it.
Meanwhile, could you please share a repro of the client script that you are using as well?

[danielvan1]

To reproduce the error you can have a look at this GitHub repo:
https://github.com/danielvan1/GarnetExample

It creates 20k random arrays and writes them to the garnet server. I then try to do a vector sum for each of the 20k arrays in the VectorSum.cs.

To reproduce the error you can go to the root folder and:

1. Run dotnet run --project Server/Server.csproj
2. Run dotnet run --project Client/Client.csproj

The expected behavior is the error described here.

[yrajas]

Thanks. Have the repro.
Looking into potential solutions.
Thanks for your patience.

[yrajas]

Please check out the new APIs as part of this PR #700 that help with buffer management like in your case.