-
why is GenerateEmbeddingsAsync returning Couldn't it just return
|
Beta Was this translation helpful? Give feedback.
Replies: 3 comments 1 reply
-
@westey-m can you take a look at this? |
Beta Was this translation helpful? Give feedback.
-
@markat1, E.g. it's possible to create views over the data without needing to create a new instance / copy of the data, avoiding potentially expensive allocations. You can read more about the benefits and usage here: Other vector related .net APIs consume Converting to and from an array is also not difficult in cases where an array is needed, but it's important to note that ToArray does a copy which incurs an allocation cost. ReadOnlyMemory<float> vector1 = new float[] { 30f, 31f, 32f, 33f };
float[] array1 = vector1.ToArray(); See https://learn.microsoft.com/en-us/dotnet/api/system.readonlymemory-1.toarray |
Beta Was this translation helpful? Give feedback.
-
thank you for your elaborated answer @westey-m! - one last question, if that's okay! In the context of implementing GenerateEmbeddingsAsync - any recommended/optimization thoughts on how I should handle embeddings returned back from for example ollama? Here's an example of me implementing GenerateEmbeddingsAsync from IEmbeddingGenerationService:
|
Beta Was this translation helpful? Give feedback.
@markat1,
ReadOnlyMemory<T>
is used in .net to work more efficiently with large blocks of data, like in the case of vectors.E.g. it's possible to create views over the data without needing to create a new instance / copy of the data, avoiding potentially expensive allocations. You can read more about the benefits and usage here:
https://learn.microsoft.com/en-us/dotnet/standard/memory-and-spans/
Other vector related .net APIs consume
ReadOnlySpan<T>
as input, which is interrelated withReadOnlyMemory<T>
and can be accessed as a property onReadOnlyMemory<T>
.See e.g. the
TensorPrimitives.CosineSimilarity
method:https://learn.microsoft.com/en-us/dotnet/api/system.numerics.tensors.tensorp…