Optimizes live docs computes for force merge in NativeEngines990KNNVectorWriter #2135

shatejas · 2024-09-23T20:43:45Z

Description

Currently live docs are computed with linear complexity even when it does not have deleted docs. This PR skips computing it

Testing

UTs and javadocs in progression. Need initial feed back

Related Issues

#2134

Check List

New functionality includes testing.
New functionality has been documented.
API changes companion pull request created.
Commits are signed per the DCO using --signoff.
Public documentation issue/PR created.

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.

quantization isn't needed Signed-off-by: Tejas Shah <[email protected]>

navneet1v

Please add java doc on all the new added classes.

navneet1v · 2024-09-23T22:07:55Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNMergeVectorValues.java

+import java.util.ArrayList;
+import java.util.List;
+
+public final class KNNMergeVectorValues {


please add java docs on top of this class.

If this is a utility, lets rename the class to reflect that. Before you added the java doc I was under impression that its some kind of vectorValues.

navneet1v · 2024-09-23T22:08:58Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNMergeVectorValues.java

+                        if (liveDocs instanceof FixedBitSet) {
+                            liveDocsInt = ((FixedBitSet) liveDocs).cardinality();
+                        } else {
+                            liveDocsInt = computeLiveDocs(values, liveDocs);
+                            values = knnVectorsReader.getFloatVectorValues(fieldInfo.name);


let add comment around this code on why we are doing it.

navneet1v · 2024-09-23T22:09:28Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNMergeVectorValues.java

+        return count;
+    }
+
+    static class KNNVectorValuesSub<T extends DocIdSetIterator> extends DocIDMerger.Sub {


why this is package private?

Needed in KNNVectorValuesIterator

navneet1v · 2024-09-23T22:10:07Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNMergeVectorValues.java

+        return subs;
+    }
+
+    public static List<KNNVectorValuesSub<ByteVectorValues>> mergeByteVectorValues(FieldInfo fieldInfo, MergeState mergeState)


add java doc.

navneet1v · 2024-09-23T22:12:54Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNVectorValuesFactory.java

+        return getVectorValues(vectorDataType, new KNNVectorValuesIterator.FieldWriterIteratorValues<>(docIdWithFieldSet, vectors));
+    }
+
+    public static <T> KNNVectorValues<T> getVectorValues(


navneet1v · 2024-09-23T22:13:28Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNVectorValuesFactory.java

+    @VisibleForTesting
    @SuppressWarnings("unchecked")
-    private static <T> KNNVectorValues<T> getVectorValues(
+    static <T> KNNVectorValues<T> getVectorValues(


why to make it package private?

Needed for unit test

lets try to avoid this and use one of the public function to test this function.

navneet1v · 2024-09-23T22:17:12Z

Please also update the changelog.

NativeEnginesKNNVectorsWriter Currently vector values are iterated irrespective of whether there are deleted docs in the segment. This makes sure they aren't Signed-off-by: Tejas Shah <[email protected]>

navneet1v · 2024-09-24T04:59:18Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNMergeVectorValues.java

+        int count = 0;
+        for (int index = 0; index < liveDocs.length(); index++) {
+            count += liveDocs.get(index) ? 1 : 0;
+        }


This will iterate over all the bits in the bitset, rather I would suggest VectorValues to iterate over the docIds and then see if the docIds is present in bitset or not. Because it can happen for a field only few docIds are present in liveDocs.

Ref: https://github.com/opensearch-project/OpenSearch/blob/main/server/src/main/java/org/apache/lucene/util/CombinedBitSet.java#L62-L68

That requires another call to knnVectorsReader.get[Float/Byte]VectorValues for each segment, which is an anticipated latency driver. IO operation might be more expensive than looping through considering merges

Check my comment here: #2135 (comment) using live docs will not give you correct doc count.

navneet1v · 2024-09-24T05:00:06Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNMergeVectorValues.java

+                ByteVectorValues values = knnVectorsReader.getByteVectorValues(fieldInfo.getName());
+                if (values != null) {
+                    final Bits liveDocs = mergeState.liveDocs[i];
+                    final int liveDocsCardinality = cardinality(liveDocs, Math.toIntExact(values.cost()));


for understanding on how much time we are spending on this operation its better to add some debug logs.

navneet1v · 2024-09-24T05:03:26Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNMergeVectorValues.java

+     * @return List of KNNVectorSub
+     * @throws IOException
+     */
+    public static KNNVectorValuesIterator.MergeFloat32VectorValuesIterator mergeFloatVectorValues(


Suggested change

public static KNNVectorValuesIterator.MergeFloat32VectorValuesIterator mergeFloatVectorValues(

public static KNNVectorValuesIterator.MergeFloat32VectorValuesIterator getMergedFloatVectorValuesIterator(

Changing this to return MergedFloatVectorValues which extends FloatVectorValues so iterator will not make sense

navneet1v · 2024-09-24T05:06:43Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNMergeVectorValues.java

+        if (liveDocs instanceof FixedBitSet) {
+            return ((FixedBitSet) liveDocs).cardinality();
+        }


why this special case for fixedbitset and not for SparseFixedBitSet ?

I don't think its being used. I was looking at the usages of LiveDocsFormats write

DocumentWritersPerThread

Pending Deletes

read only returns FixedBits. Moreover while debugging I did not see SparseFixedBitSet usage. I can change it to instanceof BitSet to be safe

I am not super confident with this and also the next loop to find the live docs. I would check with lucene community first before making any assumptions here to avoid any possible regression, since this is on the write path.

community first before making any assumptions

Can you elaborate about the concerns with BitSet usage, as in performance or functionality? cardinality is supposed to return set bits and lucene seems to have optimized it for Fixed and Sparse with bit shifting

the next loop to find the live docs

For this is the concern performance or functionality?

Asking because we should do our due diligence before reaching out to the lucene community

@shatejas I think we don't need to reach out to Lucene, I thought of the case where this bitset count will fail and you can try this out with your cluster.

Live docs tells the total number of docs which are not deleted on the segment. Now a doc may or may not contain the vector field. So consider a case where there are 3 docs in the segment.
doc1 : has vector field : doc is deleted
doc2: has no vector field : doc is not deleted
doc3: has vector field : doc is not deleted.

With the above case the bitset will give live docs as 2 but for the vector field the live docs is only 1.

Hence, using the bitset.cardinality() will never give you right number of live docs.

and this is the reason why lucene doesn't use this during segment merges during the training.

cc: @jmazanec15 and @heemin32

navneet1v · 2024-09-24T05:12:40Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNVectorValues.java

     */
-    @Deprecated
    public long totalLiveDocs() {


if we removing deprecated from here, we should also update the java docs.

Missed as there was no @deprecated javadoc tag. The convention is to add @deprecated in javadocs tag and give a reason next to it instead of combining it with existing

navneet1v · 2024-09-24T05:14:34Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNVectorValuesIterator.java

+
+        @Override
+        public int advance(int docId) throws IOException {
+            throw new UnsupportedOperationException();


add the exception message.

navneet1v · 2024-09-24T05:30:18Z

src/main/java/org/opensearch/knn/index/vectorvalues/KNNVectorValuesIterator.java

+        public abstract U vectorValue() throws IOException;
+    }
+
+    class MergeFloat32VectorValuesIterator extends MergeSegmentVectorValuesIterator<FloatVectorValues, float[]> {


Why we went with creating the iterator route here? rather we could have just extended the FloatVectorValues and ByteVectorValues. This would have avoided all these iterators and new strategy classes. Because FloatVectorValues extends the DISI and a lot of code could have been easily reused.

Ref: https://github.com/apache/lucene/blob/branch_9x/lucene/core/src/java/org/apache/lucene/codecs/KnnVectorsWriter.java#L286

The problem with LuceneMergedFloatVector values was it was not calculating the cost() correctly, hence we needed some wrapper classes to calculate the liveDocs aka cost correctly.

One suggestion here I would have is to go in lucene and actually fix the MergedFloat32VectorValues to ensure that its cost() function is giving right cost.

Didn't realize its a DISI, changing it

We can take up the lucene contribution as a follow up

We can take up the lucene contribution as a follow up

Please create a gh issue and ref it here.

shatejas · 2024-10-02T16:04:58Z

After more experiments we figured that this live docs iteration is not the root cause, This only yields 32ms gains for 1.6M vectors adding an overhead of maintaining the merge vectors code.

Decided to discard this

Makes sure KNNVectorValues aren't recreated unnecessarily when

e348524

quantization isn't needed Signed-off-by: Tejas Shah <[email protected]>

shatejas mentioned this pull request Sep 23, 2024

Makes sure KNNVectorValues aren't recreated unnecessarily when #2133

Merged

5 tasks

shatejas marked this pull request as ready for review September 23, 2024 21:26

shatejas requested review from heemin32, navneet1v, VijayanB, vamshin, jmazanec15, naveentatikonda, junqiu-lei, martin-gaievski, ryanbogan and luyuncheng as code owners September 23, 2024 21:26

shatejas force-pushed the force-merge-reg-live-docs branch from 0a280c1 to 2ae6af5 Compare September 23, 2024 21:37

navneet1v reviewed Sep 23, 2024

View reviewed changes

shatejas force-pushed the force-merge-reg-live-docs branch from 2ae6af5 to 4fcd529 Compare September 24, 2024 00:46

Optimizes totalLiveDocs calculation for merge in

316a903

NativeEnginesKNNVectorsWriter Currently vector values are iterated irrespective of whether there are deleted docs in the segment. This makes sure they aren't Signed-off-by: Tejas Shah <[email protected]>

shatejas force-pushed the force-merge-reg-live-docs branch from 4fcd529 to 316a903 Compare September 24, 2024 00:50

navneet1v reviewed Sep 24, 2024

View reviewed changes

shatejas closed this Oct 2, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Optimizes live docs computes for force merge in NativeEngines990KNNVectorWriter #2135

Optimizes live docs computes for force merge in NativeEngines990KNNVectorWriter #2135

shatejas commented Sep 23, 2024

navneet1v left a comment

navneet1v Sep 23, 2024

navneet1v Sep 24, 2024

navneet1v Sep 23, 2024

navneet1v Sep 23, 2024

shatejas Sep 24, 2024

navneet1v Sep 23, 2024

navneet1v Sep 23, 2024

navneet1v Sep 23, 2024

shatejas Sep 24, 2024

navneet1v Sep 24, 2024

navneet1v commented Sep 23, 2024

navneet1v Sep 24, 2024

navneet1v Sep 24, 2024

shatejas Sep 25, 2024

navneet1v Sep 25, 2024

navneet1v Sep 24, 2024

navneet1v Sep 24, 2024

shatejas Sep 25, 2024

navneet1v Sep 24, 2024

shatejas Sep 25, 2024

navneet1v Sep 25, 2024

shatejas Sep 25, 2024

navneet1v Sep 25, 2024

navneet1v Sep 24, 2024

shatejas Sep 25, 2024

navneet1v Sep 24, 2024

navneet1v Sep 24, 2024 •

edited

Loading

shatejas Sep 25, 2024

navneet1v Sep 25, 2024

shatejas commented Oct 2, 2024

	public static KNNVectorValuesIterator.MergeFloat32VectorValuesIterator mergeFloatVectorValues(
	public static KNNVectorValuesIterator.MergeFloat32VectorValuesIterator getMergedFloatVectorValuesIterator(

Optimizes live docs computes for force merge in NativeEngines990KNNVectorWriter #2135

Optimizes live docs computes for force merge in NativeEngines990KNNVectorWriter #2135

Conversation

shatejas commented Sep 23, 2024

Description

Testing

Related Issues

Check List

navneet1v left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

navneet1v commented Sep 23, 2024

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

navneet1v Sep 24, 2024 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

shatejas commented Oct 2, 2024

navneet1v Sep 24, 2024 •

edited

Loading