Concurrency optimization for native graph loading #2345
Conversation
Signed-off-by: Ganesh Ramadurai <[email protected]> minor updates
Gankris96
force-pushed
the
concurrent-graph-load
branch
from
7cb8710 to
8e90b88
Compare
Gankris96
changed the title
|
Please add an entry in the changelog. |
| // close the indexInput | ||
| @Override | ||
| public void close() { | ||
| if (readStream != null) { |
There was a problem hiding this comment.
Could you set preload=false in here?
| try { | ||
| indexSizeKb = Math.toIntExact(directory.fileLength(vectorFileName) / 1024); | ||
| readStream = directory.openInput(vectorFileName, IOContext.READONCE); | ||
| readStream.seek(0); |
There was a problem hiding this comment.
Curious, why we need this? Is this for those IndexInput subclasses having a lazy loading kind of stuffs?
| * Preloads the entry by opening the indexInput | ||
| */ | ||
|
|
||
| public abstract void preload(); |
There was a problem hiding this comment.
-
Can we give a meaningful name? What it does is actually doing a preparation before 'loading' bytes into memory. e.g. opening a stream
-
Can we make it have an empty body?
Then let those subclasses need this to add their implementation.
void preload() {
}
| @@ -328,6 +328,8 @@ public NativeMemoryAllocation get(NativeMemoryEntryContext<?> nativeMemoryEntryC | |||
|
|
|||
| // Cache Miss | |||
| // Evict before put | |||
| // preload the graph file before proceeding to load the graph into memory | |||
There was a problem hiding this comment.
It's not preloading the graph file, isn't it? 🤔
Opening a stream != loading the graph file, I believe
| @@ -75,6 +91,18 @@ public static class IndexEntryContext extends NativeMemoryEntryContext<NativeMem | |||
| @Getter | |||
| private final String modelId; | |||
|
|
|||
| @Setter | |||
| @Getter | |||
There was a problem hiding this comment.
Could you remove @Setter here?
This variable should be modified from outside other than testing.
Instead of having a public scope getter, please find another way to change it in testings.
|
HI @Gankris96, thank you for the PR. |
Hi @0ctopus13prime - yes i am working on getting the benchmarking numbers. Primarily trying to test it on remote store backed index to see the performance gains. Will update with Benchmarking numbers soon. |
Gankris96
requested review from
heemin32,
navneet1v,
VijayanB,
vamshin,
jmazanec15,
naveentatikonda,
junqiu-lei,
martin-gaievski,
ryanbogan,
luyuncheng and
shatejas
as code owners
Description
Fixes #2265
Refactors the graph load into a 2 step approach detailed here - #2265 (comment)
This will help to move out the opening of indexInput file outside of the synchronized block so that the graphfile can be downloaded in parallel even if the graph load and createIndexAllocation are inside synchronized block.
Related Issues
Resolves #2265
Check List
--signoff.By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.
For more information on following Developer Certificate of Origin and signing off your commits, please check here.