Added support for weaviate vector database #353
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
S1LV3RJ1NX
requested changes
Oct 7, 2024
backend/vectordb.requirements.txt
Outdated
| @@ -2,4 +2,4 @@ | |||
| singlestoredb==1.0.4 | |||
|
|
|||
| ### Weaviate client (in progress) | |||
There was a problem hiding this comment.
Can remove this in progress comment.
| # TODO (chiragjn): Revisit this, we should not be letting `value` be empty | ||
| document_ids.add(group.get("groupedBy", {}).get("value", "") or "") | ||
| return list(document_ids) | ||
| pass |
There was a problem hiding this comment.
This function can be removed as it is not in the base class.
| # TODO (chiragjn): Revisit this, we should not be letting `value` be empty | ||
| document_ids.add(group.get("groupedBy", {}).get("value", "") or "") | ||
| return list(document_ids) | ||
| pass | ||
|
|
||
| def delete_documents(self, collection_name: str, document_ids: List[str]): |
There was a problem hiding this comment.
This function can be removed as it is not in the base class.
Co-authored-by: Prathamesh Saraf <[email protected]>
|
Hello @d-vignesh |
|
This is a visualization of the code changes in the PR, generated using Vibinex. Nodes are files/functions/objects. Edges denote function calls. Call Stack Diff%%{init: { 'theme': 'neutral', 'themeVariables': { 'fontSize': '20px' }, 'flowchart': { 'nodeSpacing': 100, 'rankSpacing': 100 } } }%%
flowchart LR
subgraph snln ["backend/modules/vector_db/weaviate.py"]
foci["debug"]
click foci href "https://github.com/Alokit-Innovations-Test/cognita/blob/f6ff788038f583d6ed5791141151c9123cb3e663/backend/modules/vector_db/weaviate.py#L0" _blank
jmue["WeaviateVectorDB"]
class jmue modified
click jmue href "https://github.com/Alokit-Innovations-Test/cognita/pull/4/files#diff-03072e343ca60d43bffe05672b8c46a0767628a2f0d5a6e41fd4a61c26006983R23" _blank
end
class snln modified
subgraph bplz ["my_lib/utils.py"]
mwgd["connect_to_local"]
click mwgd href "https://github.com/Alokit-Innovations-Test/cognita/blob/f6ff788038f583d6ed5791141151c9123cb3e663/my_lib/utils.py#L0" _blank
fyaq["Client"]
click fyaq href "https://github.com/Alokit-Innovations-Test/cognita/blob/f6ff788038f583d6ed5791141151c9123cb3e663/my_lib/utils.py#L0" _blank
end
subgraph fzbb ["my_lib/models.py"]
pcmy["_get_records_to_be_updated"]
click pcmy href "https://github.com/Alokit-Innovations-Test/cognita/blob/f6ff788038f583d6ed5791141151c9123cb3e663/my_lib/models.py#L0" _blank
fyjh["create_class"]
click fyjh href "https://github.com/Alokit-Innovations-Test/cognita/blob/f6ff788038f583d6ed5791141151c9123cb3e663/my_lib/models.py#L0" _blank
hreq["res.get"]
click hreq href "https://github.com/Alokit-Innovations-Test/cognita/blob/f6ff788038f583d6ed5791141151c9123cb3e663/my_lib/models.py#L0" _blank
end
subgraph ducq ["weaviate/auth.py"]
ugei["AuthApiKey"]
click ugei href "https://github.com/Alokit-Innovations-Test/cognita/blob/f6ff788038f583d6ed5791141151c9123cb3e663/weaviate/auth.py#L0" _blank
end
subgraph mgnc ["weaviate/classes.py"]
ctyt["collections.create"]
click ctyt href "https://github.com/Alokit-Innovations-Test/cognita/blob/f6ff788038f583d6ed5791141151c9123cb3e663/weaviate/classes.py#L0" _blank
nkvn["delete_many"]
click nkvn href "https://github.com/Alokit-Innovations-Test/cognita/blob/f6ff788038f583d6ed5791141151c9123cb3e663/weaviate/classes.py#L0" _blank
ozvf["keys"]
click ozvf href "https://github.com/Alokit-Innovations-Test/cognita/blob/f6ff788038f583d6ed5791141151c9123cb3e663/weaviate/classes.py#L0" _blank
end
subgraph vvhg ["cognita/backend/modules/vector_db/weaviate.py"]
wylz["api_key"]
click wylz href "https://github.com/Alokit-Innovations-Test/cognita/blob/f6ff788038f583d6ed5791141151c9123cb3e663/cognita/backend/modules/vector_db/weaviate.py#L0" _blank
end
classDef modified stroke:black,fill:yellow
classDef added stroke:black,fill:#b7e892,color:black
classDef deleted stroke:black,fill:red
jmue =="<a target='_blank' href='https://github.com/Alokit-Innovations-Test/cognita/pull/4/files#diff-03072e343ca60d43bffe05672b8c46a0767628a2f0d5a6e41fd4a61c26006983L34'>Line 34</a>" =====>fyjh
jmue =="<a target='_blank' href='https://github.com/Alokit-Innovations-Test/cognita/pull/4/files#diff-03072e343ca60d43bffe05672b8c46a0767628a2f0d5a6e41fd4a61c26006983R119'>Line 119</a>" =====>foci
jmue =="<a target='_blank' href='https://github.com/Alokit-Innovations-Test/cognita/pull/4/files#diff-03072e343ca60d43bffe05672b8c46a0767628a2f0d5a6e41fd4a61c26006983R96'>Line 96</a>" =====>foci
jmue =="<a target='_blank' href='https://github.com/Alokit-Innovations-Test/cognita/pull/4/files#diff-03072e343ca60d43bffe05672b8c46a0767628a2f0d5a6e41fd4a61c26006983R30'>Line 30</a>" =====>wylz
jmue =="<a target='_blank' href='https://github.com/Alokit-Innovations-Test/cognita/pull/4/files#diff-03072e343ca60d43bffe05672b8c46a0767628a2f0d5a6e41fd4a61c26006983R133'>Line 133</a>" =====>ozvf
jmue =="<a target='_blank' href='https://github.com/Alokit-Innovations-Test/cognita/pull/4/files#diff-03072e343ca60d43bffe05672b8c46a0767628a2f0d5a6e41fd4a61c26006983R124'>Line 124</a>" =====>nkvn
jmue =="<a target='_blank' href='https://github.com/Alokit-Innovations-Test/cognita/pull/4/files#diff-03072e343ca60d43bffe05672b8c46a0767628a2f0d5a6e41fd4a61c26006983R35'>Line 35</a>" =====>ctyt
jmue =="<a target='_blank' href='https://github.com/Alokit-Innovations-Test/cognita/pull/4/files#diff-03072e343ca60d43bffe05672b8c46a0767628a2f0d5a6e41fd4a61c26006983R47'>Line 47</a>" =====>pcmy
jmue =="<a target='_blank' href='https://github.com/Alokit-Innovations-Test/cognita/pull/4/files#diff-03072e343ca60d43bffe05672b8c46a0767628a2f0d5a6e41fd4a61c26006983R26'>Line 26</a>" =====>mwgd
jmue =="<a target='_blank' href='https://github.com/Alokit-Innovations-Test/cognita/pull/4/files#diff-03072e343ca60d43bffe05672b8c46a0767628a2f0d5a6e41fd4a61c26006983L23'>Line 23</a>" =====>fyaq
jmue =="<a target='_blank' href='https://github.com/Alokit-Innovations-Test/cognita/pull/4/files#diff-03072e343ca60d43bffe05672b8c46a0767628a2f0d5a6e41fd4a61c26006983L125'>Line 125</a>" =====>hreq
jmue =="<a target='_blank' href='https://github.com/Alokit-Innovations-Test/cognita/pull/4/files#diff-03072e343ca60d43bffe05672b8c46a0767628a2f0d5a6e41fd4a61c26006983L25'>Line 25</a>" =====>ugei
linkStyle 1,2,3,4,5,6,7,8 stroke:green,stroke-width:8
linkStyle 0,9,10,11 stroke:red,stroke-width:10
To modify DiffGraph settings, go to your Vibinex settings page. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Tested the changes locally using weaviate cloud instance.
Initial Insertion(Added two files, file1 and file2):
Incremental Insertion(Added two more file, file3 and file4):
Weaviate dashboard(4 objects inserted to the collection):
