-
Notifications
You must be signed in to change notification settings - Fork 3.7k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add G-retriever (GNN+LLM) example #9167
Conversation
@Kh4L reviews welcome |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #9167 +/- ##
==========================================
- Coverage 87.33% 87.21% -0.13%
==========================================
Files 460 477 +17
Lines 30385 31051 +666
==========================================
+ Hits 26536 27080 +544
- Misses 3849 3971 +122 ☔ View full report in Codecov by Sentry. |
@akihironitta @rusty1s addressed reviews, plz lmk if anything else needed to merge. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sorry for the extreme delay :( Will have a look during this weekend (long weekend in UK) :)
@akihironitta anything else needed to merge? |
Splits #9167 into multiple PRs. --------- Co-authored-by: puririshi98 <[email protected]> Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Rishi Puri <[email protected]> Co-authored-by: Akihiro Nitta <[email protected]>
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## web-qsp-integ #9167 +/- ##
================================================
Coverage ? 87.31%
================================================
Files ? 481
Lines ? 31345
Branches ? 0
================================================
Hits ? 27369
Misses ? 3976
Partials ? 0 ☔ View full report in Codecov by Sentry. |
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
for more information, see https://pre-commit.ci
1. #9462 2. #9480 3. **->** #9481 4. #9167 --- Breaking down PR #9167 further --------- Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Akihiro Nitta <[email protected]> Co-authored-by: rusty1s <[email protected]>
LLM
class #9462nn.models.GRetriever
#9480WebQSPDataset
#9481repro:
Latest NVIDIA PyG container
+
git config --global credential.helper store; huggingface-cli login; cd /opt/pyg; pip uninstall -y torch-geometric; rm -rf pytorch_geometric; git clone -b gnn-llm-model-integration https://github.com/pyg-team/pytorch_geometric.git; cd /opt/pyg/pytorch_geometric; pip install .; pip install peft datasets transformers pcst_fast sentencepiece; python3 examples/llm_plus_gnn/g_retriever.py
old PR: #9154
note: pure cpu is 220x slower than pure GPU using a single Grace Hopper (for llama-7b)
info:
tried gemma, performs worse in all train/val/test metrics. most likely needs some tuning, will leave this as future work as part of the community sprint to try many LLM and GNN combos and tune them. Therefore keeping the default llama2
the new gemma-v2 is also much worse than llama2