New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Improve Llama2 and gpt_neox performance with Habana fused RoPE and RMSNorm #321

Merged

regisss merged 7 commits into main from habana_llm

Aug 8, 2023

Collaborator

mandy-li commented Aug 7, 2023

What does this PR do?

Improve Llama2 inference performance by using habana fused RoPE and RMSNorm
Improve gpt_neox inference performance by using habana fused RoPE

Before submitting

This PR fixes a typo or improves the docs (you can dismiss the other checks if that's the case).
Did you make sure to update the documentation with your changes?
Did you write any new necessary tests?


          Improve Llama2 and gpt_neox performance by using Habana fused RoPE an…

a560085

…d RMSNorm

mandy-li requested review from fxmarty, regisss and ZhaiFeiyue

August 7, 2023 16:20

HuggingFaceDocBuilderDev commented Aug 7, 2023 •

edited

Loading

The documentation is not available anymore as the PR was closed or merged.

mandy-li added 3 commits

August 7, 2023 10:24


          Fixed reformat issues

76e15bb


          Fixed reformat issues

1b6f82d


          Fixed reformat issues

d7fc0f7

ZhaiFeiyue reviewed

View reviewed changes

optimum/habana/transformers/models/gpt_neox/modeling_gpt_neox.py Show resolved Hide resolved

ZhaiFeiyue reviewed

View reviewed changes

optimum/habana/transformers/models/llama/modeling_llama.py Show resolved Hide resolved

ZhaiFeiyue reviewed

View reviewed changes

optimum/habana/transformers/models/llama/modeling_llama.py Show resolved Hide resolved

ZhaiFeiyue reviewed

View reviewed changes

optimum/habana/transformers/models/gpt_neox/modeling_gpt_neox.py Outdated Show resolved Hide resolved

ZhaiFeiyue reviewed

View reviewed changes

optimum/habana/transformers/models/llama/modeling_llama.py Outdated Show resolved Hide resolved

ZhaiFeiyue reviewed

View reviewed changes

optimum/habana/transformers/models/llama/modeling_llama.py Outdated Show resolved Hide resolved

mandy-li and others added 3 commits

August 7, 2023 22:01


          Update optimum/habana/transformers/models/gpt_neox/modeling_gpt_neox.py

77eb7f9

Co-authored-by: ZhaiFeiyue <[email protected]>


          Update optimum/habana/transformers/models/llama/modeling_llama.py

f62a496

Co-authored-by: ZhaiFeiyue <[email protected]>


          Update optimum/habana/transformers/models/llama/modeling_llama.py

0cf3325

Co-authored-by: ZhaiFeiyue <[email protected]>

Collaborator

ZhaiFeiyue commented Aug 8, 2023

@mandy-li nice PR 👍

ZhaiFeiyue approved these changes

View reviewed changes

regisss approved these changes

View reviewed changes

Collaborator

regisss left a comment

LGTM!

regisss merged commit f54e025 into main

9 checks passed

regisss deleted the habana_llm branch

August 8, 2023 23:06

schoi-habana pushed a commit that referenced this pull request


          Improve Llama2 and gpt_neox performance with Habana fused RoPE and RM…

7b6f024

…SNorm (#321)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet