Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update PyTorch pin and enable MPS qops #725

Merged
merged 3 commits into from
May 9, 2024
Merged

Conversation

malfet
Copy link
Contributor

@malfet malfet commented May 8, 2024

Updated pin to .dev20240507 and allow linter:int8 and linter:int4 acceleration on MPS

Test Plan:

% python3 torchchat.py generate stories110M --dtype float16 --quantize '{"linear:int4": {"groupsize": 32}}' --prompt "Once upon a time," --device mps

crashed before, yields 44 tokens/sec after. And

% python3 torchchat.py generate stories110M --dtype float16 --quantize '{"linear:int8": {"groupsize": 0}}' --prompt "Once upon a time," --device mps

yielded 39 tokens/sec before and 41 tokens/sec after

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Meta Open Source bot. label May 8, 2024
@malfet malfet changed the title Update PyTorch pin Update PyTorch pin to .dev20240507 May 8, 2024
@malfet malfet requested a review from mikekgfb May 8, 2024 17:43
qops.py Show resolved Hide resolved
@malfet malfet changed the title Update PyTorch pin to .dev20240507 Update PyTorch pin and enable MPS qops May 8, 2024
install_requirements.sh Outdated Show resolved Hide resolved
malfet and others added 2 commits May 8, 2024 17:26
And enable linter:int8 and linter:int4 acceleration on MPS
@mikekgfb mikekgfb merged commit 8a59fd3 into main May 9, 2024
36 checks passed
@mikekgfb mikekgfb deleted the malfet/udpate-torch-pin branch May 9, 2024 08:35
@malfet
Copy link
Contributor Author

malfet commented May 9, 2024

Thank you for merging, but it would be good to preserve PR description in the commit message rather than just a commit history.
This PR contains a tentative fix for #728 by uninstalling triton

malfet added a commit that referenced this pull request Jul 17, 2024
* Update PyTorch pin

And enable linter:int8 and linter:int4 acceleration on MPS

* Update run-readme-pr.yml

* Update install_requirements.sh
malfet added a commit that referenced this pull request Jul 17, 2024
* Update PyTorch pin

And enable linter:int8 and linter:int4 acceleration on MPS

* Update run-readme-pr.yml

* Update install_requirements.sh
malfet added a commit that referenced this pull request Jul 17, 2024
* Update PyTorch pin

And enable linter:int8 and linter:int4 acceleration on MPS

* Update run-readme-pr.yml

* Update install_requirements.sh
malfet added a commit that referenced this pull request Jul 17, 2024
* Update PyTorch pin

And enable linter:int8 and linter:int4 acceleration on MPS

* Update run-readme-pr.yml

* Update install_requirements.sh
malfet added a commit that referenced this pull request Jul 17, 2024
* Update PyTorch pin

And enable linter:int8 and linter:int4 acceleration on MPS

* Update run-readme-pr.yml

* Update install_requirements.sh
malfet added a commit that referenced this pull request Jul 17, 2024
* Update PyTorch pin

And enable linter:int8 and linter:int4 acceleration on MPS

* Update run-readme-pr.yml

* Update install_requirements.sh
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
CLA Signed This label is managed by the Meta Open Source bot.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants