Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Unify scaled INT8 mm #878

Closed
wants to merge 8 commits into from
Closed

Unify scaled INT8 mm #878

wants to merge 8 commits into from

Conversation

gau-nernst
Copy link
Collaborator

Fixes #862

@pytorch-bot PyTorch Bot
Copy link

pytorch-bot bot commented Sep 12, 2024 •
edited
Loading

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/ao/878

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 1564c4c with merge base 8236a87 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 12, 2024
gau-nernst
gau-nernst commented Sep 12, 2024
View reviewed changes
test/kernel/test_autotuner.py
Comment on lines -52 to -60
@parameterized.expand(
[
("cuda", torch.bfloat16),
("cpu", torch.bfloat16),
("cuda", torch.float16),
("cpu", torch.float16),
]
)
def test_int_scaled_mm(self, device, dtype):
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dtype is always overriden to BF16 later

@gau-nernst
Copy link
Collaborator Author

The triton kernel in torchao.kernel.intmm_triton is highly coupled to torchao.kernel.autotuner, which makes it unnecessary complex to separate them out. intmm_triton also has inference in mind, with exhaustive autotuning and cached config, which is not suitable for training. Closing this for now.

@gau-nernst gau-nernst closed this Sep 16, 2024
@gau-nernst gau-nernst deleted the int8_mm branch September 16, 2024 01:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Unify scaled INT8 matmul
2 participants
@gau-nernst @facebook-github-bot