Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Punet IREE Testing #385

Open
saienduri opened this issue Nov 4, 2024 · 0 comments
Open

Punet IREE Testing #385

saienduri opened this issue Nov 4, 2024 · 0 comments
Assignees

Comments

@saienduri
Copy link
Contributor

We currently https://github.com/iree-org/iree/tree/main/experimental/regression_suite/shark-test-suite-models/sdxl and https://github.com/iree-org/iree/tree/main/experimental/benchmarks/sdxl in IREE project for testing sdxl on every commit for fp16 model and int8 model with fp16/fp8 attention. We are currently not testing the spec file (due to compiler productization discussions), so the performance testing doesn't fully correlate with what we track in the MLPerf CI. Also, currently, there is no performance testing for the int8 punet model.

Next steps would be to add both of these as a not required to pass presubmit item in IREE CI, so it is easy for Nithin to track and days are not wasted chasing down regressions. Eventual goal is to completely migrate from using spec files and have it baked into the compiler itself.

@saienduri saienduri self-assigned this Nov 4, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant