Skip to content

Commit

Permalink
Add benchmarking RFC
Browse files Browse the repository at this point in the history
  • Loading branch information
ddunl committed Dec 12, 2023
1 parent 50f8f3f commit 9b54a99
Showing 1 changed file with 22 additions and 0 deletions.
22 changes: 22 additions & 0 deletions rfcs/20231212-benchmarking-rfc.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
# OpenXLA Benchmarking
| Status | Proposed |
| :------------ | :------------------------------------------------------ |
| **RFC #** | [101](https://github.com/openxla/community/pull/101) |
| **Authors** | David Dunleavy ([email protected]) |
| **Sponsor** | Puneith Kaul ([email protected]) |
| **Updated** | 2023-12-12

## Objective
Gather feedback on what benchmarking infrastructure would be helpful for the OpenXLA community.

## Proposal
Previous benchmarking efforts like those described in the
[Benchmarking Strategy RFC](https://github.com/openxla/community/blob/main/rfcs/20230505-benchmarking-strategy.md)
have been disrupted due to changes in broader strategy in the OpenXLA ecosystem.
Since then, there have been some independent discussions about benchmarking for XLA,
but we are interested in getting a pulse on what's most important to the community.

## Questions
What openly available models are most interesting to measure?
Is using models in HLO/StableHLO form sufficient? Or is it preferable to do framework level benchmarks?
What backends/targets/configurations are most important to benchmark? Single vs multi-host?

0 comments on commit 9b54a99

Please sign in to comment.