We read every piece of feedback, and take your input very seriously.
To see all available qualifiers, see our documentation.
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
The testing framework should compare runtime for each test from candidate to reference and flag significant* differences as a regression.
*"significant" is in the eye of the beholder
If a candidate test runs more than xx% slower than the reference test, it should be labeled as a regression
No differences in test runtimes are noted or flagged
For each candidate test run by the testing framework, a runtime should be noted and compared against the runtime.
The text was updated successfully, but these errors were encountered:
No branches or pull requests
The testing framework should compare runtime for each test from candidate to reference and flag significant* differences as a regression.
*"significant" is in the eye of the beholder
Expected Behavior
If a candidate test runs more than xx% slower than the reference test, it should be labeled as a regression
Current Behavior
No differences in test runtimes are noted or flagged
Possible Solution
For each candidate test run by the testing framework, a runtime should be noted and compared against the runtime.
The text was updated successfully, but these errors were encountered: