Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Comparison View without R #141

Merged
merged 233 commits into from
Jul 12, 2023
Merged

Comparison View without R #141

merged 233 commits into from
Jul 12, 2023

Conversation

smarr
Copy link
Owner

@smarr smarr commented May 6, 2023

This PR aims to remove the use of R to generate the comparison view.

Key elements for this are:

  • compute statistics in pure TypeScript/JavaScript
  • render plots, currently with Chart.js
  • recreate the whole report based on a modular and testable structure of partial templates using EJS

Various Little Bits

  • avoid render time difference as -0 and -0.00, it just looks odd
  • make sure the overview plots are shown (path issue)
  • generate the inline plot per benchmark
  • rework code to calculate statistics, needed to combine and cleanup all the different versions
  • make sure we have a button for the warmup plot
  • add tests for the conversion between data formats, perhaps remove the need for the conversion
  • add support for indicating missing data, and what is missing
  • implement on-demand warmup plot backend and frontend based on uPlot
  • render warmup plots with uPlot, needs to show compilation time and GC time, as before
  • sort benchmarks by name in the table, and then by command line a second level criterion
  • overview stats seem possibly buggy with 0.0 values and NaN on http://localhost:33333/TruffleRuby/compare-new/e348f68100ff857dbc773771976af616d8957027..6f0ce7210ccdd42d1f890dce46c074a3c950c624
  • make sure the microbenchmarks, which differ in arguments are rendered correctly. Example: http://localhost:33333/TruffleRuby/compare-new/e348f68100ff857dbc773771976af616d8957027..6f0ce7210ccdd42d1f890dce46c074a3c950c624#ruby-micros-TruffleRuby-native-interp
  • make sure we have access to profile information
  • make sure we have all features, or tracked issues
  • make sure to use the same font in CI to make image diffs give useful results on github actions
  • add support for comparing across executors
  • restructure the code, to have a useful structure that helps dealing with the now larger codebase
  • cleanup and simplify code and types

smarr added 22 commits July 11, 2023 16:02
Signed-off-by: Stefan Marr <[email protected]>
This is a simplified part from the R we had.

Signed-off-by: Stefan Marr <[email protected]>
- and add tests

Signed-off-by: Stefan Marr <[email protected]>
Signed-off-by: Stefan Marr <[email protected]>
Signed-off-by: Stefan Marr <[email protected]>
Signed-off-by: Stefan Marr <[email protected]>
@smarr
Copy link
Owner Author

smarr commented Jul 12, 2023

Performance results don't show any difference since I haven't switched things over to the new comparison view for the benchmarks.

https://rebench.dev/ReBenchDB/compare/8c3573de3f577ca773954bee7656a85488c861a0..17149607b27ab21f576ed9f71a9df8b3bb100df7

@smarr smarr merged commit 1216682 into master Jul 12, 2023
2 checks passed
@smarr smarr deleted the compare-without-R branch July 12, 2023 08:48
@smarr smarr linked an issue Jul 12, 2023 that may be closed by this pull request
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Development

Successfully merging this pull request may close these issues.

Profiling info: add baseline/change labels to the profiles
1 participant