Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: benchmark for modulo on bv_decide #5653

Merged
merged 1 commit into from
Oct 8, 2024
Merged

Conversation

hargoniX
Copy link
Contributor

@hargoniX hargoniX commented Oct 8, 2024

This verifies a bit hack from here: https://en.wikipedia.org/wiki/Lehmer_random_number_generator#Sample_C99_code

I previously ran the SMTLIB equivalent this with Bitwuzla in my crypto class and got the following numbers:

  • 22s with Bitwuzla
  • Z3 and CVC5 don't yet terminate after > 2min

Now withbv_decide the overall timing is 33.7s, consisting of:

  • 5s of checking the LRAT cert
  • 5s of trimming the LRAT cert from 800k to 300k proof steps
  • remainder actual solving time

So running bv_decide like a normal SMT solver without verifying the result of the SAT solver would yield approximately ~24s.

@hargoniX
Copy link
Contributor Author

hargoniX commented Oct 8, 2024

!bench

@github-actions github-actions bot temporarily deployed to lean-lang.org/lean4/doc October 8, 2024 14:57 Inactive
@github-actions github-actions bot added the toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN label Oct 8, 2024
@leanprover-community-bot
Copy link
Collaborator

Mathlib CI status (docs):

  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase abae95e1701a081701ef132a35b3b409594e3907 --onto 3e75d8f74297bc258352f8d89f71496aacefe7ae. (2024-10-08 15:17:49)

@leanprover-bot
Copy link
Collaborator

Here are the benchmark results for commit b2366db.
There were significant changes against commit abae95e:

  Benchmark             Metric   Change
  ================================================
+ bv_decide_realworld   maxrss    -1.2% (-225.1 σ)

Copy link
Contributor

@tobiasgrosser tobiasgrosser left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice.

tests/bench/bv_decide_mod.lean Show resolved Hide resolved
@hargoniX hargoniX added this pull request to the merge queue Oct 8, 2024
Merged via the queue into master with commit 248864c Oct 8, 2024
15 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants