Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Major cleanup #197

Merged
merged 12 commits into from
Jul 19, 2024
Merged

Major cleanup #197

merged 12 commits into from
Jul 19, 2024

Conversation

jeremyfowers
Copy link
Collaborator

@jeremyfowers jeremyfowers commented Jul 18, 2024

Closes #196 #193 #183 #182

Summary

  • Complete rework of how cached builds are accessed
    • All cache loads are explicitly accessed by the new load-build Tool and turnkeyml.state.load_state() API
      • The load-build tool takes a BUILD_NAME_state.yaml file as input, loads the state of that build, a passes it to the next Tool in the sequence.
      • Provides a --skip-policy argument to make it easy to manage evaluation of large amounts of builds, similar to the skip policy in the old turnkey cache benchmark.
    • The ignition module has been removed
    • The --rebuild CLI argument has been removed
    • The turnkey cache benchmark Tool has been removed
      • Benchmark a pre-built model with turnkey -i path/to/state.yaml load-build benchmark
  • The build_model() API has been removed
  • The benchmark_files() API has been renamed to evaluate_files()
    • Function body has been simplified a lot
    • Only Sequence and Tools should write to stats. All code in the evaluate_files() body has been moved under Sequence.launch().
    • Sequence.launch() does everything needed to manage the sequence, such as launching the monitor. This used to be the responsibility of evaluate_files().
  • A new class, FirstTool, has been added to provide convenience and guardrails for Tools that are meant to start a sequence, such as load-build, load-onnx, and discover.
  • Special behavior for the exp.SkipBuild exception: any changes to stats are rewound to before that build was loaded, such that skipping a build does not perturb the stats in any way.

@jeremyfowers jeremyfowers self-assigned this Jul 18, 2024
@jeremyfowers jeremyfowers marked this pull request as ready for review July 18, 2024 14:54
Copy link
Collaborator

@danielholanda danielholanda left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good and works well

src/turnkeyml/cli/cli.py Show resolved Hide resolved
src/turnkeyml/common/filesystem.py Outdated Show resolved Hide resolved
src/turnkeyml/sequence/sequence.py Outdated Show resolved Hide resolved
Copy link
Collaborator

@ramkrishna2910 ramkrishna2910 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks really good! I was able test locally on my system.
Added a minor comment.

src/turnkeyml/cli/cli.py Show resolved Hide resolved
Co-authored-by: Daniel Holanda <[email protected]>
Signed-off-by: Jeremy Fowers <[email protected]>
@jeremyfowers jeremyfowers enabled auto-merge (squash) July 19, 2024 13:58
@jeremyfowers jeremyfowers enabled auto-merge (squash) July 19, 2024 14:01
@jeremyfowers jeremyfowers merged commit 4313a20 into refresh Jul 19, 2024
6 checks passed
@jeremyfowers jeremyfowers deleted the 183-add-an-entry-stage-class branch July 19, 2024 14:12
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants