Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat(eager): Eager & Execute for easier debugging #59

Open
wants to merge 4 commits into
base: release/0.11.0
Choose a base branch
from

Conversation

philtweir
Copy link
Contributor

@philtweir philtweir commented Oct 18, 2024

✨ Feature: Eager & Execute for easier debugging

Description

Provides ability to eagerly evaluate or execute a lazy evaluation.

Eager: that is, never to use laziness at all and the task/workflow decorator calls the wrapped function immediately.
Execute: once a lazy evaluation exists (e.g. workflow result) calling execute gets a value from it, without using the backend (i.e. by calling all the functions).

The difference between execute and normal dask evaluation is that dewret does not use an executor, and simply calls the functions recursively, which may simplify debugging.

Changes Made

  • Command-line and construct-time flag for --eager to get an immediate result printed to stdout.
  • Addition of execute to workflow, that will call any tasks to get a final result after the workflow/task is constructed.
  • test_executions.py: script to confirm the new functionality

Additional Information

Requested given the nonlinear nature of debugging. This has been confirmed to allow step-wise debugging in PyCharm, by adding --eager into the command line.

Tip: adding DEFAULT_WORKFLOW = workflow_to_test to the bottom of a workflow file, and making the command-line for debugging to be -m dewret $FilePath$ DEFAULT_WORKFLOW --eager will save having to change the configuration to swap between debugged workflows.

Checklist

  • I have included a detailed description of the feature.
  • I have provided relevant documentation for my feature.
  • I have added the necessary tests.

Who can review?

@KamenDimitrov97 @elleryames (written by @philtweir and @edufnt )

@philtweir philtweir changed the title feat(eager): add eager evaluation and immediate task evaluation/execu… feat(eager): Eager & Execute for easier debugging Oct 18, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant