Fix runtime environment (scheduler, validation, etc.) #5570
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
hjoliver
force-pushed
the
fix-scheduler-env
branch
2 times, most recently
from
2429a07
to
86cecb0
Compare
8 tasks
|
Thanks for opening this! I have one of the examples of this use case. In my case - the Git repository containing Cylc flow also contains a folder of model parameter files. I want to run some Cylc tasks across the sites/scenarios specified in the folder without having to update the Jinga2 list each time another site/scenario is added! |
8 tasks
hjoliver
changed the title
hjoliver
force-pushed
the
fix-scheduler-env
branch
2 times, most recently
from
d6979b7
to
cb4928e
Compare
hjoliver
force-pushed
the
fix-scheduler-env
branch
from
cb4928e
to
fd57352
Compare
|
This is ready to go. @oliver-sanders and @wxtim - plz review or reassign at your end. |
hjoliver
force-pushed
the
fix-scheduler-env
branch
from
fd57352
to
7a1aea6
Compare
hjoliver
force-pushed
the
fix-scheduler-env
branch
3 times, most recently
from
e24d07e
to
c43197c
Compare
|
(Apologies reviewers - some stray debug code had broken the tests at the last minute ... should be fixed now!) |
wxtim
approved these changes
Jun 23, 2023
There is no run directory for a source workflow...
Add basic config doc string; Clarify a comment; Tweak installed check. Co-authored-by: Tim Pillinger <[email protected]>
hjoliver
force-pushed
the
fix-scheduler-env
branch
from
23cd18b
to
2bf8bd3
Compare
Co-authored-by: Oliver Sanders <[email protected]>
|
Changes still look good. |
oliver-sanders
approved these changes
Jun 28, 2023
wxtim
added a commit
to wxtim/cylc
that referenced
this pull request
Jun 29, 2023
…c into feature.plus_s_list_of_strings * 'feature.plus_s_list_of_strings' of github.com:wxtim/cylc: Update tests/unit/test_templatevars.py Fix runtime environment (scheduler, validation, etc.) (cylc#5570) Bump pypa/gh-action-pypi-publish from 1.8.6 to 1.8.7 Fix simulation mode TypeError prevent tests symlinking subprocctx: fix xtrigger __str__ xtriggers: improve parser
wxtim
pushed a commit
to wxtim/cylc
that referenced
this pull request
Jun 29, 2023
Fix workflow environment variables when in source directory
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently in
config.pywe infer a run directory, and work and share sub-directories, and export associated environment variables, even when parsing a source workflow (which may not be installed) or when parsing an installed workflow but not running the scheduler (in which case work etc. may not exist).The original motivation for this PR was:
There are use cases where src-dir needs to be distinguished from run-dir at config parsing time. Example: dependency graph constructed on the fly according to the content of some workflow sub-dir. For validation of the source workflow, the location of this content is relative to the source dir; for validation of the installed workflow (or at scheduler start-up) it must be relative to the run directory.
#5589 now provides a better solution to that problem, but in any case the exported environment should be consistent with the workflow location (as it is accessible to the template processor during parsing).
Check List
CONTRIBUTING.mdand added my name as a Code Contributor.Applied any dependency changes to bothsetup.cfg(andconda-environment.ymlif present).CHANGES.mdentry included if this is a change that can affect usersIf this is a bug fix, PR should be raised against the relevant. (we're currently aiming all changes at 8.2.0)?.?.xbranch