-
Notifications
You must be signed in to change notification settings - Fork 67
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
standardize taskgraph cache name and location for all models #1380
Conversation
HISTORY.rst
Outdated
all models. It is now called ``taskgraph_cache`` and located in the top | ||
level of the workspace directory. | ||
(`#1230 <https://github.com/natcap/invest/issues/1230>`_) | ||
* Coastal Blue Carbon |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Not sure why this is showing up in the diff - it was already in main.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks for doing this @emlys , a nice quality of life improvement. Just a few quick questions.
|
||
suffix = utils.make_suffix_string(args, 'results_suffix') | ||
intermediate_dir = os.path.join( | ||
args['workspace_dir'], INTERMEDIATE_DIR_NAME) | ||
output_dir = os.path.join( | ||
args['workspace_dir'], OUTPUT_DIR_NAME) | ||
|
||
utils.make_directories([output_dir, intermediate_dir, taskgraph_cache_dir]) | ||
utils.make_directories([output_dir, intermediate_dir]) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
TaskGraph creates the taskgraph cache dir if it doesn't exist already, right?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Correct!
'outflow_direction_path': 'outflow_direction.tif', | ||
'outflow_weights_path': 'outflow_weights.tif', | ||
'kc_path': 'kc.tif', |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Were these deletions not being used?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Yes, they were unused
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks Emily!
Description
Fixes #1230
I went with
taskgraph_cache
for the name of the directory. I put it at the top level in the workspace because not all models have an intermediate directory.A few models used a cache directory that contained the
taskgraph.db
file as well as some other less-important intermediate outputs. I split these up so the taskgraph cache has its own directory and the intermediate outputs go in theintermediate
directory.Added output specs for SWY cache outputs that were missing them.
Checklist