You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Initially I thought it would be great to include a date in the run log directory structure, but it's become a huge pain, and makes it very difficult to use checkpointing.
Remove the date component of the directory structure, and determine what the correct structure should be in order to use checkpointing correctly across multiple sets of staged files.
I.e. staged files in dir A should have checkpoints that don't interfere with staged files in dir B checkpoints.
The text was updated successfully, but these errors were encountered:
Initially I thought it would be great to include a date in the run log directory structure, but it's become a huge pain, and makes it very difficult to use checkpointing.
Remove the date component of the directory structure, and determine what the correct structure should be in order to use checkpointing correctly across multiple sets of staged files.
I.e. staged files in dir A should have checkpoints that don't interfere with staged files in dir B checkpoints.
The text was updated successfully, but these errors were encountered: