Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Get info about workflows and pipelines from db instead of hard coding #577

Merged
merged 12 commits into from
Aug 22, 2024

Conversation

aschroed
Copy link
Member

@aschroed aschroed commented Aug 5, 2024

Needs careful review and testing

  • Refactor of wfr_utils to utilize properties and tags on updated schemas for workflow and experiment_type to generate info about accepted versions of workflows and pipelines
  • Updated wfr_checks and wfr_encoded_checks to utilize the new info from db
  • provided additional keyword args to override the accepted versions (useful for testing or some cases where re-runs of updated pipelines are needed)
  • Fix bug that allowed more than 2 errored fastq_first_line runs happen

assert wfr_name in workflow_details
assert key
workflow_details = get_workflow_details(key)
assert wfr_name in workflow_details
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ran into an error at this point in a check running--it looks like workflow_details is just {}. I will keep exploring!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This makes sense as the properties that were added to the Workflow and ExperimentType items are not in webdev and I guess the version of fourfront there is likely prior to the schema changes so they can't be added. Note that for testing particular workflows you can pass in this info as a parameter but that kind of defeats some of the purpose of testing. I will consider options for making this backward compatible so it can be used on webdev without errors.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Oh, of course!

@clarabakker
Copy link
Member

Okay, I managed to get a QC workflow to run and patch on webdev, as well as a processing workflow using the stepper function. It was a little tricky to get working on webdev for the processing pipeline, but it definitely works.

I guess I'll turn the review back to you for the addition to prevent fastq-first-line from running over 2 errors and to bump versions.

@aschroed aschroed changed the title DRAFT - do not merge until tested Get info about workflows and pipelines from db instead of hard coding Get info about workflows and pipelines from db instead of hard coding Aug 22, 2024
@aschroed aschroed merged commit 878ad79 into master Aug 22, 2024
1 check failed
@aschroed aschroed deleted the ajs_wf_vers_upd branch August 22, 2024 16:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants