Sbachmei/mic 5476/add optional schema arg to config constructor #127
Conversation
src/easylink/pipeline_graph.py
Outdated
| subgraph = ImplementationGraph(self).subgraph(nodes) | ||
| # NOTE: The subgraph is not necessarily able to be topologically sorted |
There was a problem hiding this comment.
@patricktnast Does this note jive with your understanding? We should talk about this. I'm confused why an nx graph isn't just always sorted. And, related, I'm confused why I am unable to access the is_sorted attr sometimes (it's not False, it's actually not set)
| from easylink.step import HierarchicalStep, InputStep, LoopStep, OutputStep, Step | ||
| from easylink.utilities.validation_utils import validate_input_file_dummy | ||
|
|
||
| SINGLE_STEP_NODES = [ |
There was a problem hiding this comment.
SINGLE_STEP_NODES, _EDGES, and _SCHEMA_PARAMS were oved from old pipeline_schema_constants/integration_test.py (the file deleted above). They are identical except for the names.
| SINGLE_STEP_SCHEMA_PARAMS = (SINGLE_STEP_NODES, SINGLE_STEP_EDGES) | ||
|
|
||
|
|
||
| BAD_COMBINED_TOPOLOGY_NODES = [ |
There was a problem hiding this comment.
everything below here is new
| @@ -575,40 +578,53 @@ def test_merge_combined_implementations_parallel(default_config_params, test_dir | |||
| check_nodes_and_edges(pipeline_graph, expected_nodes, expected_edges) | |||
|
|
|||
|
|
|||
| def test_cycle_error(default_config_params) -> None: | |||
There was a problem hiding this comment.
These three tests are identical and the two new ones are as well aside from needing to pass in a custom PipelineSchema - hence the refactor to a single parametrized test.
stevebachmeier
requested review from
aflaxman,
albrja,
hussain-jafari,
patricktnast,
rmudambi and
zmbc
as code owners
| env: | ||
| INPUT_ENV_VARS: "DUMMY_CONTAINER_MAIN_INPUT_FILE_PATHS,DUMMY_CONTAINER_SECONDARY_INPUT_FILE_PATHS" | ||
| outputs: | ||
| step_4b_main_output: result.parquet | ||
| step_4b_r: | ||
| steps: | ||
| - step_4b | ||
| image_path: /mnt/team/simulation_science/priv/engineering/er_ecosystem/images/python_pandas.sif | ||
| script_cmd: python /dummy_step.py | ||
| image_path: /mnt/team/simulation_science/priv/engineering/er_ecosystem/images/r-image.sif |
Yeah I agree. I'll think about creating different metadata files at some point. But I'm not even sure this will end up being the actual container for "real" metadata anyway, right? |
stevebachmeier
deleted the
sbachmei/mic-5476/add-schema-arg-to-config
branch
Add optional
schemaarg to Config constructorDescription
Changes and notes
This implements a new and optional arg to the Config constructor
that allows us to pass in an on-the-fly allowable PipelineSchema.
This is primarily useful for testing purposes.
This unlocked for two new tests along the way.
Verification and Testing
slow tests pass