From 2d98a45fc96ec2424a2b03de839183fc5e109386 Mon Sep 17 00:00:00 2001 From: Oliver Sanders Date: Thu, 27 Jul 2023 11:30:50 +0100 Subject: [PATCH 1/2] doc tweeks --- src/7-to-8/major-changes/cylc-install.rst | 25 +++++++++++++++-------- src/7-to-8/summary.rst | 23 +++++++++++++-------- 2 files changed, 31 insertions(+), 17 deletions(-) diff --git a/src/7-to-8/major-changes/cylc-install.rst b/src/7-to-8/major-changes/cylc-install.rst index f02500b67d..28cb5170ff 100644 --- a/src/7-to-8/major-changes/cylc-install.rst +++ b/src/7-to-8/major-changes/cylc-install.rst @@ -118,21 +118,30 @@ See :ref:`the user guide ` for more details. Migrating From ``rose suite-run`` --------------------------------- -The ``rose suite-run`` command has been replaced by ``cylc install``. +The ``rose suite-run`` command did three things: -.. code-block:: bash +1. It validated the workflow. +2. It installed the workflow into the ``cylc-run`` directory. +3. It started a scheduler to run the workflow. - # rose 2019 / Cylc 7 - $ rose suite-run +At Cylc 8, there is a separate command for each of these three things: - # rose 2 / Cylc 8 +.. code-block:: console + + $ cylc validate $ cylc install - $ cylc play + $ cylc play + +But for convenience there is a compound command which does all three: + +.. code-block:: console + + $ cylc vip # vip=validate-install-play -Support for the ``rose-suite.conf`` file is provided by the :ref:`Cylc Rose` +Support for the ``rose-suite.conf`` file is now provided by the :ref:`Cylc Rose` plugin which must be installed for Rose integration. -.. spoiler:: Installation +.. spoiler:: Installation Details :class: hint See the :ref:`installation` section for instructions. diff --git a/src/7-to-8/summary.rst b/src/7-to-8/summary.rst index 342e702372..02a34cce05 100644 --- a/src/7-to-8/summary.rst +++ b/src/7-to-8/summary.rst @@ -39,9 +39,11 @@ Upgrading To Cylc 8 There have been some configuration changes at Cylc 8. To upgrade your Cylc 7 suite to a Cylc 8 workflow, first make sure it validates -in Cylc 7 without any warnings, then rename the workflow configuration file -from ``suite.rc`` to ``flow.cylc``, then run ``cylc validate`` in Cylc 8 and -take action on any warnings. +in Cylc 7 without any warnings, you can now run it in compatibility mode. + +To upgrade the workflow to run without compatibility mode, rename the workflow +configuration file from ``suite.rc`` to ``flow.cylc``, run ``cylc validate`` +in Cylc 8 and take action on any warnings. .. TODO Add ref to breaking changes section within Major changes, once created, including optional ouputs. @@ -71,12 +73,14 @@ At Cylc 8, there are two UIs available to monitor and control your workflows: .. code-block:: bash - cylc gui + cylc gui Command Changes --------------- -``cylc run `` at Cylc 7 has become ``cylc play ``. +``cylc run `` has become ``cylc play ``. + +``rose suite-run`` is now ``cylc vip`` (**V**\ alidate, **I**\ nstall, **P**\ lay). .. seealso:: @@ -84,8 +88,8 @@ Command Changes * Major Changes: :ref:`728.play_pause_stop` * Major Changes: :ref:`MajorChangesCLI` -At Cylc 8, use ``cylc pause `` to pause a workflow, halting all job -submission. To restart the workflow, use ``cylc play ``. +At Cylc 8, use ``cylc pause `` to pause a workflow, preventing new +job submissions. To restart the workflow, use ``cylc play ``. To start a fresh run, use ``cylc install`` and play it safely in the new run directory. @@ -96,12 +100,13 @@ Task/Job States --------------- :term:`Tasks ` are nodes in the abstract workflow graph, representing -applications to run at the appropriate point in the workflow. A :term:`job ` +applications to run at the appropriate point in the workflow. + +A :term:`job ` is the script (and subsequent process) submitted by Cylc to actually run the application. A task can have multiple jobs as the result of automatic retries or manual re-triggering. - The 13 task/job states in Cylc 7 have been simplified to 8. Tasks and jobs have been separated and states of both can be viewed in the GUI. From 07a7ac41f07f89d903e96ab69ac4d929996b7ce6 Mon Sep 17 00:00:00 2001 From: Oliver Sanders Date: Thu, 27 Jul 2023 16:36:54 +0100 Subject: [PATCH 2/2] optional output extension **Sibling:** https://github.com/cylc/cylc-flow/pull/5651 Document the changes outlined in https://cylc.github.io/cylc-admin/proposal-optional-output-extension.html --- src/7-to-8/major-changes/suicide-triggers.rst | 3 + src/glossary.rst | 9 +- src/reference/changes.rst | 116 +++- .../writing-workflows/scheduling.rst | 511 +++++++++++++----- 4 files changed, 486 insertions(+), 153 deletions(-) diff --git a/src/7-to-8/major-changes/suicide-triggers.rst b/src/7-to-8/major-changes/suicide-triggers.rst index c24e925b4c..f15cd3c0ef 100644 --- a/src/7-to-8/major-changes/suicide-triggers.rst +++ b/src/7-to-8/major-changes/suicide-triggers.rst @@ -91,6 +91,9 @@ Also, the suicide triggers can be removed. - # Remove the "foo" task in the fail case. - recover => ! foo +At least one optional output must be generated, so in this example ``foo`` must +either succeed or fail. + In Cylc 7, suicide triggers were used to remove tasks that did not complete during runtime. Cylc 8's event-driven graph handling allows such graph branching using optional output syntax, without the need for suicide triggers. diff --git a/src/glossary.rst b/src/glossary.rst index 4affe15b47..c826c8b21f 100644 --- a/src/glossary.rst +++ b/src/glossary.rst @@ -1479,8 +1479,12 @@ Glossary optional output Optional :term:`task outputs ` are marked with a question mark in the :term:`graph`, e.g. ``foo:x?``, or ``foo:fail?``, or - ``foo?`` (short for ``foo:succeed?``). The may or may not be completed at - runtime. Optional outputs are primarily used for :term:`graph branching`. + ``foo?`` (short for ``foo:succeed?``). + + One or more optional outputs must be generated when the task runs, + otherwise the workflow will :term:`stall`. This allows the workflow + to follow different pathways through the graph according to runtime + events, this is called :term:`graph branching`. .. seealso:: @@ -1489,7 +1493,6 @@ Glossary required output - expected output Task outputs that are not marked as :term:`optional ` in the :term:`graph` are required to be completed at runtime. If not, the :term:`scheduler` retains the task as :term:`incomplete` pending user diff --git a/src/reference/changes.rst b/src/reference/changes.rst index 3c1fcb618c..5515b856c0 100644 --- a/src/reference/changes.rst +++ b/src/reference/changes.rst @@ -20,6 +20,112 @@ For more detail see the component changelogs: * `metomi-rose-changelog`_ * `metomi-isodatetime-changelog`_ +---------- + +Cylc 8.3.0 +---------- + +.. TODO - uncomment before 8.3.0 release + + .. admonition:: Cylc Components + :class: hint + + :cylc-flow: `8.3 `__ + :cylc-uiserver: `1.4 `__ + :cylc-rose: `1.4 `__ + +Optional Outputs +^^^^^^^^^^^^^^^^ + +Optional outputs (introduced in Cylc 8.0.0) are a way of validating the outcome +of a task to ensure it "completed" as intended. E.G, by default, if a task +fails, it will considered :term:`incomplete ` and the +workflow will stall. + +Before (Cylc 8.0.0 - 8.1.4) + All optional outputs were completely "optional", + that is to say, they may or may not be generated when a task runs. + +From Cylc 8.3.0 onwards + When optional outputs are defined in a workflow, Cylc + now requires one or more of them to be generated by the task at runtime. + +So for this example: + +.. code-block:: cylc-graph + + a:x => x + a:y => y + x | y => b + +The task must succeed (success is required unless marked otherwise) and at +least one of the outputs ``x`` or ``y`` must be generated. + +In Python syntax that condition looks like this: + +.. code-block:: python + + succeeded and (x or y) + +If this condition is not met, then the task will be considered +:term:`incomplete ` altering you to the problem. + +This change solves an issue where the pathway through the graph could be +interrupted. +E.G. in the above example, the intention is that Cylc will follow either the +``a:x`` branch *or* the ``a:y`` branch, however, with Cylc <=8.1.4 it is +possible that neither the ``x`` or ``y`` outputs would be generated, in which +case the task ``b`` would never be run. + +This behaviour can be overridden using the +:cylc:conf:`[runtime][]completion` configuration. +E.G. To restore the old behaviour + +.. code-block:: cylc + + [runtime] + [[a]] + # this sets the condition that Cylc uses to determine whether a + # task has completed + completion = succeeded or failed + +The new ``completion`` configuration opens up the potential for much +more powerful task diagnostics with :term:`custom outputs `. + +See :ref:`Graph Branching` for more information about optional outputs +and examples including the new ``completion`` configuration. + + +Task Expiry +^^^^^^^^^^^ + +As a result of the optional outputs change above, task expiry can now be used +in combination with optional outputs e.g: + +.. code-block:: cylc-graph + + # run "x" if "a" succeeds + a? => x + # run "y" if "a" expires + a:expired? => y + +In this example, ``a:fail?`` is not handled in the graph, so if ``a`` fails, +it will be incomplete and the workflow will stall altering you to the problem. + +This change means you need to add a ``?`` mark symbol after ``:expired`` +triggers in the graph. + +The new ``completion`` configuration can also be used in combination with +task expiry e.g: + +.. code-block:: cylc + + [runtime] + [[a]] + # this task is allowed to succeed or expire but not fail + completion = succeeded or expired + + ---------- Cylc 8.2.0 @@ -28,9 +134,9 @@ Cylc 8.2.0 .. admonition:: Cylc Components :class: hint - :cylc-flow: `8.2 `__ - :cylc-uiserver: `1.3 `__ - :cylc-rose: `1.3 `__ + :cylc-flow: `8.2 `__ + :cylc-uiserver: `1.3 `__ + :cylc-rose: `1.3 `__ Configure The Default View @@ -126,11 +232,11 @@ Combined Commands Two new commands have been added as short-cuts for common working patterns: -``cylc vip`` +``cylc vip`` Validate, install and plays a workflow, equivalent to: .. code-block:: bash - + cylc validate cylc install cylc play diff --git a/src/user-guide/writing-workflows/scheduling.rst b/src/user-guide/writing-workflows/scheduling.rst index f8f018587e..04abe24d89 100644 --- a/src/user-guide/writing-workflows/scheduling.rst +++ b/src/user-guide/writing-workflows/scheduling.rst @@ -1537,8 +1537,12 @@ Required Outputs :term:`required ` (the default) or :term:`optional `. -The scheduler requires all task outputs to be completed at runtime, unless they -are marked with ``?`` as optional. This allows it to correctly diagnose +By default, all task outputs referenced in the graph are *required*, meaning +that they must be generated generated at runtime. +Outputs can be marked as *optional* using the ``?`` character. + +Required and optional outputs allow Cylc to understand when tasks have +successfully completed which is used to diagnose :term:`workflow completion`. [2]_ Tasks that finish without completing required outputs [3]_ are retained as @@ -1546,8 +1550,9 @@ Tasks that finish without completing required outputs [3]_ are retained as retriggered after a bug fix. .. note:: - Incomplete tasks stall the workflow if there is nothing else to do (see - :ref:`workflow completion`). They also count toward the :term:`runahead + + Incomplete tasks :term:`stall` the workflow if there is nothing else to do + (see :ref:`workflow completion`). They also count toward the :term:`runahead limit`, because they may run again once dealt with. This graph says task ``bar`` should trigger if ``foo`` succeeds: @@ -1558,19 +1563,11 @@ This graph says task ``bar`` should trigger if ``foo`` succeeds: Additionally, ``foo`` is required to succeed, because its success is not marked as optional. If ``foo`` does not succeeded, the scheduler will not run ``bar``, -and ``foo`` will be retained as an incomplete task. - -Here, ``foo:succeed``, ``bar:x``, and ``baz:fail`` are all required outputs: - -.. code-block:: cylc-graph - - foo - bar:x - baz:fail +``foo`` will be retained as an incomplete task and the workflow will stall. Tasks that appear with only custom outputs in the graph are also required to succeed. -Here, ``foo:succeed`` is a required output, as well as ``foo:x``, unless it is -marked as optional elsewhere in the graph: +Here, ``foo:succeed`` is a required output, as well as ``foo:x``, unless +``foo:succced`` is marked as optional elsewhere in the graph: .. code-block:: cylc-graph @@ -1595,36 +1592,82 @@ Optional Outputs .. versionadded:: 8.0.0 + Optional outputs remove the need for + :term:`suicide triggers `. + +.. versionchanged:: 8.3.0 + + Before 8.3.0, Cylc would allow no optional outputs to be generated. + From 8.3.0 onwards, Cylc requires one or more optional outputs to be + generated by default, this can be overridden using + :cylc:conf:`[runtime][]completion`. + Optional outputs are marked with ``?``. They may or may not be completed by the task at runtime. -Like the first example above, the following graph says task ``bar`` should -trigger if ``foo`` succeeds: +In this example, Cylc will follow one of two paths depending on whether the +task ``foo`` succeeds or fails: -.. code-block:: cylc-graph +.. list-table:: + :class: grid-table - foo? => bar # short for "foo:succeed? => bar" + - + - | -But now ``foo:succeed`` is optional, so we might expect it to fail sometimes. -And if it does fail, it will not be marked as an incomplete task. + .. code-block:: cylc-graph -Here, ``foo:succeed``, ``bar:x``, and ``baz:fail`` are all optional outputs: + # the :success case + foo:succeeded? => bar + # the :fail case + foo:fail? => baz + # joining the two graphs back together + bar | baz => pub -.. code-block:: cylc-graph + - .. digraph:: example + :align: center - foo? - bar:x? - baz:fail? + subgraph cluster_success { + label = ":succeed" + color = "green" + fontcolor = "green" + style = "dashed" + bar + } -Success and failure (of the same task) are mutually exclusive, so they must -both be optional if one is optional, or if they both appear in the graph: + subgraph cluster_failure { + label = ":fail" + color = "red" + fontcolor = "red" + style = "dashed" -.. code-block:: cylc-graph + baz + } - foo? => bar - foo:fail? => baz + foo -> bar + foo -> baz + bar -> pub + baz -> pub +The outputs ``foo:succeeded`` and ``foo:failed`` are both optional in this +example because they are followed by the ``?`` character. + +This is an example of :term:`graph branching`. Whether a +particular branch is taken or not depends on which optional outputs are +completed at runtime. For more information see :ref:`Graph Branching`. + +Cylc requires one or more optional outputs to be generated, so this means +that ``foo`` must either succeed or fail (i.e. it cannot expire or fail to +submit). + +Optional outputs do not affect *triggering* (i.e. what tasks run and when). +They just tell the scheduler what to do with the task if it finishes without +completing the output providing a form of validation. + +.. note: + + Success and failure (of the same task) are mutually exclusive, so they must + both be optional if one is optional. .. warning:: @@ -1632,50 +1675,72 @@ both be optional if one is optional, or if they both appear in the graph: graph, to avoid ambiguity. -If a task generates multiple custom outputs, they should all be declared optional -if you do not expect them to be completed every time the task runs: +Custom Completion Conditions +^^^^^^^^^^^^^^^^^^^^^^^^^^^^ -.. code-block:: cylc-graph +By default, if :ref:`User Guide Optional Outputs` are defined in the graph, +then one or more of these outputs must also be generated in order for the task +to be completed. - # model:x, :y, and :z are all optional outputs: - model:x? => proc-x - model:y? => proc-y - model:z? => proc-z +In this example, ``a:succeeded`` is optional, however, ``a:fail`` does not +appear in the graph so only one optional output is defined for this task. -This is an example of :term:`graph branching` from optional outputs. Whether a -particular branch is taken or not depends on which optional outputs are -completed at runtime. For more information see :ref:`Graph Branching`. +.. code-block:: cylc + + [scheduling] + [[graph]] + # only run the task "b" if the task "a" succeeds + # (i.e. if the task "a" fails, do nothing) + R1 = a? => b -Leaf tasks (with nothing downstream of them) can have optional outputs. In the -following graph, ``foo`` is required to succeed, but it doesn't matter whether -``bar`` succeeds or fails: +Because one or more optional outputs must be completed, the workflow will +stall if ``a`` fails. The reason for this is that there are multiple reasons +why a task might not succeed: -.. code-block:: cylc-graph +* It might fail to submit (``a:submit-fail``). +* It could fail during execution (``a:fail``). +* It could be canceled as a tesult of :ref:`expiry ` (``a:expire``). - foo => bar? +To prevent the workflow stalling, you need to tell Cylc that it is ok for this +task to fail. You can either do that by handling failure in the graph (as in the example +above) e.g: +.. code-block:: cylc-graph -.. note:: + a? => b + a:fail? => c - Optional outputs do not affect *triggering*. They just tell the scheduler - what to do with the task if it finishes without completing the output. +Or, if you don't need to run any tasks in the event of failure you can use +:cylc:conf:`[runtime][]completion` to tell Cylc that failure is ok: - This graph triggers ``bar`` if ``foo`` succeeds, and does not trigger - ``bar`` if ``foo`` fails: +.. code-block:: cylc - .. code-block:: cylc-graph + [scheduling] + [[graph]] + R1 = a? => b + [runtime] + [[a]] + # this task may either succeed or fail + completion = succeeded or failed + [[b]] - foo => bar +This completion overrides the default completion condition and can be used +to configure more advanced validation checking e.g: - And so does this graph: +.. code-block:: python - .. code-block:: cylc-graph + # the task may either succeed or fail + succeeded or failed - foo? => bar + # the task may fail, but only if it also produces the output "x" + succeeded or (failed and x) - The only difference is whether or not the scheduler regards ``foo`` as - incomplete if it fails. + # the task must succeed and produce one of the pairs a, b or c, d. + succeeded and ((a and b) or (c and d)) +For more information, see +:cylc:conf:`completion <[runtime][]completion>` in the +workflow configuration section. Finish Triggers ^^^^^^^^^^^^^^^ @@ -1814,38 +1879,45 @@ Here Cylc will follow one of two "branches" depending on the outcome of task ``b Task ``d`` will run after either ``c`` or ``r`` succeeds. -.. digraph:: example - :align: center - subgraph cluster_success { - label = ":succeed" - color = "green" - fontcolor = "green" - style = "dashed" +.. list-table:: + :class: grid-table - c - } + - + - | - subgraph cluster_failure { - label = ":fail" - color = "red" - fontcolor = "red" - style = "dashed" + .. code-block:: cylc-graph - r - } + # the success path + a => b? => c + # the fail path + a => b:fail? => r + # either way, carry on with the rest of the workflow + c | r => d - a -> b -> c -> d - b -> r -> d + - .. digraph:: example + :align: center -.. code-block:: cylc-graph + subgraph cluster_success { + label = ":succeed" + color = "green" + fontcolor = "green" + style = "dashed" + + c + } - # the success path - a => b? => c - # the fail path - a => b:fail? => r - # either way, carry on with the rest of the workflow - c | r => d + subgraph cluster_failure { + label = ":fail" + color = "red" + fontcolor = "red" + style = "dashed" + + r + } + + a -> b -> c -> d + b -> r -> d The ``?`` symbol denotes an :term:`optional output` which allows the graph to branch. @@ -1854,32 +1926,42 @@ Note the last line of the graph ``c | r => d`` allows the graph to continue on to ``d`` regardless of the path taken. This is an :term:`artificial dependency`. + +Recover Task +^^^^^^^^^^^^ + Branching is often used for automatic failure recovery. Here's a simple example: -.. code-block:: cylc-graph +.. list-table:: + :class: grid-table - foo => bar - bar:fail? => recover - bar? | recover => baz + - + - | + .. code-block:: cylc-graph -.. digraph:: Example - :align: center + foo => bar + bar:fail? => recover + bar? | recover => baz - subgraph cluster_1 { - label = ":fail" - color = "red" - fontcolor = "red" - style = "dashed" - recover - } + - .. digraph:: Example + :align: center - foo -> bar - bar -> recover - recover -> baz [arrowhead="onormal"] - bar -> baz [arrowhead="onormal"] + subgraph cluster_1 { + label = ":fail" + color = "red" + fontcolor = "red" + style = "dashed" + + recover + } + + foo -> bar + bar -> recover + recover -> baz [arrowhead="onormal"] + bar -> baz [arrowhead="onormal"] The ``recover`` task would (presumably) analyse the failure of ``bar`` and, if @@ -1892,11 +1974,102 @@ A more realistic example might have several tasks on each branch. The ``bar``, but configured differently to avoid the failure. +Advanced Failure Handling Example +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ + +Using the ``:success`` and ``:fail`` outputs to control graph branching is good +for simple examples, however, the ``:fail`` output is often has too broad a +scope. + +For example, here's a workflow which gets data from a configured URL. + +.. list-table:: + :class: grid-table + + - + - .. code-block:: cylc + + [scheduling] + initial cycle point = 2000 + [[graph]] + P1Y = """ + get_data:fail? => get_fallback_data + get_data? | get_fallback_data => do_something + """ + + [runtime] + [[get_data]] + script = wget "$URL/$CYLC_TASK_CYCLE_POINT" + + - .. digraph:: Example + :align: center + + subgraph cluster_1 { + label = ":fail" + color = "red" + fontcolor = "red" + style = "dashed" + + get_fallback_data + } + + get_data -> get_fallback_data + get_data -> do_something + get_fallback_data -> do_something + +The intention here is that if the data isn't ready yet, the workflow will +switch to an alternate source. + +The problem is that there are multiple reasons why the ``get_data`` task might +fail e.g: + +* There might be an error in the task script. +* The ``$URL`` might not be defined. +* There could be a syntax error in the ``script``. +* There could be network problems. +* The request might fail due to authentication errors. +* There might not be enough disk space to hold the data. + +To make this example a bit safer, we can use a :term:`custom output` to capture +the particular failure case we are expecting (i.e. the data isn't ready yet) +to separate it from other failures: + +.. code-block:: cylc + + [scheduling] + initial cycle point = 2000 + [[graph]] + P1Y = """ + get_data:data_no_ready? => get_fallback_data + get_data? | get_fallback_data => do_something + """ + + [runtime] + [[get_data]] + # if the "wget" command fails, send the exit code back + # using "cylc message" + script = wget "$URL/$CYLC_TASK_CYCLE_POINT" || cylc message -- $? + + # allow this task to fail, but only if the error + # was "data_not_ready" + completion = succeeded or (failed and data_not_ready) + + # turn particular exit codes into custom outputs + # (these can then be used in the graph) + [[[outputs]]] + network_failure = 4 + authentication_failure = 6 + data_not_ready = 8 + +This helps to separate the error case you are trying to handle from other +possible errors. + + Message Trigger Example ^^^^^^^^^^^^^^^^^^^^^^^ Branching is particularly powerful when using :ref:`MessageTriggers` (i.e. -:term:`custom outputs `) to define multiple parallel paths in +:term:`custom outputs `) to define multiple alternative paths in the graph. In the following graph there is a task called ``showdown`` which produces one @@ -1907,49 +2080,54 @@ As with the previous example each path begins with a different :term:`optional output` of a particular task and ends with an "or" dependency to allow the workflow to continue regardless of the path taken. -.. code-block:: cylc-graph - - # branch the graph depending on the outcome of "showdown" - showdown:good? => good - showdown:bad? => bad - showdown:ugly? => ugly - # join the graph back together - good | bad | ugly => fin - - -.. digraph:: Example - :align: center - - subgraph cluster_1 { - label = ":good" - color = "green" - fontcolor = "green" - style = "dashed" - - good - } - subgraph cluster_2 { - label = ":bad" - color = "red" - fontcolor = "red" - style = "dashed" - - bad - } - subgraph cluster_3 { - label = ":ugly" - color = "purple" - fontcolor = "purple" - style = "dashed" - - ugly - } - showdown -> good - showdown -> bad - showdown -> ugly - good -> fin [arrowhead="onormal"] - bad -> fin [arrowhead="onormal"] - ugly -> fin [arrowhead="onormal"] +.. list-table:: + :class: grid-table + + - + - .. code-block:: cylc-graph + + # branch the graph depending + # on the outcome of "showdown" + showdown:good? => good + showdown:bad? => bad + showdown:ugly? => ugly + # join the graph back together + good | bad | ugly => fin + + + - .. digraph:: Example + :align: center + + subgraph cluster_1 { + label = ":good" + color = "green" + fontcolor = "green" + style = "dashed" + + good + } + subgraph cluster_2 { + label = ":bad" + color = "red" + fontcolor = "red" + style = "dashed" + + bad + } + subgraph cluster_3 { + label = ":ugly" + color = "purple" + fontcolor = "purple" + style = "dashed" + + ugly + } + showdown -> good + showdown -> bad + showdown -> ugly + good -> fin [arrowhead="onormal"] + bad -> fin [arrowhead="onormal"] + ugly -> fin [arrowhead="onormal"] You can test run this example making ``showdown`` randomly generate one of the @@ -1999,7 +2177,50 @@ When using message triggers in this way there are two things to be aware of: messages in succession, after writing out corresponding *good*, *bad*, and *ugly* files. - Check that you understand how your tasks work, if they use custom outputs. + +Flaky Pipeline Example +^^^^^^^^^^^^^^^^^^^^^^ + +In this example, Cylc will get as far through the chain ``a => b => c`` as it +can. If an error occurs at any point, Cylc will stop at that point i.e: + +* If ``a`` succeeds, run ``b``. +* If ``a`` fails, do nothing (the workflow will not stall). +* If ``b`` succeeds, run ``c``. +* If ``b`` fails, do nothing (the workflow will not stall). +* If ``c`` fails, do nothing (the workflow will not stall). + +.. list-table:: + :class: grid-table + + - + - | + + | + + .. code-block:: cylc + + [scheduling] + [[graph]] + R1 = a? => b? => c? + + [runtime] + [[a, b, c]] + completion = succeeded or failed + + - .. digraph:: Example + :align: center + + a -> b [label=":succeed"] + b -> c [label=":succeed"] + a -> a_error [color="red", fontcolor="red" label=":fail"] + b -> b_error [color="red", fontcolor="red" label=":fail"] + c -> c_error [color="red", fontcolor="red" label=":fail"] + + a_error [shape="none" label="do nothing" fontcolor="red"] + b_error [shape="none" label="do nothing" fontcolor="red"] + c_error [shape="none" label="do nothing" fontcolor="red"] + Limiting Workflow Activity