Feature: Sim mode at runtime

[wxtim]

Recheck simulation mode settings at runtime if a broadcast might have changed them. If it has, update these settings.

A waypoint on the road to skip mode.

Based upon #5712. Drafted until that is in.

Check List

• I have read CONTRIBUTING.md and added my name as a Code Contributor.
• Contains logically grouped changes (else tidy your branch by rebase).
• Does not contain off-topic changes (use other PRs for other changes).
• Applied any dependency changes to both setup.cfg (and conda-environment.yml if present).
• Tests are included (or explain why tests are not needed).
• CHANGES.md entry included if this is a change that can affect users
• Cylc-Doc pull request opened if required at cylc/cylc-doc/pull/XXXX.
• If this is a bug fix, PR should be raised against the relevant ?.?.x branch.

[MetRonnie]

• Broadcasting execution retry delays does not cause a change when running in simulation mode
• Also, in the Edit Runtime form, submission retry delays shows as 1 for some reason even though it is not set in flow.cylc

[wxtim]

• Broadcasting execution retry delays does not cause a change when running in simulation mode

I cannot replicate this on the command line. Given the context of your comment I tried it from the gui and still couldn't replicate.

In fact some of the tests I've written should fail if that were the case.

Can you come up with a concrete example?

Replicated the submission retry delays thing. Pretty clueless about that though.

[wxtim]

@MetRonnie Here is the cause of the value in the GUI's submission retry delays box. It also happens on master.

I think that it's probably as it's meant to be: In simulation mode, simulate retries after 1 second whatever the retry delay in live mode. However, I can't see that documented anywhere.

[MetRonnie]

• Broadcasting execution retry delays does not cause a change when running in simulation mode

I cannot replicate this on the command line. Given the context of your comment I tried it from the gui and still couldn't replicate.

In fact some of the tests I've written should fail if that were the case.

Can you come up with a concrete example?

[scheduler]
    allow implicit tasks = True
[scheduling]
    initial cycle point = 2024
    runahead limit = P0
    [[graph]]
        P1M = foo => bar

[runtime]
    [[foo]]
        execution retry delays = 100*PT10S
        script = false
        [[[simulation]]]
            fail cycle points = all
            default run length = PT10S
            fail try 1 only = False

cylc broadcast sim-mode -n foo -s 'execution retry delays=20*PT1S'

But it still keeps saying

WARNING - [20240101T0000Z/foo waiting job:06 flows:1] retrying in PT10S

(The Edit Runtime form shows it correctly but it is not having an effect)

[wxtim]

@MetRonnie - This is a good spot. I think that the broadcasts probably need to be applied in the TaskJobManager._simulation_submit_task_jobs method to ensure that we are broadcasting all task settings, not just those in [simulation]. I think you might also have found problems clearing broadcasts with things as they were, since I was changing the itask.tdef.rtconfig not a copy of it.

[MetRonnie]

Conflict resolution looks good

[wxtim]

Conflict resolution looks good

Thank you @hjoliver

[MetRonnie]

Genuine test failure presumably: tests/f/modes/05-sim-trigger.t

[hjoliver]

I'm looking at the broken test -it's just a log-grep issue after cylc set changes to log messages... I've pushed a fix to the PR branch.