Action inputs to dispatch n-runs of a single test in CI

[carlydf]

What changed?

Add workflow dispatch options to the functional tests Github Action to allow us to run n-iterations of a single functional test with a configurable timeout.

WARNING: This will definitely be oomkilled for n>=100, likely for n=>50 too. I suggest to start with n=10 and then try n=20 to see how the memory goes.

Why?

To aid in the diagnosis and treatment of flaky tests.

How did you test it?

Tested in github actions.
Here is the action run normally, with no new input parameters: https://github.com/temporalio/temporal/actions/runs/9962219294

Here is the action run on one test multiple times: https://github.com/temporalio/temporal/actions/runs/9962244076

Potential risks

Our functional test pipeline could be broken by this PR, but we would notice that pretty immediately

Documentation

How to run it yourself

1. Check out this branch cdf/rerun-functional-test
2. Optionally check out a new branch off of it, and work on your test changes there so that you can access this CI tool and your test changes together
3. Go to https://github.com/temporalio/temporal/actions/workflows/run-tests.yml
4. Click "run workflow" on the upper right hand side
5. Toggle branch to "cdf/rerun-functional-test"
6. Set Commit SHA to the latest commit on the branch
7. Select your desired options
8. Click the green "Run workflow" button

Is hotfix candidate?

[rodrigozhou]

Can we create a separate workflow file for this purpose?
run-tests.yml is complicated enough that adding this seems added complexity unnecessarily.
Also, as you noted, running misc-checks is also unnecessary when running a single test.

[dnr]

Can we create a separate workflow file for this purpose? run-tests.yml is complicated enough that adding this seems added complexity unnecessarily. Also, as you noted, running misc-checks is also unnecessary when running a single test.

A separate workflow file that we run only rarely will rot and be likely to be broken when someone wants to use it. Integrating it into the main one makes it much more likely to be maintained and working