Getting errors quite regularly in practice

[ChrisSargent]

We have this running as a CI job, it probably gets run 8 or 10 times a day and currently is failing about 50% of the time. I've attached some screenshots of the logs. It looks like maybe observatory is doing some throttling but then the retries are too close together and it gets blocked, not quite sure though.

Happy to try to help with a PR if you have an idea where to start looking?

Thanks,

Chris

[simonireilly]

Hi Chris,

Thanks for this.

I've seen this myself and had retries put at 30 within the action in order to try and counter that.

I do think Observatory throttles, and in their docs they recommend running your own version of the tool - I believe via docker.

Let me do a deeper dig.

All contributions welcome of course, the code for this project is fairly light, it's just exec'ing the observatory CLI.

[simonireilly]

At first glance I would say the problem lies in the underlying CLI, here:

https://github.com/mozilla/observatory-cli/blob/a4635a41522dee9bec5f3dd21063d95422166d58/index.js#L124-L183

But following on from that it looks like here:

observatory-github-action/src/tools/observatory-runner.ts

Lines 25 to 37 in 8ede7d2

	try {
	await exec.exec(
	'npx',
	['observatory-cli', sanitizedHostName, '--format=json', '--attempts=30'],
	options
	);
	} catch (e) {
	if (e instanceof Error) {
	core.setFailed(e.message);
	} else {
	core.setFailed('An unknown error occurred');
	}
	}

In this action I might try something different - I'll implement an exponential backoff logic here - because I think the linear single second retry is not configurable:

mozilla/observatory-cli#25

[simonireilly]

@ChrisSargent I have cut a beta at: https://github.com/simonireilly/observatory-github-action/releases/tag/v0.1.1-beta.1

Can you try:

- name: Observatory Github Action
  id: observatory
  uses: simonireilly/[email protected]
  with:
    web_host: https://example.com

Would be good to know if this increases the stability by reducing the total number of requests.

[simonireilly]

Should be closed by https://github.com/simonireilly/observatory-github-action/releases/tag/v0.1.1

Re-open if it continues