Revert changes to HyP3 job retry strategy

[jtherrmann]

Jira: https://asfdaac.atlassian.net/browse/TOOL-2366

Note: The above link is accessible only to members of ASF.

---

The new retry strategy was implemented in #1871

Under the new retry strategy:

• Each attempt is a separate Batch object so inspecting all three failures is more difficult.
• When a job is retried, it goes to the back of the queue. This is particularly painful during giant processing campaigns in custom deployments.
• If a hyp3 deployment occurs between retry attempts, then all subsequent retries fail because the Step Function doesn’t have perms to submit outdated Batch job definition.
  • If the AMI has changed with a deployment, then all jobs fail their current attempt, which means that under the new retry strategy, all jobs fail permanently when the AMI changes.

[jtherrmann]

These jobs should succeed: https://hyp3-test-api.asf.alaska.edu/jobs?user_id=jtherrmann&name=jth%20test%20retry%20revert

This job should fail after three attempts: https://hyp3-test-api.asf.alaska.edu/jobs/9aacccfd-3d76-4219-9cb6-58802f8a1798
(duplicate of job https://hyp3-api.asf.alaska.edu/jobs/7d9892ab-bfca-40b9-ad03-0eefbb726950)

Edit: Works as expected.

[jtherrmann]

Validating changes in prod:

These jobs should succeed: https://hyp3-api.asf.alaska.edu/jobs?user_id=jtherrmann&name=jth%20test%20retry%20revert

This job should fail after 3 attempts: https://hyp3-api.asf.alaska.edu/jobs/00147ea7-77f6-4e98-8d1c-4210bdd11b1a

Edit: Works as expected.