File not deleted after failure to upload

[rgaudin]

When we give up trying to upload a file, it is left on the local filesystem.

This is due to our reliance on RQ's Retry mechanism for retries which does stop retrying the job after the requested number of attempts but does not inform us that attempts reached max.

Not knowing if this is the last execution, we can't delete the file.

We need to either finds a way to get this information or run clean up jobs. Maybe via FailedJobRegistry…