Keep track of the logs retrieval process #2046
Labels
destine
DestinE related
discussion
The issue is created to keep track a discussion
enhancement
New feature or request
Milestone
In our experiments and workflows we have several job, some run on platforms such as Slurm. When the jobs run on these platforms, we launch not only the jobs on the remote host, but also a sidekick process locally, used to sync the remote logs back to the local machine.
This side process, the log recovery process (née thread), has had some issues in the past, and @mcastril gave a great suggestion some days ago in an Autosubmit meeting about keeping track of the logs retrieval process.
This would require us to know the jobs that have to have logs recovered (we already know that), and update their statuses somewhere (preferably database, so that #1914 works too, and users without NFS also have this feature) as the log recovery process progresses.
The text was updated successfully, but these errors were encountered: