Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Keep track of the logs retrieval process #2046

Open
kinow opened this issue Dec 20, 2024 · 0 comments
Open

Keep track of the logs retrieval process #2046

kinow opened this issue Dec 20, 2024 · 0 comments
Labels
destine DestinE related discussion The issue is created to keep track a discussion enhancement New feature or request

Comments

@kinow
Copy link
Member

kinow commented Dec 20, 2024

In our experiments and workflows we have several job, some run on platforms such as Slurm. When the jobs run on these platforms, we launch not only the jobs on the remote host, but also a sidekick process locally, used to sync the remote logs back to the local machine.

This side process, the log recovery process (née thread), has had some issues in the past, and @mcastril gave a great suggestion some days ago in an Autosubmit meeting about keeping track of the logs retrieval process.

This would require us to know the jobs that have to have logs recovered (we already know that), and update their statuses somewhere (preferably database, so that #1914 works too, and users without NFS also have this feature) as the log recovery process progresses.

@kinow kinow added enhancement New feature or request destine DestinE related discussion The issue is created to keep track a discussion labels Dec 20, 2024
@kinow kinow added this to the Autosubmit 4.2.0 milestone Dec 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
destine DestinE related discussion The issue is created to keep track a discussion enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant