Draft: AiiDA generator

[kinow]

In GitLab by @agoscinski on Oct 30, 2024, 13:28

As mentioned in issue https://earth.bsc.es/gitlab/es/autosubmit/-/merge_requests/300 this MR uses the work in MR https://earth.bsc.es/gitlab/es/autosubmit/-/merge_requests/300 as base to create a generator for the AiiDA engine. I take a bit different approach than MR https://earth.bsc.es/gitlab/es/autosubmit/-/merge_requests/300 in the sense that no AiiDA object is generated in autosubmit but a completely self-contained folder with all scripts and files together with a script that runs the AiiDA workflow. This has the advantage that the aiida workflow can be generated without having aiida installed. At the moment running the quicksetup job of autosubmit works.

TODOs:

• The creation of the file needs to be modularized to become maintainable
• Check if all autosubmit parameters are correctly passed
• Add warning when Wrappers and MAX_PROCESSES is used
• Test if remote works
• Figure out how default parameters are correctly determined in AiiDA, current remote solution is not so clean
• I might add a conda environment file (or pixi.toml) with dependencies and a README describing the folder structure.
• Stress test mHM workflow https://github.com/kinow/auto-mhm-test-domains
• Stress test a workflow with more dependencies (members/chunks/splits), pending on the BSC side to provide an example workflow
• Stress test Autosubmit -> AiiDA on MN5, pending on the BSC side to set it up and test (~1/2 hours to set up AiiDA, plus the dev env and workflows)