Filtering and Sorting of experiments

[LuiggiTenorioK]

In GitLab by @eferre1 on Oct 31, 2024, 10:29

Hi @LuiggiTenorioK, I'm creating this issue to ask for new options on the main screen of the GUI, for filtering and sorting the experiments that appear.

• Right now we can sort by default (not sure how it sorts), ascending/descending creation date (it is when you run an autosubmit create?), and EXPID.
• And we can filter by autosubmit version and user (I think they need to be exact match, at least the user), and also by some other parameters on the main search box.

It would be useful, at least for me, to have more options here, like the next ones:

• Sorting by autosubmit version, for most users this is the same as creation date, but I tend to re-use experiments and update them, so it doesn't match and would make it easier to look for older experiments that I'm not using anymore
• Sorting by user, I'm not sure if it would be used, but sometimes I search experiments by description and then need to compare a couple from different users and they are all mixed (I could filter out user by user but having all on the screen and then opening the ones I need is faster than those multiple searches).
• Sorting/filtering by HPC. Not sure if it could be that much heplful, but if it is not that hard to implement and maintain, it would help users when checking which of their experiments is running where or to look for a reference experiment from another user on that platform (for example, for myself, it would help to keep track of what experiments still are on mn4, without having to look one by one and page by page). That could be done if the HPC could be searched for in the search box, or maybe filtered on another section there, or if sorting by HPC was possible, not sure what is easier or more useful to have for this point.
• Filtering by autosubmit major version (or at least not any too specific one). As mentioned, the filtering boxes for autosubmit and user need to match exactly and doesn't allow wildcards, so if I want to see all experiments with expid a7a* from either major version, 3 or 4, I can't set the search for a7a and autosubmit version to 3 or 3* to see all of the experiments, as it needs to be for example 3.15.14. A good enough option would be to at least allow the 3 vs 4 filtering, but adding wildcards here would be even more useful, as it would allow to check any experiment using the 3.15.X version for example.
• Filtering with user wildcards. Same as above, even if not that huge impact here, it is sometimes useful as I can search for a users experiment without knowing the full username (in some cases like mine, usernames have a number and not writing it on the filter box makes it show nothing). Bit of context: Usually when a user asks me for help and forget to give me the expid, I can look at the GUI to find the experiment and just typing the name first letter and a couple letters from the surname is enough, but doing that on the search bar might show experiments that have that combination on the description.

If you find any of the options mentioned above too hard to implement/maintain, don't hesitate to discard them, as I think that most of them will be used mostly by myself and not a lot of other users. Let me know if there is anything else that might be useful for you (context on use cases, etc).

Thanks.

[LuiggiTenorioK]

• Right now we can sort by default (not sure how it sorts), ascending/descending creation date (it is when you run an autosubmit create?), and EXPID.

I think the sort-by creation date can be a little confusing since the date you see in the experiment cards is the modification date. We understand as creation date as the date when the experiment directory was created which should be when autosubmit create was used.

If you find any of the options mentioned above too hard to implement/maintain, don't hesitate to discard them, as I think that most of them will be used mostly by myself and not a lot of other users. Let me know if there is anything else that might be useful for you (context on use cases, etc).

Most of these sort/filter options are doable. The main problem is that we need to focus on syncing the data we have in the file system to what we have in the DB to be as fresh as possible. This is because doing a query to the DB is way more efficient than having to fetch all the yaml/conf files of all experiments (which is unfeasible) to do the search.

[LuiggiTenorioK]

mentioned in merge request autosubmit-api!38

[LuiggiTenorioK]

@eferre1 the new version deployed on https://earth.bsc.es/autosubmitapp/ includes the new filter by HPC. Additionally, filters by owner, autosubmit version, and hpc allow using wildcards (*) and negation (!) symbols.

The sorting options will be evaluated and implemented in a future version.