Scheduler: Refactor interface to make it more generic

[sphuber]

New interface makes fewer assumptions about how an implementation should submit or kill job, or how information about active jobs is to be retrieved.

[chrisjsewell]

As mentioned, this is working with https://github.com/aiidateam/aiida-firecrest

[chrisjsewell]

In principle, these methods should no longer be required on the transport, but not needed for this PR: https://github.com/aiidateam/aiida-firecrest/blob/eb7f7518857794a99bae3568c74f0981b695dd79/aiida_firecrest/transport.py#L521-L533

[chrisjsewell]

Also, as a follow-up for the transport, here I moved the currently hard-coded verdi computer test logic onto the transport, since some tests are specific to a "bash-type" transport: https://github.com/chrisjsewell/aiida_core/blob/092bf00c327e213b3de3fc1650df54218a88df54/aiida/client/implementation.py#L74

[khsrali]

@sphuber I tried this PR to submit a job using aiida-firecrests on CSCS, and it works.
If you think of any set of specific manual testing that you think it might be important to check, let me know, and I'll check.

[khsrali]

I went through this changes, apart from overlooking things, also manually tested the main functionalities of the slurm scheduler with an HPC through core.ssh transport plugin.

tested:
engine.submit() & engine.run()
verdi process kill works as expected.

In addition, I directly test scheduler functions:

from aiida import load_profile
from aiida.orm.utils.loaders import load_computer
load_profile()
computer = load_computer('cscs-eiger')
scheduler = computer.get_scheduler()
transport = computer.get_transport()
transport.open()
scheduler.set_transport(transport)
jobid = scheduler.submit_job('/capstor/scratch/cscs/akhosrav/test-scheduler/with_sleep/', '_aiidasubmit.sh')
scheduler.get_jobs(jobid)
scheduler.kill_job(jobid)

All is good.

I haven't manually tested other scheduler plugins refactored in this PR, but I don't foresee any problems with them. Changes made in this PR are reasonable and straightforward to track down.

I would suggest having this PR merged before the release.

[khsrali]

Once again, I believe this could get merged. I've been using it for a while with no issues.
@mbercx do you still want to check this for hyperQ?

[codecov]

Codecov Report

Attention: Patch coverage is 95.08197% with 3 lines in your changes missing coverage. Please review.

Project coverage is 77.83%. Comparing base (ef60b66) to head (e3620e0).
Report is 116 commits behind head on main.

Files with missing lines	Patch %	Lines
src/aiida/schedulers/plugins/bash.py	94.74%	2 Missing ⚠️
src/aiida/engine/daemon/execmanager.py	66.67%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #6043      +/-   ##
==========================================
+ Coverage   77.51%   77.83%   +0.32%     
==========================================
  Files         560      564       +4     
  Lines       41444    41911     +467     
==========================================
+ Hits        32120    32616     +496     
+ Misses       9324     9295      -29     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

approved

[khsrali]

Cheers @sphuber