Merge pull request #313 from libAtoms/aims_profile_update

Aims profile update

.github/workflows/pytests.yml

@@ -42,7 +42,7 @@ jobs:
 
       - name: Install latest ASE from gitlab
         run: |
-          python3 -m pip install git+https://gitlab.com/ase/ase.git
+          python3 -m pip install ase 
           echo -n "ASE VERSION "
           python3 -c "import ase; print(ase.__file__, ase.__version__)"
 

README.md

@@ -8,11 +8,11 @@ The main functions of Workflow is to efficiently parallelise operations over a s
 
 For examples and more information see [documentation](https://libatoms.github.io/workflow/)
 
-NOTE: because of the very large time intervals between official ASE releases, `wfl` is typically
-set up for (and tested against) the latest ASE gitlab repo `master` branch.  Recent changes
-that require this support include variable cell minimization using `FrechetCellFilter` and
-`Espresso` calculator configuration. See documentation link above for installation instructions.
+`wfl` and its dependendensies may be installed via `pip install wfl`. 
 
+NOTE: (as of 14 June 2024) `wfl` is only tested against the latest ASE pip release, currently v3.23.0. 
+For the time being, v3.22.1 is the mininum version listed as a (pip) prerequisite of `wfl`, because
+it at least mostly works, but it may not be fully compatible and is not actively tested.
 
 # Recent changes
 

docs/source/examples.fhiaims_calculator.ipynb

@@ -10,12 +10,13 @@
     "This example illustrates the usage of Workflow for efficiently performing density-functioncal theory (DFT) calculations with the all-electron electronic structure code [FHI-Aims](https://fhi-aims.org/).\n",
     "\n",
     "At first, we initialize a `ConfigSet` with a couple of (periodic and non-periodic) systems stored in `Input_Structures.xyz`.\n",
-    "We also define an `OutputSpec` to handle the output that will additionally comprise values calculated by FHI-Aims.\\\n",
+    "We also define an `OutputSpec` to handle the output that will additionally comprise values calculated by FHI-Aims.\n",
+    "\n",
     "Next, we define various parameter that will be used during the calculations.\n",
     "Most of them define parameter settings for the DFT calculation (that will be written in the `control.in` input file of FHI-Aims).\n",
     "Note that for non-periodic systems the calculations will be performed with the exact same settings, but without parameter specific to periodic systems (e.g. `k_grid`, `compute_analytical_stress`, etc.).\n",
-    "The last parameter `calculator_exec` defines the command used to call FHI-Aims and start an individual calculation, including specifications for the parallelization of that specific calcuation and excluding any redirection of the output (i.e. without e.g. `>> aims.out`).\\\n",
-    "Now we define a `calculator` of the form ` (calc_constructor, args, kwargs)` using the `Aims` calculator of Workflow and the parameter dictionary we have just defined.\n",
+    "\n",
+    "With these parameters and Workflow's `Aims` Calculator object, we define a `calculator` tuple of the form ` (calc_constructor, args, kwargs)`.\n",
     "Finally, we apply the `generic.calculate()` function to process all the input structures in parallel (defined via the `autopara_info` settings). The values of the calculated `properties` will be written into `Output_Structures.xyz` with keys that have the specified `output_prefix`. Note that for non-periodic systems the property \"stress\" will be ignored."
    ]
   },
@@ -58,7 +59,6 @@
     "        'compute_analytical_stress': True,\n",
     "        'KS_method':                 'parallel',\n",
     "        'k_grid_density':            1e-1,\n",
-    "        'calculator_exec':           'srun -n2 --exclusive --mem=2GB aims.210313.scalapack.mpi.x',\n",
     "        }\n",
     "\n",
     "\n",
@@ -78,6 +78,35 @@
     ")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "id": "8ec7a593-9b4f-41cc-a5f9-208440978742",
+   "metadata": {},
+   "source": [
+    "As of ASE version 3.23, the file-based calculators rely on a configuration file that specifies the information about the particular installation of the electronic structure code ([see ASE's docummentation](https://wiki.fysik.dtu.dk/ase/ase/calculators/calculators.html#calculator-configuration)). \n",
+    "\n",
+    "An example `~/.config/ase/config.ini`:\n",
+    "\n",
+    "```\n",
+    "[aims]\n",
+    "command=mpirun -n 32 path/to/aims/aims.231208.scalapack.mpi.x\n",
+    "default_species_directory=path/to/aims/species_defaults/defaults_2020/light/\n",
+    "```\n",
+    "\n",
+    "Alternatively, the species directory maybe included among the keyword arguments:\n",
+    "\n",
+    "```\n",
+    "aims_kwargs[\"species_dir\"] = \"path/to/aims/species_defaults/defaults_2020/light/\"\n",
+    "```\n",
+    "\n",
+    "Instead of the command in ase's config file, a `profile` may be provided among the keyword arguments:\n",
+    "\n",
+    "```\n",
+    "from ase.calculators.aims import AimsProfile\n",
+    "aims_kwargs[\"profile\"] = AimsProfile(\"mpirun -n 32 path/to/aims/aims.231208.scalapack.mpi.x\")\n",
+    "```"
+   ]
+  },
   {
    "cell_type": "code",
    "execution_count": null,
@@ -101,7 +130,7 @@
  ],
  "metadata": {
   "kernelspec": {
-   "display_name": "Python 3 (ipykernel)",
+   "display_name": "Python 3",
    "language": "python",
    "name": "python3"
   },
@@ -115,7 +144,7 @@
    "name": "python",
    "nbconvert_exporter": "python",
    "pygments_lexer": "ipython3",
-   "version": "3.9.13"
+   "version": "3.8.19"
   }
  },
  "nbformat": 4,

docs/source/examples.index.md

@@ -7,7 +7,6 @@ Examples often show examples of more than one thing. Below is a list of common o
 ## Evaluate structures with a calculator
 
 - ORCA: [ORCA via python script](examples.orca_python.md)
-- MACE: [Parallelize MACE calculator](examples.mace.md)
 - EMT: 
     - [First example](first_example.md)
     - [Iterative GAP fitting ](examples.mlip_fitting.md)

docs/source/first_example.md

@@ -82,7 +82,7 @@ from ase.calculators.emt import EMT
 from wfl.calculators import generic
 from wfl.autoparallelize import AutoparaInfo
 from wfl.configset import ConfigSet, OutputSpec
-from wfl.autoparallelize.remote info import RemoteInfo()
+from wfl.autoparallelize import RemoteInfo
 from expyre.resources import Resources
 
 atoms = []
@@ -124,7 +124,7 @@ python evaluate_emt.py
 Workflow also allows to submit (remotely) queued jobs automatically, by interfacing with ExPyRe ([docummentation](https://libatoms.github.io/ExPyRe/), [repository](https://github.com/libAtoms/ExPyRe/tree/main/expyre)). In this example, instead of calling the above python script in a queue submission script, the modified python script is called from the head node and the parallelisation mechanism behind `generic.calculate()` sets up and submits the job and returns the results like the script normally would. To enable remote submission, `RemoteInfo` must be added to `AutoparaInfo`. 
 
 ```
-from wfl.autoparallelize import RemoteInfo()
+from wfl.autoparallelize import RemoteInfo
 from expyre.resources import Resources
 
 remote_info = RemoteInfo(
@@ -145,12 +145,13 @@ remote_info = RemoteInfo(
 The available clusters are listed in `config.json` file, by default at `~/.expyre/config.json`:
 
 ```
-"local": { "host": null,
-        "scheduler": "sge",
-        "commands": ["conda activate myenv"],
-        "header": ["#$ -pe smp {num_cores}"],
-        "partitions": {"standard" : {"num_cores": 16, "max_time": "168h", "max_mem": "200GB"}}
-             }
+{"systems":
+    {"local": { "host": null,
+            "scheduler": "sge",
+            "commands": ["conda activate myenv"],
+            "header": ["#$ -pe smp {num_cores}"],
+            "partitions": {"standard" : {"num_cores": 16, "max_time": "168h", "max_mem": "200GB"}}
+                 }
 }}
 ```
 

docs/source/index.rst

@@ -21,21 +21,16 @@ Quick start that installs all of the mandatory dependencies:
 
 .. code-block:: sh
 
-	python3 -m pip install git+https://github.com/libAtoms/workflow
+	python3 -m pip install wfl
 
-.. warning::
+.. note::
 
-    `wfl` requires ASE, so `ase` is listed as a `pip` dependency,
-    and if not already installed, `pip install` will install the latest
-    `pypi` release.  However, because of the large delay in producing new
-    releases, the latest `pypi` version is often quite old, and `wfl`
-    has some functionality that requires a newer version.  To ensure
-    a sufficiently up-to-date version is available, before installing
-    `wfl` install the latest `ase` from gitlab, with a command such as
+    (as of 14 June 2024) `wfl` is only tested against the latest
+    ASE pip release, currently v3.23.0.  For the time being, v3.22.1 is
+    the mininum version listed as a (pip) prerequisite of `wfl`, because
+    it at least mostly works, but it may not be fully compatible and is
+    not actively tested.
 
-    .. code-block:: sh
-
-            python3 -m pip install git+https://gitlab.com/ase/ase
 
 ***************************************
 Repository

docs/source/operations.calculators.md

@@ -25,10 +25,5 @@ Further see [autoparallelization page](overview.parallelisation.rst) and [exampl
 
 ASE's calculators that write & read files to & from disk must to be modified if they were to be parallelized via Workflow's `generic` calculator. Specifically, each instance of calculator must execute the calculation in a separate folder so processes running in parallel don't attempt to read and write to the same files. Workflow handles the files, as well as creation and clean-up of temporary directories. 
 
-Currently, [ORCA], VASP, QuantumEspresso and FHI-Aims are compatible with the `generic` calculator; CASTEP parallelization is accessed via `calculators.dft.evaluate_dft()`.
-
-## Special calculators
-
-Finally, there is a (currently broken) non-conventional "Basin Hopping" calculator implementation for ORCA. 
-`BasinHoppingORCA()` runs multiple single point evaluations perturbing the initial guess of the wavefunction each time. It returns the results corresponding to the global minimum and lowest-energy solution.  
+Currently, ORCA, VASP, QuantumEspresso, CASTEP and FHI-Aims are compatible with the `generic` calculator.
 

tests/calculators/test_aims.py

@@ -1,23 +1,41 @@
 import os
 import pytest
+from pathlib import Path
+
+from packaging.version import Version
 
 import numpy as np
+import ase
 from ase import Atoms
 from ase.build import bulk
 
 import wfl.calculators.aims
 from wfl.calculators import generic
 from wfl.configset import OutputSpec
 
+if Version(ase.__version__) < Version("3.23"):
+    aims_prerequisites = pytest.mark.skip(reason="Aims tests are only supported for ASE v3.23, please update.")
+
+else:
+
+    from ase.config import cfg as ase_cfg
+    from ase.calculators.aims import AimsProfile
 
-aims_prerequisites = pytest.mark.skipif(
-        condition='ASE_AIMS_COMMAND' not in os.environ or 'AIMS_SPECIES_DIR' not in os.environ
-                   or not os.environ['AIMS_SPECIES_DIR'].endswith('light')
-                   or 'OMP_NUM_THREADS' not in os.environ or os.environ['OMP_NUM_THREADS'] != "1",
-        reason='Missing env var ASE_AIMS_COMMAND or ASE_SPECIES_DIR or ' +
-               'ASE_SPECIES_DIR does not refer to light-settings or ' +
-               'OMP_NUM_THREADS or OMP_NUM_THREADS has not correctly been set to 1'
-        )
+    if "aims" in ase_cfg.parser:
+        profile = AimsProfile.from_config(ase_cfg, "aims")
+        species_dir = vars(profile).get("default_species_directory", None)
+    else:
+        species_dir = None
+
+    aims_prerequisites = pytest.mark.skipif(
+        condition = 'aims' not in ase_cfg.parser or species_dir is None
+                    or Path(species_dir).name != "light"
+                    or 'OMP_NUM_THREADS' not in os.environ or os.environ['OMP_NUM_THREADS'] != "1",
+        reason='Missing "aims" in ase\'s configuration file or "default_species_directory" ' +
+                    'in "aims" configuration or "default_species_directory"" does not refer' +
+                    'to "light" settings or missing "OMP_NUM_THREADS" or "OMP_NUM_THREADS" ' +
+                    'is not set to 1.'
+    )
 
 
 @pytest.fixture
@@ -41,7 +59,7 @@ def parameters_nonperiodic():
     }
     return parameters
 
-
+@aims_prerequisites
 def test_setup_calc_params(parameters_nonperiodic):
 
     parameters = parameters_nonperiodic
@@ -57,7 +75,7 @@ def test_setup_calc_params(parameters_nonperiodic):
     parameters.update(parameters_periodic)
 
     # needed so new ASE versions don't complain about a lack of configuration
-    parameters["calculator_exec"] = "_DUMMY_"
+    parameters["profile"] = AimsProfile("_DUMMY_")
 
     # PBC is FFF
     atoms = Atoms("H")
@@ -150,3 +168,6 @@ def test_generic_aims_calculation(tmp_path, parameters_nonperiodic):
     assert si2.arrays["Aims_forces"][0, 0] == pytest.approx(expected=-0.29253217, abs=1e-3)
     assert si2.arrays["Aims_forces"][:, 1:] == pytest.approx(0.0)
     assert si2.arrays["Aims_forces"][0] == pytest.approx(-1 * si2.arrays["Aims_forces"][1])
+
+
+

tests/calculators/test_calc_generic.py

@@ -1,6 +1,7 @@
 import sys
 from os.path import join
 from io import StringIO
+import pytest
 
 import numpy as np
 from ase import Atoms
@@ -138,7 +139,7 @@ def test_generic_autopara_defaults():
     sys.stderr = sys.__stderr__
     assert "num_inputs_per_python_subprocess=3" in l_stderr.getvalue()
 
-
+@pytest.mark.xfail(reason="Waiting for update to work with ASE3.23")
 def test_generic_DFT_autopara_defaults(tmp_path, monkeypatch):
     ats = [Atoms('Al2', positions=[[0,0,0], [1,1,1]], cell=[10]*3, pbc=[True]*3) for _ in range(50)]
 

tests/calculators/test_qe.py

@@ -57,7 +57,6 @@ def qe_cmd_and_pseudo(tmp_path_factory):
 
     return cmd, pspot_file
 
-
 def test_qe_kpoints(tmp_path, qe_cmd_and_pseudo):
 
     qe_cmd, pspot = qe_cmd_and_pseudo

tests/local_scripts/gelzinyte.workstation.sh

@@ -0,0 +1,7 @@
+#!/bin/bash
+
+export ASE_CONFIG_PATH=${HOME}/.config/ase/pytest.config.ini
+
+# Aims
+pytest -v -s -rxXs  ../calculators/test_aims.py
+