prepobs-bgc

prepobs-bgc is a set of scripts to prepare csv files with BGC variables for EnKF prepobs.

Getting started

Requirements

Having conda installed is necessary to use this project. More informations on how to download conda can be found here.

Sigma2 Betzy installation

Following modules are needed to be loaded:

module load Python/3.11.3-GCCcore-12.3.0
module load Anaconda3/2022.05

For conda installation on Betzy, see here.

Documentation

This project has a more exhaustive documentation which has been created using mkdocs.

The following command (executed at root level) will load the documentation:

make view-docs

If make is not installed, one must manually create the environment with the following commands before displaying the documentation:

conda env create --file environment.yml --prefix ./.venv
conda activate ./.venv
poetry install --group docs
mkdocs serve

The documentation should then be available at the following url: localhost:8000. The same documentation uploaded by a command:

make deploy-doc

can be found at prepobs_bgc.

Running the Scripts

To run a script from the scripts folder and named script_1.py, one can use the following command:

Using make

make run-script-1

make will install the correct environment and run the script

Any .py file loctaed in the scripts folder can be run with a make rule starting by run- and ending with the name of the file (without the .py extension) and where all underscores ('_') ar replaced by hyphens ('-'). These rules are created dynamically so if a new script is added, there is no modification to apply to Makefile to use the corresponding rule

Without make

Virtual environment must have been installed

conda activate ./.venv

Activate virtual environment

poetry install --without dev,docs

Install all required libraries

python scripts/script_1.py

Run script.

Configuration files

Each scripts has an associated configuration to set up all necessary parameters. By default, these configuration don't exists but can be created from a 'default configuration' existing in config/default. If these copy don't exist, the following command will create the file:

Using make

make copy-default-config

Without make

Manually copy/paste all scripts from config/default into config. Or use the following command:

for name in config/default/*.toml; do cp config/default/$(basename ${name}) config/_$(basename ${name}) ; done

Before running any script, one has to verify that all parameters indicated in the configuration are relevant. For example, one has to fill in all PATH parameters in the providers.toml configuration file to indicate where the providers data can be find.

Make rules cheatsheet

Installation

make all

Create the environment, install all libraries and copy the configuration files (if needed).

make copy-default-config

Copy default configuration files to the config folder if default files have been modified or if the configuration file doesn't exist.

make pre-commit

Install git pre-commit hooks to ensure that the code meets editing standards before committing to github.

make install-dev

Install the environment as make all would do and install git hooks to ensure that the code meets editing standards before committing to github.

Cleaning

make clean

'Clean' the repository environment: remove virtual environment folder and git hooks.

make clean-dirs

'Clean' the outputs: remove all bgc_fig and bgc_data directories.

Documentation

make view-docs

Create the environment, install documentation-related libraries and build the documentation website locally. The documentation is then accessible from a browser at the localhost:8000 adress. See MkDocs documentation on mkdocs serve for more informations.

make build-docs

Create the environment, install documentation-related libraries and build the documentation website into the 'site' folder. See MkDocs documentation on mkdocs build for more informations.

make deploy-docs

Create the environment, install documentation-related libraries and deploy documentation to a github branch. See MkDocs documentation on mkdocs deploy for more informations.

Running scripts

make run-any-script

Create the environment, install scripts-running-related libraries and runs the scripts/any_script.py python script. 'any-script' can be replaced by anything in order to run a script. For example, calling make run-another-script will run the scripts/another_script.py python script (if it exists). To make this rule work, the following syntax must be respected:

• script must be a python script
• script must be in the scripts/ folder
• underscores ('_') must be replaced by hyphens ('-') in the script name
• extension ('.py') must be removed from the script's name
• rule must start with the run- prefix

License :

MIT