Skip to content

Tooling

Jump to bottom
Jean-Noël Grad edited this page May 13, 2024 · 10 revisions

This page assumes that you have set up a working build environment for ESPResSo. Instructions can be found in the Installation section of the User Guide.

The following sections will explain how to set up a development environment for ESPResSo. If you encounter any issue, please refer to the Installation FAQ.

Table of Contents

Build system tooling

You will need a recent version of CMake to compile the code. If your version of CMake is too old, you can install a newer one with the pip package manager: 

python3 -m pip install --user 'cmake==3.22'

The build time can be reduced by using ccache. This can be enabled by the CMake option ESPRESSO_BUILD_WITH_CCACHE, e.g. 

cmake -D ESPRESSO_BUILD_WITH_CCACHE=ON .

The project doesn't provide a custom CMakeGraphVizOptions configuration file, but it is still possible to export the list of CMake target dependencies as a graph with the following commands: 

cmake . -D ESPRESSO_BUILD_TESTS=OFF
cmake --graphviz=targets-deps-graph.dot . ..
dot -Tps targets-deps-graph.dot -o targets-deps-graph.ps

Static analysis

The build system integrates well with static analysis tools:

Clang-Tidy: 

cmake .. -D ESPRESSO_BUILD_WITH_CLANG_TIDY=ON

Cppcheck: 

cmake .. -D ESPRESSO_BUILD_WITH_CPPCHECK=ON -D CMAKE_BUILD_TYPE=Debug -D ESPRESSO_BUILD_WITH_CUDA=OFF

scan-build: 

scan-build cmake .. -D CMAKE_BUILD_TYPE=Debug -D ESPRESSO_BUILD_WITH_CUDA=OFF -D ESPRESSO_BUILD_WITH_PYTHON=OFF
scan-build -k -o static-analysis/ make -j$(nproc)
scan-build -k -o static-analysis/ make -j$(nproc) check_unit_tests

include-what-you-use: 

cmake .. -D CMAKE_CXX_INCLUDE_WHAT_YOU_USE="iwyu;-Xiwyu;--no_fwd_decls;-Xiwyu;--max_line_length=120;-Xiwyu;--keep=config.hpp"

Python packages

Make sure all dependencies listed in requirements.txt are installed on your machine. This can be achieved with the following command: 

python3 -m pip install --user -r requirements.txt

or with the following command, if you only want a specific subset: 

python3 -m pip install --user -c requirements.txt cython numpy scipy matplotlib

The pip package manager will typically install the most recent version of a package, unless a version that meets the requirements is already available on the computer. You can pin a package version like so: 

python3 -m pip install --user 'scipy==1.6.0'

Some of these dependencies will install binaries, e.g. sphinx, jupyter and autopep. These binaries need to be accessible in your terminal. This is achieved by setting the $PATH environment variable. This variable can be set automatically by editing your ~/.bashrc and adding the following line: 

export PATH="${HOME}/.local/bin/:${HOME}/bin${PATH:+:${PATH}}"

This shell command adds ~/.local/bin: in front of the $PATH variable. If the variable was an empty string, it will set it to ~/.local/bin. The colon symbol acts as a separator between paths. You can avoid this step by adopting Python environments, which automatically set environment variables up for you. This will be discussed in the next section.

Environments

We strongly recommend using Python environments to isolate packages required by ESPResSo from packages installed system-wide. This can be achieved using venv, conda, or any similar tool. Inside an environment, commands of the form sudo apt install python3-numpy python3-scipy can be rewritten as python3 -m pip install numpy scipy, and thus do not require root privileges.

Depending on your needs, you may choose to install all ESPResSo dependencies inside the environment, or only the subset of dependencies not already satisfied by your workstation or cluster. For the exact syntax to create and configure an environment, please refer to the tool documentation; the next subsections will show how to get started with the most common Python environment tools.

Python environment tools may allow you to install a Python executable that is more recent than the system-wide Python executable. Be aware this might lead to compatibility issues if Cython accidentally picks up the system-wide Python.h header file. In that scenario, you will have to manually adapt the C++ compiler include paths to find the correct Python.h header file.

venv

Installing venv: 

sudo apt install python3-venv

Creating a new environment called "espresso" in the current working directory: 

python3 -m venv espresso
source espresso/bin/activate
python3 -m pip install -c requirements.txt cython numpy scipy matplotlib
deactivate

Cluster users who rely on module files to load dependencies should opt for the following alternative, which automatically detects the existing Cython and NumPy packages provided by the cluster environment. Only packages not detected system-wide will be installed by pip: 

module load ESPResSo/4.2.1-foss-2022a # loads ESPResSo, Python, numpy, matplotlib
python3 -m venv --system-site-packages espresso
source espresso/bin/activate
python3 -m pip install -c requirements.txt cython numpy scipy matplotlib
deactivate
module purge

In both cases, the system-wide Python executable will be picked up. All pip-installed Python packages will be stored in a local folder called "espresso".

The deactivate command works similar to module purge. Python scripts must be executed between the source and deactivate commands.

To use that environment in JupyterLab, register it in a new kernel: 

source espresso/bin/activate
python3 -m pip install ipykernel "jupyterlab>=4.0.8" "PyOpenGL>=3.1.5"
python3 -m ipykernel install --user --name=ESPResSo
deactivate

Please be aware the "ESPResSo" kernel will be registered outside the environment, typically in your home folder. You can later inspect the list of registered kernels and delete unwanted ones with the following commands: 

jupyter kernelspec list
jupyter kernelspec uninstall ESPResSo

The JupyterLab main menu will now show a new Python kernel called "ESPResSo" that uses the virtual environment.

conda

Please refer to Installing conda to find the procedure for your specific operating system. 

# this is just an example for Anaconda 2024 on Ubuntu...
wget https://repo.anaconda.com/archive/Anaconda3-2024.02-1-Linux-x86_64.sh 
bash Anaconda3-2024.02-1-Linux-x86_64.sh 
source "${HOME}/anaconda3/etc/profile.d/conda.sh"
export PYTHONPATH="${HOME}/anaconda3/pkgs"

Creating a new environment called "espresso" in the global registry: 

conda create --name espresso python cython numpy scipy pandas
conda activate espresso
conda deactivate

The system-wide Python executable will be picked up by the environment. To install a local version of Python, use the following alternative: 

conda create --name espresso python=3.12 cython numpy scipy pandas
conda activate espresso
conda deactivate

Linter and code formatter

The project uses autopep8, ClangFormat and cmake-format for code formatting and pylint for linting. Their execution is managed by pre-commit.

We recommend that contributors install the necessary dependencies and run the linter and formatters before opening a PR. The Python dependencies are listed at the end of requirements.txt and can be installed locally using the following syntax: 

python3 -m pip install --user -c requirements.txt autopep8 pylint pycodestyle

Here are a few commands to get you started with pre-commit: 

# help page
pre-commit run --help
# run on all files modified since the last commit in upstream
pre-commit run --files $(git diff --name-only upstream/python)
# run on all files
pre-commit run --all-files
# run on all files, except pylint and autopep8
SKIP=pylint,autopep8 pre-commit run --all-files

To avoid typing pre-commit commands every time, the tool can be configured with git hooks to automate the linting and formatting actions during certain git events, such as commits. This behavior can be temporarily disabled with SKIP=pylint,autopep8 git commit -m "message". Please refer to the pre-commit manual for how to set up git hooks in your ESPResSo fork.

Git

Profile setup

Set up your GitHub profile with a verified email address, and with an authentication method to be able to push your changes from the command line, for example via SSH (see Connecting to GitHub with SSH).

Set up your git profile with the information from your GitHub profile: 

# set username/email globally (remove --global to set locally in the .git folder)
git config --global user.email "[email protected]"
git config --global user.name "John Doe"
git config --global push.default simple

Use the same user name and email address as the ones in your GitHub account if you want your commits to be attached to your GitHub account (they will appear in your statistics and be decorated with your profile picture). If the information doesn't match, a default grey icon will appear next to your commits instead of your profile picture.

Repository setup

Create a fork of the ESPResSo project on GitHub with the Fork button, then download the fork on your workstation and set the upstream branch with the following commands (assuming your GitHub login is username): 

git clone --recursive [email protected]:username/espresso.git
cd espresso
git remote add upstream [email protected]:espressomd/espresso.git

At this stage, git remote -v should display the following: 

origin	[email protected]:username/espresso.git (fetch)
origin	[email protected]:username/espresso.git (push)
upstream	[email protected]:espressomd/espresso.git (fetch)
upstream	[email protected]:espressomd/espresso.git (push)

The remote origin refers to your fork, while upstream refers to the main ESPResSo repository.

Publishing your work

Here is a typical workflow to commit changes locally and push them to the fork: 

# fetch the latest commit upstream
git fetch upstream python
# create a new branch to fix ticket #443
git checkout -b fix-443 upstream/python
# do the changes
sed -i '/cmake_policy(SET CMP0025 NEW)/d' CMakeLists.txt
# commit the changes
git add CMakeLists.txt
git commit -m 'CMake: Remove policy CMP0025'
# push the branch to the fork
git push -u origin fix-443

You can now open a PR on the main ESPResSo repository.

For more details, please refer to Collaborating with issues and pull requests.

Docker

You can reproduce the build environment used in our CI pipeline using Docker. The Docker images are available on GitHub and DockerHub (under espressomd). They can also be built locally: the Docker image tag contains the commit SHA and name of the corresponding Dockerfile in espressomd/docker.

The exact ESPResSo build and test sequence can be found in .gitlab-ci.yml. Here is how to pull an image locally and run the maxset CI job on Ubuntu 24.04: 

docker run --shm-size=512m --user espresso -it ghcr.io/espressomd/docker/ubuntu:f7f8ef2c0ca93c67aa16b9f91785492fb04ecc1b bash
git clone --depth=1 --recursive -b python https://github.com/espressomd/espresso.git
cd espresso
export with_ccache=true build_procs=$(nproc) check_procs=$(nproc)
export CC=gcc-9 CXX=g++-9 with_cuda=false myconfig=maxset with_coverage=false
export with_scafacos=true with_stokesian_dynamics=true
bash maintainer/CI/build_cmake.sh

Please note the Docker image tag is subject to change! We update these Dockerfiles every 4 months on average.

IDE

VS Code

Visual Studio Code and its Live Share extension can be quite convenient for pair programming on C++ code. Both programmers need to use the same version of the IDE. It can be installed on Linux workstations without admin rights. Below is the installation and configuration procedure for version 1.59.

Installing VS code:

  • a .tar.gz archive can be downloaded from the official website (https://code.visualstudio.com)
  • run the executable (VSCode-linux-x64/bin/code on Linux)
Installing VS Code extensions:
  • Ctrl+Shift+X to open the Extensions panel
  • if not already installed, search in the Marketplace and install "C/C++", "CMake", "CMake Tools", "Live Share"
Configuring VS Code Live Share:
  • click on the Live Share icon (left panel of VS Code, the icon looks like an arrow above a circle)
  • you can now join sessions as an anonymous user
To create VS Code Live Share sessions, you need to log in using either a GitHub or a Microsoft account:
  • configure Live Share: click sign-in
  • with GitHub: select sign-in from GitHub, your browser will open a new tab
  • click approve in the GitHub page (when you get confirmation, do not close the tab yet)
  • if nothing happens in VS Code, go back to the GitHub page and click "Having trouble? Click here for user code directions" to get the token, and copy-paste it in VS Code
  • you should now be logged in (icon with your name in the bottom-left corner of the VS Code interface)
To run python scripts with pypresso:
  • install the "Python" extension
  • click the gear at the bottom left and choose Settings
  • search for "Default Interpreter Path"
  • update the field with the absolute path to the pypresso executable in the build directory
Clone this wiki locally