Exact Script for Reproducing the Results on rMD17 Dataset

[yilunliao]

Hello.

Thanks for open-souring the great work.

I have some questions regarding rMD17:

1. Can you provide the exact script for reproducing the rMD17 results in the paper? I check the documentation but do not find a script for rMD17.
2. Can you also point out how and where you download rMD17 dataset? I think we need to download some .xyz files on our own before running training.

Thank!

[ilyes319]

Hey,

1. There have been quite a lot of changes to the interface, so it is not the original script, but here is an input that will reproduce the result for ethanol:

python ./mace/scripts/run_train.py \
    --name="MACE_ethanol" \
    --train_file="/rds/user/ib467/hpc-work/Equivariant_GNN/mace_rmd17/rmd17_ethanol_train.xyz" \
    --valid_fraction=0.05 \
    --test_file="/rds/user/ib467/hpc-work/Equivariant_GNN/mace_rmd17/rmd17_ethanol_test.xyz" \
    --energy_weight=8.0 \
    --forces_weight=1000.0 \
    --config_type_weights='{"Default":1.0}' \
    --E0s='average' \
    --model="ScaleShiftMACE" \
    --interaction_first="RealAgnosticResidualInteractionBlock" \
    --interaction="RealAgnosticResidualInteractionBlock" \
    --num_interactions=2 \
    --error_table="TotalMAE" \
    --max_ell=3 \
    --hidden_irreps='256x0e + 256x1o + 256x2e' \
    --num_cutoff_basis=5 \
    --correlation=3 \
    --r_max=6.0 \
    --scaling='rms_forces_scaling' \
    --batch_size=5 \
    --max_num_epochs=3500 \
    --lr=0.01 \
    --patience=200 \
    --swa_forces_weight=1000.0 \
    --swa_energy_weight=8.0 \
    --swa_lr=0.001 \
    --weight_decay=5e-7 \
    --ema \
    --ema_decay=0.99 \
    --amsgrad \
    --default_dtype="float32"\
    --swa \
    --start_swa=2000 \
    --clip_grad=10 \
    --device=cuda \
    --seed=3 \
    --restart_latest \

For other molecules just change the energy_weight to the number of atoms in this molecule.

2. Here the link: https://figshare.com/articles/dataset/Revised_MD17_dataset_rMD17_/12672038. For each molecule, sample 1000 for training (using 50 for val) and 1000 for test. Note that the units are in kcal/mol and you need to convert them to eV for MACE.

[ilyes319]

Here is a conversion script:

import numpy as np
import ase.io
import argparse


def parse_args():
    parser = argparse.ArgumentParser()
    parser.add_argument('--input', type=str, required=True)
    parser.add_argument('--output', type=str, required=True)
    parser.add_argument('--pbc', type=bool, default=True)
    parser.add_argument('--cell', type=str, default='[[0, 0, 0], [0, 0, 0], [0, 0, 0]]')
    args = parser.parse_args()
    return args

def npz_to_xyz(input_name, output_name, cell=None, pbc=False):
    file = np.load(input_name)
    n_structures = len(file['coords'])
    for i in range(n_structures):
        atoms = ase.Atoms(symbols=file['nuclear_charges'], positions=file['coords'][i])
        atoms.info['energy'] = 0.04336411531 * file['energies'][i]
        atoms.cell = np.array(eval(cell))
        atoms.pbc = pbc
        atoms.arrays['forces'] = 0.04336411531 * file['forces'][i]
        ase.io.write(output_name, atoms, format='extxyz', append=True)


npz_to_xyz(input_name=r"rmd17_benzene.npz", output_name=r"rmd17_benzene.xyz", cell='[[0, 0, 0], [0, 0, 0], [0, 0, 0]]', pbc=False)

We use the original set of indices provided to select which ones to use. But some people also select randomly.

[yilunliao]

I see. I will try this later.

Thank a lot for the help, @ilyes319 and @davkovacs !

[yilunliao]

Hi @ilyes319 and @davkovacs

I have several questions regarding reproducing the results of ethanol:

1. In the splits provided by revised MD17 dataset, they have index_train_0X.csv with X from 1 to 5. Can you please let me know which splits you used?
2. I cannot reproduce the result on ethanol using index_train_01.csv and index_test_01.csv. The paper reports energy MAE = 0.4 meV and force MAE = 2.4 meV/Å while I ran the above command but got (validation) energy MAE = 0.5 meV and force MAE = 2.3 meV/Å and (test) energy MAE = 0.8 meV and force MAE = 3.0 meV/Å. The differences are significant, so I surmise this can be due to the split being used.

Thanks!

[ilyes319]

Hi @yilunliao, can you share your log file with me, so I can have a look.

[yilunliao]

Sure. Here is the log.

I slightly modify the code and evaluate on the test set every 10 epochs to see how the errors decrease.

Thanks!