Issues with create_lammps_model.py

[MusannaGalib]

Hello,

I am trying to use my trained .model file to torch script-compiled LAMMPS_MACE model using create_lammps_model.py but facing the following error RuntimeError: No CUDA GPUs are available. I am confused about it. Is it because I used --device=cuda \ in the training? I installed branch mace in LAMMPS for CPU use only.

Please give your suggestion on it. Thanks.

(ML_MACE) galibubc@cedar1 ~/scratch/ML_MACE/trial_alumina/MACE_models $ python /home/galibubc/scratch/ML_MACE/trial_alumina/mace/mace/cli/create_lammps_model.py MACE_alumina_model.model

/home/galibubc/ML_MACE/lib/python3.11/site-packages/torch/cuda/__init__.py:619: UserWarning: Can't initialize NVML
  warnings.warn("Can't initialize NVML")
Traceback (most recent call last):
  File "/home/galibubc/scratch/ML_MACE/trial_alumina/mace/mace/cli/create_lammps_model.py", line 92, in <module>
    main()
  File "/home/galibubc/scratch/ML_MACE/trial_alumina/mace/mace/cli/create_lammps_model.py", line 66, in main
    model = torch.load(
            ^^^^^^^^^^^
  File "/home/galibubc/ML_MACE/lib/python3.11/site-packages/torch/serialization.py", line 1025, in load
    return _load(opened_zipfile,
           ^^^^^^^^^^^^^^^^^^^^^
  File "/home/galibubc/ML_MACE/lib/python3.11/site-packages/torch/serialization.py", line 1446, in _load
    result = unpickler.load()
             ^^^^^^^^^^^^^^^^
  File "/home/galibubc/ML_MACE/lib/python3.11/site-packages/e3nn/util/codegen/_mixin.py", line 109, in __setstate__
    smod = torch.jit.load(buffer)
           ^^^^^^^^^^^^^^^^^^^^^^
  File "/home/galibubc/ML_MACE/lib/python3.11/site-packages/torch/jit/_serialization.py", line 160, in load
    cpp_module = torch._C.import_ir_module_from_buffer(
                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: No CUDA GPUs are available

Training config used

!python3 ./mace/scripts/run_train.py \
  --name="MACE_alumina_model" \
  --model_dir="MACE_models" \
  --log_dir="MACE_models" \
  --checkpoints_dir="MACE_models" \
  --results_dir="MACE_models" \
  --train_file="/home/galibubc/scratch/ML_MACE/trial_alumina/Datasets/train.extxyz" \
  --test_file="/home/galibubc/scratch/ML_MACE/trial_alumina/Datasets/test.extxyz" \
  --valid_file="/home/galibubc/scratch/ML_MACE/trial_alumina/Datasets/validation_dataset.extxyz" \
  --E0s="average" \
  --energy_key="energy" \
  --forces_key="forces" \
  --model="MACE" \
  --hidden_irreps='128x0e' \
  --r_max=3.0 \
  --batch_size=10 \
  --valid_batch_size=10 \
  --max_num_epochs=150 \
  --num_interactions=2 \
  --num_channels=128 \
  --max_L=1 \
  --correlation=3 \
  --ema \
  --ema_decay=0.99 \
  --error_table='PerAtomRMSE' \
  --amsgrad \
  --default_dtype="float32" \
  --device=cuda \
  --seed=123 \
  --swa \
  #--wandb \
  #--wandb_project="MACE_Alumina" \
  #--wandb_entity="imazine-galib" \
  #--wandb_name="MACE_Alumina" \
  #--wandb_log_hypers="true"

[MusannaGalib]

The solution is saving the model again to convert gpu .model to cpu .model: if --save_cpu is not used during training!

import torch
#this has to be done in GPU to convert GPU .model to cpu .model

# Load the model (on a machine with GPU)
model = torch.load("/home/galibubc/scratch/ML_MACE/trial_alumina/MACE_models/MACE_alumina_model.model")

# Move the model to the CPU
model_cpu = model.to("cpu")

# Save the model as a CPU-compatible .model file
torch.save(model_cpu, "/home/galibubc/scratch/ML_MACE/trial_alumina/MACE_models/MACE_alumina_model_cpu.model")

Then create_lammps_model.py can be used to get the lammps.pt file:

!python /home/galibubc/scratch/ML_MACE/trial_alumina/mace/mace/cli/create_lammps_model.py /home/galibubc/scratch/ML_MACE/trial_alumina/MACE_models/MACE_alumina_model_cpu.model