[Question] How to Manually Control Training Steps in Omnisafe PPO-Lag

[wayne-weiwei]

Required prerequisites

• I have read the documentation https://omnisafe.readthedocs.io.
• I have searched the Issue Tracker and Discussions that this hasn't already been reported. (+1 or comment there if it has.)
• Consider asking first in a Discussion.

Questions

Thank you very much for creating such an excellent tool. I am currently using the PPO-Lag algorithm from the Omnisafe library to train a custom environment. Is that possible to manually control the training process in the custom environment instead of fully relying on the automated python train_policy.py --algo PPOLag --env-id SafetyPointGoal1-v0 --parallel 1 --total-steps 10000000 --device cpu --vector-env-nums 1 --torch-threads 1, for example by using a loop like this:

action, _states = model.learn(obs)
obs, reward, terminated, truncated, info = env.step(action）

If so, what is the recommended way to implement such a manual training loop in Omnisafe, while maintaining the core functionalities of PPO-Lag for action prediction and environment interaction?

Thank you for your time, and for developing Omnisafe! I look forward to any advice or recommendations you can offer.