Added no graphics monitor feature.

docs/Installation.md

@@ -43,21 +43,21 @@ Consequently, to install and use the ML-Agents Toolkit you will need to:
 strongly recommend that you install Unity through the Unity Hub as it will
 enable you to manage multiple Unity versions.
 
-### Install **Python 3.10.12** or Higher
+### Install **Python 3.10.12**
 
-We recommend [installing](https://www.python.org/downloads/) Python 3.10.
+We recommend [installing](https://www.python.org/downloads/) Python 3.10.12.
 If you are using Windows, please install the x86-64 version and not x86.
 If your Python environment doesn't include `pip3`, see these
 [instructions](https://packaging.python.org/guides/installing-using-linux-tools/#installing-pip-setuptools-wheel-with-linux-package-managers)
 on installing it. We also recommend using [conda](https://docs.conda.io/en/latest/) or [mamba](https://github.com/mamba-org/mamba) to manage your python virtual environments.
 
 #### Conda python setup
 
-Once conda has been installed in your system, open a terminal and execute the following commands to setup a python 3.10 virtual environment
+Once conda has been installed in your system, open a terminal and execute the following commands to setup a python 3.10.12 virtual environment
 and activate it.
 
 ```shell
-conda create -n mlagents python=3.10 && conda activate mlagents
+conda create -n mlagents python=3.10.12 && conda activate mlagents
 ```
 
 ### Clone the ML-Agents Toolkit Repository (Recommended)

docs/Migrating.md

@@ -5,7 +5,7 @@
 TODO: update ml-agents-env package version before release
 --->
 ## Migrating to the ml-agents-envs 0.30.0 package
-- Python 3.10 is now the minimum version of python supported due to [python3.6 EOL](https://endoflife.date/python).
+- Python 3.10.12 is now the minimum version of python supported due to [python3.6 EOL](https://endoflife.date/python).
   Please update your python installation to 3.10.12 or higher.
 - The `gym-unity` package has been refactored into the `ml-agents-envs` package. Please update your imports accordingly.
 - Example:

docs/Python-LLAPI-Documentation.md

@@ -634,22 +634,21 @@ class UnityEnvironment(BaseEnv)
 #### \_\_init\_\_
 
 ```python
- | __init__(file_name: Optional[str] = None, worker_id: int = 0, base_port: Optional[int] = None, seed: int = 0, no_graphics: bool = False, timeout_wait: int = 60, additional_args: Optional[List[str]] = None, side_channels: Optional[List[SideChannel]] = None, log_folder: Optional[str] = None, num_areas: int = 1)
+ | __init__(file_name: Optional[str] = None, worker_id: int = 0, base_port: Optional[int] = None, seed: int = 0, no_graphics: bool = False, no_graphics_monitor: bool = False, timeout_wait: int = 60, additional_args: Optional[List[str]] = None, side_channels: Optional[List[SideChannel]] = None, log_folder: Optional[str] = None, num_areas: int = 1)
 ```
 
 Starts a new unity environment and establishes a connection with the environment.
 Notice: Currently communication between Unity and Python takes place over an open socket without authentication.
 Ensure that the network where training takes place is secure.
 
-:string file_name: Name of Unity environment binary.
-:int base_port: Baseline port number to connect to Unity environment over. worker_id increments over this.
-If no environment is specified (i.e. file_name is None), the DEFAULT_EDITOR_PORT will be used.
-:int worker_id: Offset from base_port. Used for training multiple environments simultaneously.
-:bool no_graphics: Whether to run the Unity simulator in no-graphics mode
-:int timeout_wait: Time (in seconds) to wait for connection from environment.
-:list args: Addition Unity command line arguments
-:list side_channels: Additional side channel for no-rl communication with Unity
-:str log_folder: Optional folder to write the Unity Player log file into.  Requires absolute path.
+:string file_name: Name of Unity environment binary. :int base_port: Baseline port number to connect to Unity
+environment over. worker_id increments over this. If no environment is specified (i.e. file_name is None),
+the DEFAULT_EDITOR_PORT will be used. :int worker_id: Offset from base_port. Used for training multiple
+environments simultaneously. :bool no_graphics: Whether to run the Unity simulator in no-graphics mode :bool
+no_graphics_monitor: Whether to run the main worker in graphics mode, with the remaining in no-graphics mode
+:int timeout_wait: Time (in seconds) to wait for connection from environment. :list args: Addition Unity
+command line arguments :list side_channels: Additional side channel for no-rl communication with Unity :str
+log_folder: Optional folder to write the Unity Player log file into.  Requires absolute path.
 
 <a name="mlagents_envs.environment.UnityEnvironment.close"></a>
 #### close

docs/Tutorial-Custom-Trainer-Plugin.md

@@ -260,7 +260,7 @@ mlagents-learn ml-agents-trainer-plugin/mlagents_trainer_plugin/a2c/a2c_3DBall.y
 ```
 
 ### Validate your implementations:
-Create a clean Python environment with Python 3.10+ and activate it before you start, if you haven't done so already:
+Create a clean Python environment with Python 3.10.12 and activate it before you start, if you haven't done so already:
 ```shell
 conda create -n trainer-env python=3.10.12
 conda activate trainer-env

localized_docs/KR/docs/Installation-Anaconda-Windows.md

@@ -13,7 +13,7 @@ ML-Agents toolkit�� ����ϱ� ����, �Ʒ��� ���
 Windows ������ Anaconda�� [�ٿ�ε�](https://www.anaconda.com/download/#windows)�ϰ� ��ġ�Ͻʽÿ�.
 Anaconda�� ��������ν�, �ٸ� ���� ������ Python�� �и��� ȯ�濡�� ������ �� �ֽ��ϴ�.
 Python 2�� ���̻� �������� �ʱ� ������ Python 3.10�� �ʿ��մϴ�. �� ���̵忡�� �츮��
-Python 3.10 ������ Anaconda 5.1 ������ ����� ���Դϴ�.
+Python 3.10.12 ������ Anaconda 5.1 ������ ����� ���Դϴ�.
 ([64-bit](https://repo.continuum.io/archive/Anaconda3-5.1.0-Windows-x86_64.exe)
 �Ǵ� [32-bit](https://repo.continuum.io/archive/Anaconda3-5.1.0-Windows-x86.exe)
 ��ũ).
@@ -65,11 +65,11 @@ ML-Agents toolkit�� �Բ� ����� ���ο� [Conda ȯ��]
 ��ɾ Ÿ���� �Ͻʽÿ�:
 
 ```sh
-conda create -n ml-agents python=3.10
+conda create -n ml-agents python=3.10.12
 ```
 
 �� ��Ű���� ��ġ�ϱ� ���� �޼����� ���� ��� `y`�� Ÿ�����ϰ� ���͸� �����ʽÿ� _(���ͳ��� ����Ǿ��ִ��� Ȯ���Ͻʽÿ�)_.
-�� �䱸�Ǵ� ��Ű������ �ݵ�� ��ġ�ؾ� �մϴ�. ���ο� Conda ȯ�濡�� Python 3.10 ������ ���Ǹ� ml-agents�� ȣ��˴ϴ�.
+�� �䱸�Ǵ� ��Ű������ �ݵ�� ��ġ�ؾ� �մϴ�. ���ο� Conda ȯ�濡�� Python 3.10.12 ������ ���Ǹ� ml-agents�� ȣ��˴ϴ�.
 
 <p align="center">
   <img src="images/conda_new.PNG" alt="Anaconda Install" width="500" border="10" />

localized_docs/KR/docs/Installation.md

@@ -40,13 +40,13 @@ git clone https://github.com/Unity-Technologies/ml-agents.git
 
 ### 파이썬과 mlagents 패키지 설치
 
-ML-Agents toolkit을 사용하기 위해 [setup.py file](../ml-agents/setup.py)에 나열된 종속성과 함께 파이썬 3.10이 필요합니다.
+ML-Agents toolkit을 사용하기 위해 [setup.py file](../ml-agents/setup.py)에 나열된 종속성과 함께 파이썬 3.10.12이 필요합니다.
 주요 종속성의 일부는 다음을 포함합니다:
 
 - [TensorFlow](Background-TensorFlow.md) (Requires a CPU w/ AVX support)
 - [Jupyter](Background-Jupyter.md)
 
-Python 3.10이 만약 설치되어 있지 않다면, [다운로드](https://www.python.org/downloads/)하고 설치하십시오.
+Python 3.10.12이 만약 설치되어 있지 않다면, [다운로드](https://www.python.org/downloads/)하고 설치하십시오.
 
 만약 당신의 파이썬 환경이 `pip3`을 포함하지 않는다면, 다음
 [지시사항](https://packaging.python.org/guides/installing-using-linux-tools/#installing-pip-setuptools-wheel-with-linux-package-managers)
@@ -64,7 +64,7 @@ pip3 install mlagents
 
 **주의:**
 
-- 현재 Python 3.10 또는 Python 3.5을 지원하지 않습니다.
+- 현재 Python 3.10.12 또는 Python 3.5을 지원하지 않습니다.
 - 만약 Anaconda를 사용하고 TensorFlow에 문제가 있다면, 다음
   [링크](https://www.tensorflow.org/install/pip)에서 Anaconda 환경에서 어떻게 TensorFlow를 설치하는지 확인하십시오.
 ### 개발을 위한 설치방법

localized_docs/RU/docs/Установка.md

@@ -33,7 +33,7 @@ ML-Agents Toolkit состоит из нескольких компоненто
 
 ### Установка **Python 3.10.12** или более поздней версии
 
-Мы рекомендуем [установить](https://www.python.org/downloads/) Python 3.10.
+Мы рекомендуем [установить](https://www.python.org/downloads/) Python 3.10.12
 Если вы используете Windows, пожалуйста, установите x86-64 версию, а не x86.
 Если у вас нет системы управления пакетами `pip3` для Python, пожалуйста,
 воспользуйтесь [этими инструкциями](https://packaging.python.org/guides/installing-using-linux-tools/#installing-pip-setuptools-wheel-with-linux-package-managers)

localized_docs/TR/docs/Installation.md

@@ -27,7 +27,7 @@ ML-Agents Toolkit 'i kurmanız için gerekenler:
 
 ### **Python 3.10.12** veya Daha Yüksek Bir Sürüm Yükleyin
 
-Python 3.10 veya daha yüksek bir sürümü [yüklemenizi](https://www.python.org/downloads/) öneriyoruz. Eğer, Windows kullanıyorsanız, lütfen x86-64 versiyonunu kurun ve asla sadece x86 isimli versiyonu kurmayın. Python ortamınız `pip3` içermiyorsa, [talimatları](https://packaging.python.org/guides/installing-using-linux-tools/#installing-pip-setuptools-wheel-with-linux-package-managers) takip ederek yükleyebilirsiniz.
+Python 3.10.12 veya daha yüksek bir sürümü [yüklemenizi](https://www.python.org/downloads/) öneriyoruz. Eğer, Windows kullanıyorsanız, lütfen x86-64 versiyonunu kurun ve asla sadece x86 isimli versiyonu kurmayın. Python ortamınız `pip3` içermiyorsa, [talimatları](https://packaging.python.org/guides/installing-using-linux-tools/#installing-pip-setuptools-wheel-with-linux-package-managers) takip ederek yükleyebilirsiniz.
 
 Windows'ta Anaconda kurulumu için destek sağlamıyor olsak da,
 önceki [Windows için Anaconda Yüklemesi (Kullanımdan Kaldırılan) Rehberine](Installation-Anaconda-Windows.md) bakabilirsiniz.

ml-agents-envs/mlagents_envs/environment.py

@@ -149,6 +149,7 @@ def __init__(
         base_port: Optional[int] = None,
         seed: int = 0,
         no_graphics: bool = False,
+        no_graphics_monitor: bool = False,
         timeout_wait: int = 60,
         additional_args: Optional[List[str]] = None,
         side_channels: Optional[List[SideChannel]] = None,
@@ -160,19 +161,18 @@ def __init__(
         Notice: Currently communication between Unity and Python takes place over an open socket without authentication.
         Ensure that the network where training takes place is secure.
 
-        :string file_name: Name of Unity environment binary.
-        :int base_port: Baseline port number to connect to Unity environment over. worker_id increments over this.
-        If no environment is specified (i.e. file_name is None), the DEFAULT_EDITOR_PORT will be used.
-        :int worker_id: Offset from base_port. Used for training multiple environments simultaneously.
-        :bool no_graphics: Whether to run the Unity simulator in no-graphics mode
-        :int timeout_wait: Time (in seconds) to wait for connection from environment.
-        :list args: Addition Unity command line arguments
-        :list side_channels: Additional side channel for no-rl communication with Unity
-        :str log_folder: Optional folder to write the Unity Player log file into.  Requires absolute path.
+        :string file_name: Name of Unity environment binary. :int base_port: Baseline port number to connect to Unity
+        environment over. worker_id increments over this. If no environment is specified (i.e. file_name is None),
+        the DEFAULT_EDITOR_PORT will be used. :int worker_id: Offset from base_port. Used for training multiple
+        environments simultaneously. :bool no_graphics: Whether to run the Unity simulator in no-graphics mode :bool
+        no_graphics_monitor: Whether to run the main worker in graphics mode, with the remaining in no-graphics mode
+        :int timeout_wait: Time (in seconds) to wait for connection from environment. :list args: Addition Unity
+        command line arguments :list side_channels: Additional side channel for no-rl communication with Unity :str
+        log_folder: Optional folder to write the Unity Player log file into.  Requires absolute path.
         """
         atexit.register(self._close)
         self._additional_args = additional_args or []
-        self._no_graphics = no_graphics
+        self._no_graphics = no_graphics or no_graphics_monitor and worker_id != 0
         # If base port is not specified, use BASE_ENVIRONMENT_PORT if we have
         # an environment, otherwise DEFAULT_EDITOR_PORT
         if base_port is None:

ml-agents/mlagents/trainers/cli_utils.py

@@ -297,6 +297,15 @@ def _create_parser() -> argparse.ArgumentParser:
         "the graphics driver. Use this only if your agents don't use visual observations.",
     )
 
+    eng_conf.add_argument(
+        "--no-graphics-monitor",
+        default=False,
+        action=DetectDefaultStoreTrue,
+        help="Whether to run the main Unity worker in graphics mode with the remaining workers in no graphics mode"
+        "(i.e. without initializing the graphics driver. Use this only if your agents don't use visual "
+        "observations.",
+    )
+
     torch_conf = argparser.add_argument_group(title="Torch Configuration")
     torch_conf.add_argument(
         "--torch-device",

ml-agents/mlagents/trainers/learn.py

@@ -99,6 +99,7 @@ def run_training(run_seed: int, options: RunOptions, num_areas: int) -> None:
         env_factory = create_environment_factory(
             env_settings.env_path,
             engine_settings.no_graphics,
+            engine_settings.no_graphics_monitor,
             run_seed,
             num_areas,
             env_settings.timeout_wait,
@@ -174,6 +175,7 @@ def write_timing_tree(output_dir: str) -> None:
 def create_environment_factory(
     env_path: Optional[str],
     no_graphics: bool,
+    no_graphics_monitor: bool,
     seed: int,
     num_areas: int,
     timeout_wait: int,
@@ -192,6 +194,7 @@ def create_unity_environment(
             seed=env_seed,
             num_areas=num_areas,
             no_graphics=no_graphics,
+            no_graphics_monitor=no_graphics_monitor,
             base_port=start_port,
             additional_args=env_args,
             side_channels=side_channels,

ml-agents/mlagents/trainers/settings.py

@@ -845,6 +845,7 @@ class EngineSettings:
     target_frame_rate: int = parser.get_default("target_frame_rate")
     capture_frame_rate: int = parser.get_default("capture_frame_rate")
     no_graphics: bool = parser.get_default("no_graphics")
+    no_graphics_monitor: bool = parser.get_default("no_graphics_monitor")
 
 
 @attr.s(auto_attribs=True)

ml-agents/mlagents/trainers/tests/results/ppo/run_logs/training_status.json

@@ -1,7 +1,7 @@
 {
     "metadata": {
         "stats_format_version": "0.3.0",
-        "mlagents_version": "0.31.0.dev0",
-        "torch_version": "1.11.0+cu102"
+        "mlagents_version": "1.1.0.dev0",
+        "torch_version": "2.1.1+cpu"
     }
 }

ml-agents/mlagents/trainers/tests/test_learn.py

@@ -103,6 +103,7 @@ def test_bad_env_path():
         factory = learn.create_environment_factory(
             env_path="/foo/bar",
             no_graphics=True,
+            no_graphics_monitor=False,
             seed=-1,
             num_areas=1,
             timeout_wait=1,
@@ -131,6 +132,7 @@ def test_commandline_args(mock_file):
     assert opt.env_settings.num_envs == 1
     assert opt.env_settings.num_areas == 1
     assert opt.engine_settings.no_graphics is False
+    assert opt.engine_settings.no_graphics_monitor is False
     assert opt.debug is False
     assert opt.env_settings.env_args is None
 
@@ -169,6 +171,33 @@ def test_commandline_args(mock_file):
     assert opt.checkpoint_settings.initialize_from is None  # ignore init if resume set
     assert opt.checkpoint_settings.resume is True
 
+    full_args = [
+        "mytrainerpath",
+        "--env=./myenvfile",
+        "--inference",
+        "--run-id=myawesomerun",
+        "--seed=7890",
+        "--train",
+        "--base-port=4004",
+        "--num-envs=2",
+        "--num-areas=2",
+        "--no-graphics-monitor",
+        "--debug",
+    ]
+
+    opt = parse_command_line(full_args)
+    assert opt.behaviors == {}
+    assert opt.env_settings.env_path == "./myenvfile"
+    assert opt.checkpoint_settings.run_id == "myawesomerun"
+    assert opt.env_settings.seed == 7890
+    assert opt.env_settings.base_port == 4004
+    assert opt.env_settings.num_envs == 2
+    assert opt.env_settings.num_areas == 2
+    assert opt.engine_settings.no_graphics_monitor is True
+    assert opt.debug is True
+    assert opt.checkpoint_settings.inference is True
+    assert opt.checkpoint_settings.resume is False
+
 
 @patch("builtins.open", new_callable=mock_open, read_data=MOCK_PARAMETER_YAML)
 def test_yaml_args(mock_file):