Could you add the documentation in this thread to README.md?

[nabenabe0928]

I am using the LCBench in YAHPO right now on a cluster for parallel setup, but I frequently get the error such that I cannot specify the model file path.

Before I go into the detail, I describe my usage:

1. I am running optimization experiments in parallel setup,
2. As child processes/threads in my implementation cannot keep the surrogate model in memory, they are loading the model file every query,
3. Due to the cluster nature, I copy the model file to a temporary directory in each computation node to be used before the run to avoid intensive I/O burden,
4. For this reason, I need to set the data path for each run individually.

Given the context, the problems are:

1. The local config, i.e. ~/.config/yahpo_gym is not protected by file lock for file contamination counter,
2. File contamination obviously causes not found error,
3. However, as each run needs to specify the local config path, we need to either secure the file lock or have unique local configs for each run.

I figured out the solution for this on the current PyPI version (, but not the latest branch version), so let me describe it here and could you please add it to the documentation?

I first call the following and I name this file yahpo_config.py:

# tmp_dir --> the temporary directory for a run
local_config.settings_path = Path(f"{tmp_dir}/.config/yahpo_gym").expanduser().absolute()
local_config.init_config()
local_config.set_data_path(tmp_dir)

Then I call the following in an experiment file named experiment.py:

local_config.settings_path = Path(f"{tmp_dir}/.config/yahpo_gym").expanduser().absolute()

This is needed for the current version because the YAHPO tries to read the YAML file from ~/.config/yahpo_gym otherwise.

Then the whole run looks like the following:

# Copy the LCBench stuff to a temporary directory for quicker loading (if not lcbench, anyways you need to copy the corresponding model file directory)
$ cp -r lcbench $TMPDIR/

# It will create a unique yaml file for this job submission
$ python yahpo_config.py

# Start an experiment
$ python experiment.py --tmp_dir $TMPDIR

Note that I assumed that the temporary directory specified by a run has an environment variable name of TMPDIR.

[sumny]

Hi @nabenabe0928,
thanks for the issue and sorry for the very late reply.
I do agree that it might be sensible to use different local paths when running things highly parallel.
I am currently thinking about incorporating a more direct solution that allows for this.
In #81 I drafted a potential solution requiring only very minimal changes - making use of the fact that the local config currently does not do anything meaningful except for storing a default data path.
In the PR I added a data_path argument that allows you to specify a different data path during construction of a BenchmarkSet.
In you example, data_path could be your tmp_dir that you pass to your experiment.py above.
I believe we can then skip the repeated local config initialization and path overriding.
Maybe you can briefly check this out and give feedback - but I am also open to any other suggestions and or further comments.
Thanks again!