Issue with optuna sweeper with list parameters?

[KirillShmilovich]

Wondering if anyone has successfully ran the Optuna hparams sweeper using lists within the choice of params?

For example, with otherwise default parameters, something like this:

params:
      model.optimizer.lr: interval(0.0001, 0.1)
      model.optimizer.weight_decay: choice(0.0, 0.001, 0.00001, 0.0000001)
      ...
      model.net.n_rows_fc: choice("[128, 32]", "[128]", "[64, 32]", "[64]")
      model.net.dropouts: choice("[0.0, 0.0, 0.0]", "[0.25, 0.25, 0.25]", "[0.4, 0.4, 0.4]", "[0.5, 0.5, 0.5]")

Yields this:

Traceback (most recent call last):                                                                                                                                                                                                            
  File "/homefs/home/shmilovk/micromamba/envs/abs_binding_affinity/lib/python3.10/site-packages/hydra/_internal/utils.py", line 220, in run_and_report                                                                                        
    return func()                                                                                                                                                                                                                             
  File "/homefs/home/shmilovk/micromamba/envs/abs_binding_affinity/lib/python3.10/site-packages/hydra/_internal/utils.py", line 466, in <lambda>                                                                                              
    lambda: hydra.multirun(                                                                                                                                                                                                                   
  File "/homefs/home/shmilovk/micromamba/envs/abs_binding_affinity/lib/python3.10/site-packages/hydra/_internal/hydra.py", line 162, in multirun                                                                                              
    ret = sweeper.sweep(arguments=task_overrides)                                                                                                                                                                                             
  File "/homefs/home/shmilovk/micromamba/envs/abs_binding_affinity/lib/python3.10/site-packages/hydra_plugins/hydra_optuna_sweeper/optuna_sweeper.py", line 52, in sweep                                                                      
    return self.sweeper.sweep(arguments)                                                                                                                                                                                                      
  File "/homefs/home/shmilovk/micromamba/envs/abs_binding_affinity/lib/python3.10/site-packages/hydra_plugins/hydra_optuna_sweeper/_impl.py", line 391, in sweep                                                                              
    raise e                                                                                                                                                                                                                                   
  File "/homefs/home/shmilovk/micromamba/envs/abs_binding_affinity/lib/python3.10/site-packages/hydra_plugins/hydra_optuna_sweeper/_impl.py", line 357, in sweep                                                                              
    values = [float(ret.return_value)]                                                                                                                                                                                                        
  File "/homefs/home/shmilovk/micromamba/envs/abs_binding_affinity/lib/python3.10/site-packages/hydra/core/utils.py", line 260, in return_value                                                                                               
    raise self._return_value                                                                                                                                                                                                                  
RuntimeError: Trying to create tensor with negative dimension -64: [32979, -64]

Strangely, the initial startup trials execute without error -- however the followup job executed by Optuna immediately fail.