Can't use multiple --try-amend statements in Easystack entry

[cgross95]

We install MATLAB using something like

eb MATLAB-2023b.eb \
    --try-amend="license_server=hostname" \
    --try-amend="license_server_port=12345" \
    --try-amend="key=abc-123"

But the equivalent Easystack will only use the last value, e.g.,

easyconfigs:
- MATLAB-2023b.eb:
    options:
      try-amend: "license_server=hostname"
      try-amend: "license_server_port=12345"
      try-amend: "key=abc-123"

gives a tweaked EasyConfig that only has key=abc123 at the bottom. license_server and license_server_port are ignored.

[Crivella]

I think this is more a YAML dictionary thing than an easybuild one.
When writing a dictionary in YAML with repeated keys, the last one will be the one considered.

Though it seem that setting it as a list does not lead to the expected behavior either

easyconfigs:
- MATLAB-2023b.eb:
    options:
      try-amend: 
        - "license_server=hostname"
        - "license_server_port=12345"
        - "key=abc-123"

== 2024-10-04 15:32:48,168 options.py:1989 DEBUG Converting dictionary {'try-amend': ['license_server=hostname', 'license_server_port=12345', 'key=abc-123']} to argument list
== 2024-10-04 15:32:48,168 options.py:2008 DEBUG Converted dictionary {'try-amend': ['license_server=hostname', 'license_server_port=12345', 'key=abc-123']} to argument list ['--try-amend=license_server=hostname,license_server_port=12345,key=abc-123']

which leads to the following in the modified EC

...
license_server = [
    'hostname',
    'license_server_port=12345',
    'key=abc-123',
]
...

[ocaisa]

What about trying

easyconfigs:
- MATLAB-2023b.eb:
    options:
      try-amend: 
        - "license_server=hostname\nlicense_server_port=12345\nkey=abc-123"

as a hack?

We could probably do something specific for try-amend when doing the dict to list argument conversion, I suspect there are very few options that we allow to appear multiple times.

[Crivella]

That seem to result to it being assigned as a multi-line string

license_server = """hostname
license_server_port=12345
key=abc-123"""

[ocaisa]

There seem to be only two options that we allow to specify multiple times: --amend and --try-amend. We can try to add these special cases to the conversion in

easybuild-framework/easybuild/tools/options.py

Lines 1996 to 2025 in 8ff6ba0

	def opts_dict_to_eb_opts(args_dict):
	"""
	Convert a dictionary with configuration option values to command-line options for the 'eb' command.
	Can by used to convert e.g. easyconfig-specific options from an easystack file to a list of strings
	that can be fed into the EasyBuild option parser
	:param args_dict: dictionary with configuration option values
	:return: a list of strings representing command-line options for the 'eb' command
	"""
	_log.debug("Converting dictionary %s to argument list" % args_dict)
	args = []
	for arg in sorted(args_dict):
	if len(arg) == 1:
	prefix = '-'
	else:
	prefix = '--'
	option = prefix + str(arg)
	value = args_dict[arg]
	if isinstance(value, (list, tuple)):
	value = ','.join(str(x) for x in value)
	if value in [True, None]:
	args.append(option)
	elif value is False:
	args.append('--disable-' + option[2:])
	elif value is not None:
	args.append(option + '=' + str(value))
	_log.debug("Converted dictionary %s to argument list %s" % (args_dict, args))
	return args

From what @Crivella said, our only option to handle this is to use:

easyconfigs:
- MATLAB-2023b.eb:
    options:
      try-amend: 
        - "license_server=hostname"
        - "license_server_port=12345"
        - "key=abc-123"

[Crivella]

Opened a PR with a possible fix:

• allow usingamend/try-amend multiple times in an easystack file entry #4667

There is a potential breaking change in doing this for existing easystacks that should be evaluated.
Though i think since now it seems impossible to simulate multiple option passes, this is still a required change