parameter names / existence for bin-wise holes

[lukasheinrich]

I wanted to make an actual discussion out of #1639

for modifiers that assign bin-wise NPs ("gammas") there are cases where the underlying histogram data is such that some bins yield invalid daat (e.g. NaN)

These NPs cannot be part of the lhood and there are a few strategies

1. remove the NPs and label the remaning ones sequentially a[0],a[1],a[2]
2. remove the NPs and label the remaning ones by bin nr a[0],a[4],a[5]
3. keep the NPs (sequential and binnr naming are the same) but keep NPs forinvalid bins constant

option 3) would guarantee that lilkelihoods details in a common grid are not sensitive to histogram detalis (that can be stochastic, like empty bins) but could inflate the number of parameters.. but setting those "forced-to-be-fixed" to floating is undefined
1,2) would induce variabl-npars likeleihoods within a "grid" for a dubious efficiency gain, "purity" in parameter

we currently do 1) but maybe w should do 3) and just provide error messages if someone tries to "release" those variables? Seems to me 1,2 don't really provide sufficient upside

cc @kratsg @alexander-held @matthewfeickert

[alexander-held]

I could see both 2) and 3) working well, with a very slight preference for 3) at the moment. In my opinion 1) is not a good option: the change in meaning of e.g. a[2] depending on how many parameters in bins to the left are pruned is unintuitive.

With 2), users may wonder why parameters for some bins are missing. The access of pruned parameters would also have to be handled (see also #1637). On the other hand, the shapesys / staterror modifiers also create all relevant parameters by hand automatically right now. While it may be easier to understand when all of them are created (and none pruned), the system does not require any user intervention right now either — it just automatically does what is needed. Adding some more "magic" to that feels like it would fit in with the design thus far.

Version 3) seems nice for consistency, and being able to know shapes ahead of time without parsing details of the config (like whether there are empty bins) seems useful as well. Parameters set to fixed could be easily skipped in printouts / visualization, and access of them (trying to set them to non-constant) could be handled to inform users that this may not be a good idea. It is not clear to me whether setting them to non-constant would make any sense: I assume the likelihood would become NaN? Similarly, I assume that changing their auxdata could cause issues.

I wonder whether for 3) there could be another flag attached to such fixed parameters to indicate why they are fixed, and to distinguish them from parameters that could in principle be set to float without causing issues (those usually would be constant due to the measurement config saying they should be). This distinction would be useful for debugging purposes. If the parameters do not carry a flag, an API to return the affected parameters could work instead.

I view the issue with 3) of resulting in more parameters as rather small: usually this should affect few bins regardless, so the change in the number of parameters should usually be small.

[lukasheinrich]

I think actually 3) is considerably easier to implement than 1) and 2) since it doesn't have to handle "holes"

[kratsg]

Indeed, 3 is a bit easier.. but 2 is possible if we pass around the masking... Because a combination of slice + mask works.. or we update parameter set slices to only slice out the relevant NPs that are being used.

[lukasheinrich]

a slight complication is whether to add constraint terms for these superfluous variables or not .. but once we gain ability to easily add remove them, I guess that resolves themselves

[lukasheinrich]

also this would require component-wise fixed paramteres (rather than paramset-wise

[alexander-held]

The constraint terms are a good point: this would need to be communicated clearly to avoid misunderstandings. For the comparison of NLLs between different models it matters whether they are removed or not.