pyhf yield table script

[Daniel-Noel]

Hi,

I attach a script I use for writing out pyhf yield tables to a pdf pyhf_utils.txt.

This uses the fitted parameters and errors from pyhf.infer.mle.fit(
data, model, return_uncertainties=True
) as recommended by @kratsg.

I realise this most likely needs some tidying up, but hopefully this is a start and will be useful to share.

Cheers,
Daniel

[matthewfeickert]

Hi @Daniel-Noel, maybe instead of text files you could make a Gist and just link it here instead?

[Daniel-Noel]

Sure, the file can be found here https://gist.github.com/Daniel-Noel/c4307b7a4355c99240d4dbc6a02913b1

[lukasheinrich]

Thanks @Daniel-Noel -- have you considered using tabulate for the yield tables? this way you can print out tables not only in latex , but also ascii etc (and it will do a lot of the formatting for you)

https://pypi.org/project/tabulate/

[Daniel-Noel]

Hi @lukasheinrich, thanks for the recommendation - I've had a look at using tabulate for the yield tables in https://gist.github.com/Daniel-Noel/b4685a0053c2ac8ee9ffafce20781592 - it seems to have tidied it up quite a bit. I've also added in rounding to the pdg recommendations

[alexander-held]

Hi, great to see this work! I gave the gist a try and it ran fine for me. I did not quite understand the implementation of the yield uncertainty calculation, and I think it is not quite correct. Here is an example for calculating the total model uncertainty per bin (pre- and post-fit), for the post-fit calculation the parameter correlations are needed. A similar calculation can be done to get the per-sample uncertainties as well.

[lukasheinrich]

is this something we can integrate either into pyhf.contrib of cabinetry?

[alexander-held]

Do you mean the yield table creation or the uncertainty calculation? The uncertainty calculation is in cabinetry, but it does not seem unreasonable to move it somewhere else. The yield table functionality is needed for analyzers, so needs to be available through cabinetry as well. I am happy to use an implementation available somewhere else, though for this specific use case I am not sure whether a nice interface can be created that allows sufficient customization over the formatting (which may be needed for analyzers).

[alexander-held]

Maybe one way to factor things is to split out a function that takes (workspace, fit results) and produces (total yield and uncertainty, per-sample yield and uncertainty) for all channels and returns that. There could then be multiple implementations that turn that output into a table, since that part can come down to preference.

[alexander-held]

Update: basic yield table functionality now exists in cabinetry scikit-hep/cabinetry#159, but the interface should be improved a bit. The uncertainty calculations are somewhat expensive for complex models, so the results should be re-used as much as possible when they are needed (e.g. for plots and tables).

[alexander-held]

Hi, I put together an example that also performs uncertainty calculation via bootstrapping in a different context: #1359 (comment). Maybe this can be useful. See also @HDembinski's nice tutorial for more information. The error propagation method via iminuit described in the tutorial can also be applied to pyhf, and is similar (though not exactly equivalent) to the cabinetry implementation. The methods use various approximations and will in general not give the exact same results.