Question about definition, naming and correlation of custom modifiers

[itsaklid]

Hello!

I'm working on a Belle II analysis that requires custom expressions of modifiers and have been using the branch to be merged in #1991

Another discussion that I have found particularly useful is this one as I'm also using cabinetry to create my model.

I've noticed that when defining multiple expressions that should be common among different channels I get the following error
InvalidModel: Trying to add paramset custom/custom_modifier on signal sample in SR2 channel but other paramsets exist with the same name.

I'm pasting below a minimal example to reproduce the error

import pyhf
from pyhf.experimental.modifiers import add_custom_modifier

new_params = {
    "x": {"inits": (1.0,), "bounds": ((-5.0,5.0),)},
}
expanded_pyhf = add_custom_modifier("custom", ["x"], new_params)

model = pyhf.Model(
{
    "channels": [
        {
            "name": "SR1",
            "samples": [
                {
                    "name": "signal",
                    "data": [10, 8],
                    "modifiers": [{"name": "custom_modifier", "type": "custom", "data": {"expr": "1-x"}}],
                },
                {
                    "name": "bkg",
                    "data": [3, 12],
                    "modifiers": []
                },
            ],
        },
         {
            "name": "SR2",
            "samples": [
                {
                    "name": "signal",
                    "data": [5, 3],
                    "modifiers": [{"name": "custom_modifier", "type": "custom", "data": {"expr": "1-x"}}],
                },
                {
                    "name": "bkg",
                    "data": [2, 8],
                    "modifiers": []
                },
            ],
        }
    ]
},
    modifier_set=expanded_pyhf,
    poi_name='',
    validate=False,
    batch_size = None
)

print(model.expected_data([0]))

Of course one can simply use different names for the new modifiers which works without pyhf complaining.

However since I'm not sure what exactly happens under the hood it made me wonder whether this is a sub-optimal setup. My current understanding is that, in pyhf, correlation is only guaranteed when modifiers share both the same name and definition. Does using independent names for the custom modifiers -even with identical expressions- lead to pyhf minimizing all these expressions independently from each other even if all of them are functions of the same parameter? Since I cannot build a model where the custom expressions share the same name, I cannot compare results in the two scenarios, therefore I thought of asking the experts here for some insights.

If this is the case is there an existing recommended approach for sharing a single custom expression across multiple samples/channels while maintaining correlation, or would this require some changes in #1991?

Thanks a lot in advance for the help!

Cheers,
Ilias

[alexander-held]

Hi, when the parameters have different names they should also end up being uncorrelated, I think you would see that model.expected_data now expects an additional parameter to be provided. I am not sure how to accomplish what you are after with the current state of #1991. This also sounds a bit like #2350. One thing that I would suggest looking into is #2579 as an alternative implementation. I do not remember if I have tried doing the same with that newer pull request but I think it would be worth an attempt.

[itsaklid]

Thanks a lot for the prompt reply!

I'll give #2579 a try then! I'll also inquire in #1991 about the status and whether I should abandon hope on that implementation in the foreseeable future.

I gave #2350 a read and I totally agree with the part on Understanding the implementation conceptually / potential streamlining The setup of #1991 is a bit confusing in terms of terminology for a user so #2350
gets a +1 from me 🙂

Thanks again for your help!