Add load_field and load_group to allow loading custom entries using GenericNeXusWorkflow

[nvaytet]

Example (from the tests):

class UserAffiliation(sl.Scope[RunType, str], str):
    """User affiliation."""

def load_user_affiliation(
    file: NeXusFileSpec[RunType],
    path: NeXusName[UserAffiliation[RunType]],
) -> UserAffiliation[RunType]:
    return UserAffiliation[RunType](load_field(file, path))

wf = GenericNeXusWorkflow(run_types=[SampleRun], monitor_types=[])
wf.insert(load_user_affiliation)
wf[Filename[SampleRun]] = loki_tutorial_sample_run_60250
wf[NeXusName[UserAffiliation[SampleRun]]] = '/entry/user_0/affiliation'
affiliation = wf.compute(UserAffiliation[SampleRun])
assert affiliation == 'ESS'

[nvaytet]

I could not figure out how to just have load_component(location, nx_class=None) and (below) wf[NeXusName[UserAffiliation]] = '/entry/user_0/affiliation'...

I'm running into the error ValueError: Expected a NeXus group as item '/entry/title' but got a field.
Help welcome!

[SimonHeybrock]

I am not sure I understand the approach at all: If we anyway add a custom "load" provider, why was the generic NeXus workflow modified? What role does it still play? Is it the location-spec mechanism?

Regarding the error, could it be related that load_component looks only within NXinstrument (except for the sample)? Is there some search logic that needs to be generalized?

[nvaytet]

Is it the location-spec mechanism

Yes, I thought we needed to do things this way to re-use the load_component properly.
If there is a less invasive way to just add a provider and not modify the generic workflow, then it would be great if you can show me?
thanks

[SimonHeybrock]

No that's fine... provided we can actually re-use load_component?

[nvaytet]

Regarding the error, could it be related that load_component looks only within NXinstrument (except for the sample)?

Sorry, the error message was confusing. I copied the wrong message (from a previous iteration).
The error is actually
ValueError: Expected a NeXus group as item '/entry/user_0/affiliation' but got a field.

So it is because load_component expects to load a group and not a field.

[SimonHeybrock]

load_component was made for any "physical component" (it handles positions, etc.). Say maybe it is reasonable and by design that it fails? Which brings us back to the question of what you gain by trying to uses the existing machinery instead of having a simple load provider?

[nvaytet]

I really don't mind either way, I could not figure out how to properly just add a simple load provider which would hook into the part of the workflow that opens the file for reading.

I thought that using the Filename[SampleRun] as input to the provider would mean we open the file multiple times, and I wanted to avoid that.
So I would welcome some pointers as to what the provider would take in as input so it can load a custom field. Thanks in advance 🙂

[SimonHeybrock]

Why not use the same inputs that load_component does, but in a reusable provider (load_field or load_group?)?

[nvaytet]

Not sure why we needed to have the monitor_types here, when we are loading detectors?
So I removed it.

[nvaytet]

I decided to add the selection as an arg here instead of going the NeXusComponentLocationSpec way as it would have required creating a new TypeVar which was not only for Component, and then creating new types like NeXusEntryLocationSpec and also some new

def entry_spec_by_name(
    filename: NeXusFileSpec[RunType], name: NeXusName[EntryTypeVar]
) -> NeXusEntryLocationSpec[EntryTypeVar, RunType]:

(the analog to component_spec_by_name: https://github.com/scipp/essreduce/blob/main/src/ess/reduce/nexus/workflow.py#L89).

The current approach is less invasive.
But I don't mind implementing the above if there would be a use for it?

Right now, since the load_field would be called inside a custom providers, and selection parameters could simple be added to that provider, e.g.

def load_custom_entry(
    file: NeXusFileSpec[RunType],
    path: NeXusName[MyEntry[RunType]],
    start: MyEntryRangeStart[RunType],
    end: MyEntryRangeEnd[RunType]
) -> MyEntry[RunType]:
    return MyEntry[RunType](load_field(file, path, selection=slice(start, end)))

Not sure what other cases could be needed?