Skip to content

ENH: add schema for "projection" entry in start document #130

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
tacaswell wants to merge 1 commit into
base: main
Choose a base branch
from
Open

ENH: add schema for "projection" entry in start document #130

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 32 additions & 2 deletions event_model/schemas/run_start.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,34 @@
"title" : "data_type",
"patternProperties": {"^([^./]+)$": {"$ref": "#/definitions/data_type"}},
"additionalProperties": false
},
"data_mapping": {
"description": "Where to get the data from",
"type": "object",
"properties" : {
"stream": {"type": "string"},
"location": {"enum" : ["event", "configuration"]},
Copy link
Member

@danielballan danielballan Feb 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What if it's in the stop document? Or elsewhere in the EventDescriptor, such as the source or shape? Would a dotted object representation be simpler and more comprehensive?

Copy link
Contributor Author

@tacaswell tacaswell Apr 21, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Being in the start document is a far more compelling argument.

I'm skeptical of embracing the dot-ness (as we have previously agreed ed that dot access on dicts is not great) so the code to munge that back to something we can actually use will be annoying, but on the other hand we can write the function once and stuff it it databroker).

"field": {"type": "string"}
},
"required" : ["stream", "location", "field"],
"additionalProperties": false
},
"technique": {
Copy link
Member

@danielballan danielballan Feb 11, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The term "technique" might be too limiting. This provides a generic mechanism for mapping any externally-defined metadata schema to the contents of the documents to come. Those schemas might be broken up by experimental technique, by downstream analysis process (applicable to more than one technique such as "scattering" and "diffraction"), by institution, by domain, etc.

Copy link
Contributor Author

@tacaswell tacaswell Apr 22, 2020

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"data_remapping", "datamap", "DateMap", "Application", "application_definitions"?

I think we have a bunch of helper functions like list_applications(h), iter_applications(h, name=None) that yields dicts (?) full of {base types or xarrays}?

"title" : "Describe how to interperet this run as the given technique",
"properties":{
"technique": {"type": "string",
"description": "The name of the technique"},
"version": {"type": "integer",
"description": "The version of the technique spec"},
"configuration" : {"type": "object",
"description": "Static information about technique"},
"data_mapping" : {
"type": "object",
"patternProperties": {"^([^./]+)$": {"$ref": "#/definitions/data_mapping"}}
}
},
"additionalProperties": false,
"required" : ["technique", "version", "configuration", "data_mapping"]
}
},
"properties": {
Expand Down Expand Up @@ -35,6 +63,10 @@
"type": "string",
"description": "Unix owner to associate this data with"
},
"techniques": {
"type": "array",
"items": {"$ref": "#/definitions/technique"}
},
"hints": {
"type": "object",
"description": "Start-level hints",
Expand All @@ -57,12 +89,10 @@
{
"type": "string",
"description": "The stream to find the datakeys in."

}
],
"additionalItems": false,
"minItems": 2

}
}
},
Expand Down
23 changes: 23 additions & 0 deletions event_model/tests/test_em.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -852,3 +852,26 @@ def check_not_filled_factory(name, doc):
rr = event_model.RunRouter([check_filled_factory], reg, fill_or_fail=True)
for name, doc in docs:
rr(name, doc)


def test_techniques():
event_model.schema_validators[event_model.DocumentNames.start].validate(
{
"uid": "abc",
"time": 0,
"techniques": [
{
"technique": "test",
"version": 0,
"configuration": {},
"data_mapping": {
"image": {
"stream": "primary",
"location": "event",
"field": "fccd",
}
},
}
],
}
)