Improve (better arguments validation, avoiding repeated creation of validator objects, etc) _validate_obj_json() in metadata.py and supporting funcs#278
Conversation
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #278 +/- ##
==========================================
- Coverage 97.63% 93.24% -4.40%
==========================================
Files 16 16
Lines 1775 1983 +208
==========================================
+ Hits 1733 1849 +116
- Misses 42 134 +92
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
d1bebc7 to
6bf8109
Compare
_validate_obj_json() in metadata.py and provide utility funcs for validating with JSON schema_validate_obj_json() in metadata.py and provide utility funcs for validating against JSON schemas
8f6c760 to
d4d802f
Compare
| ) | ||
|
|
||
| # Ensure the schema is valid | ||
| validator_cls.check_schema(schema) |
There was a problem hiding this comment.
wouldn't that be called for the same schema over and over again for every instance?
A util func for validating a metadata instance of a DANDI model against the JSON schema of the model
Additionally, replace relative import with absolute import in `test_util.py` The relative import was referring to an ancestor. See https://docs.astral.sh/ruff/rules/relative-imports/
Improve type annotations. Make boolean parameter keyword parameter. Make it more correct and robust by using `validate_json()` underneath.
d4d802f to
b6dbb83
Compare
Refactor code to allow caching of jsonschema validators. Specifically, `_get_jsonschema_validator()` caches validators for schemas downloaded from the `dandi/schema` repo, and `_get_jsonschema_validator_local()` caches validator for schemas generated from locally defined Pydantic models. In this refactoring, additional appropriate validations of arguments to involved funcs are added.
_validate_obj_json() in metadata.py and provide utility funcs for validating against JSON schemas_validate_obj_json() in metadata.py and supporting funcs
|
@yarikoptic I think the code is good now. However, I did quite a bit of validation of function args. I could have used |
_validate_obj_json() in metadata.py and supporting funcs_validate_obj_json() in metadata.py and supporting funcs
This PR make the following improvement relating to validating metadata instances against a DANDI model's JSON schema.
_validate_obj_json()so that it take a jsonschema validator instead of a JSON schema as argument to avoid recreation of validators for the same schema."$schema"property in a provide schema to determine the appropriate validator classvalidate_json(), to validate a data instance in general, without themissing_okparameter.dandi_jsonschema_validator, that decides the default/fallback validator class to used based on the"schemaVersion"property of the given schema instead of the"schemaVersion"field of a given instance. (A bug fix)_get_jsonschema_validator()and_get_jsonschema_validator_local(), with cache to avoid repeated creation of jsonschema validators for the same schemas. Note:functools.cache()instead offunctools.lru_cache()is used, for improved performance, in these functions because valid inputs to these functions have been limited by validation, so there is no worry about the cache growing without bound._validate_obj_json()using the aforementioned utility functionsschema_mapinmetadata.pytoSCHEMA_MAPfor the reference dictionary is a constant.