Added $filters to AAS QL & enhanced Fragment Fieldidentifiers#532
Added $filters to AAS QL & enhanced Fragment Fieldidentifiers#532Martin187187 wants to merge 38 commits intoadmin-shell-io:IDTA-01002-3-2_workingfrom
Conversation
…3-2_working Idta 01002 3 2 working
Updated the description of the $filters option in the Query Language section to clarify the filtering of Submodel elements.
Updated query language documentation to include details on filtering out values based on 'HIDE_VALUE' in semanticId.
documentation/IDTA-01002-3/modules/ROOT/partials/bnf/grammar.bnf
Dismissed
Show dismissed
Hide dismissed
documentation/IDTA-01002-3/modules/ROOT/partials/bnf/grammar.bnf
Dismissed
Show dismissed
Hide dismissed
| "USEFORMULA": { | ||
| "type": "string" | ||
| }, | ||
| "FRAGMENT": { |
There was a problem hiding this comment.
There was a significant discrepancy between the security JSON schema and the API JSON schema. In the security specification, the fragment object is nested inside the filter object, but this is not the case in the API schema. I have now updated both documents to use the security version for consistency.
There was a problem hiding this comment.
So is it a bug for 3.0.2?
|
blocked by #548 |
|
Hi @Martin187187 , thanks for addressing the remarks, I will wait for the #548 to be concluded first and then resume the review. |
|
@mdanish98 #548 has been merged. |
|
@Martin187187 is the PR still in "draft" or now in the "ready for review" state? |
I will have to resolve conflicts caused by the bugfixes. So it is currenlty only a draft. |
|
Do we really add |
| } | ||
|
|
||
| ---- | ||
| a| Selects Nameplate submodels and filters out `AddressInformation.Zipcode` data. |
There was a problem hiding this comment.
I have the feeling the semantics is opposite to the semantics of Security for FILTER. See example
https://industrialdigitaltwin.io/aas-specifications/IDTA-01004/v3.0.2/annex/text-access-rule-examples.html#_example_with_filter_statement
For Security it means: "filter" but not "Filter out".
The filter in example of Security says: only display those for which the condition is true. "Filter out", i.e. do not return the other specific Asset IDs.
In your example with condition "false" it is the same semantics (do not display any ZIPCODE because condition not true for any ZIPCODE)
but for HIDE_VALUE, the second example below, it is different: only semanticIds with "HIDE_VALUE" would be returned, all others would not be returned: but you explain it the other way around... so just add "$not"?
Could we add a Submodel example and the corresponding Result similar to the table before?
| Major Changes: | ||
|
|
||
| * ... | ||
| * new: Enhanced the FieldIdentifier Fragment Definition from just plain string to a BNF specification. Extended the query language and security with field-based filters (`$filters` & `FILTERS` clauses) using field identifier fragments, and aligned the OpenAPI schemas and JSON Schema for queries and security rules accordingly. Addition does not introduce breaking changes because it only adds new optional field (https://github.com/admin-shell-io/aas-specs-api/issues/517[#517]) |
There was a problem hiding this comment.
inconsistent, sometimes it is "$filter", sometimes it is "$filters" or it is "$filters" in JSON but $filter in BNF?
| "$condition": { | ||
| "$eq": [ | ||
| { | ||
| "$field": "$sm#semanticId" |
There was a problem hiding this comment.
to be identical with BPN is should be $sm#semanticId.keys[].value, semanticId short cut only means $sm#semanticId.keys[0].value
| ---- | ||
| $sm#semanticId $eq | ||
| "https://admin-shell.io/idta/nameplate/3/0/Nameplate" | ||
| $filter $sme#value $sm#semanticId.keys[].value $eq "HIDE_VALUE" |
There was a problem hiding this comment.
| $filter $sme#value $sm#semanticId.keys[].value $eq "HIDE_VALUE" | |
| $filter $sme#value | |
| $not ($sm#semanticId.keys[].value $eq "HIDE_VALUE") |
| $sm#semanticId $eq | ||
| "https://admin-shell.io/idta/nameplate/3/0/Nameplate" | ||
| $filter $sme#value $sm#semanticId.keys[].value $eq "HIDE_VALUE" | ||
| $filter $sm#semanticId.keys[] $sm#semanticId.keys[].value $eq "HIDE_VALUE" |
There was a problem hiding this comment.
$sm#semanticId instead of $sm#semanticId.keys[] because complet semanticId displayed, not only the keys
| ---- | ||
| $sm#semanticId $eq | ||
| "https://admin-shell.io/idta/nameplate/3/0/Nameplate" | ||
| $filter $sme.AddressInformation.Zipcode false |
There was a problem hiding this comment.
| $filter $sme.AddressInformation.Zipcode false | |
| $filter $sme.AddressInformation.Zipcode | |
| false |
| $sm#semanticId $eq | ||
| "https://admin-shell.io/idta/nameplate/3/0/Nameplate" | ||
| $filter $sme#value $sm#semanticId.keys[].value $eq "HIDE_VALUE" | ||
| $filter $sm#semanticId.keys[] $sm#semanticId.keys[].value $eq "HIDE_VALUE" |
There was a problem hiding this comment.
| $filter $sm#semanticId.keys[] $sm#semanticId.keys[].value $eq "HIDE_VALUE" | |
| $filter $sm#semanticId | |
| $not( $sm#semanticId.keys[].value $eq "HIDE_VALUE" ) |
| "description": "This schema contains all classes that are shared between the AAS Query Language and the AAS Access Rule Language.", | ||
| "$ref": "#/definitions/Root", | ||
| "definitions": { | ||
| "Root": { |
There was a problem hiding this comment.
Root seems not to be shared, root completely different for Query and ACL, no?
| @@ -1 +1 @@ | |||
| .... | |||
There was a problem hiding this comment.
probably it would be better to include a .json file for better checks (same in Part 1 vor Value-Only Schema)
| <grammar> ::= <query> | <AllAccessPermissionRules> | ||
|
|
||
| <query> ::= <selectStatement>? <logicalExpression> | ||
| <query> ::= <selectStatement>? <logicalExpression> ( <ws> "$filter" (<ws> <queryFilterItem> )* )? |
There was a problem hiding this comment.
in your examples in query-language.adoc you use more than one "$filter" statement (with sinlge queryFilterItem), with implicit semantics "AND", here only one "$filter" statement but several queryFilterItems...
| } | ||
|
|
||
| ---- | ||
| a| Selects Nameplate submodels and filters out `AddressInformation.Zipcode` data. |
There was a problem hiding this comment.
I would not use the term "select" because there is no $select statement. A query with no $Select statement returns what exactly?
|
I recommend to add an annex with complete examples like in Security: https://industrialdigitaltwin.io/aas-specifications/IDTA-01004/v3.0.2/annex/text-access-rule-examples.html |
| <Filter> ::= | ||
| "FILTER:" <ws> ( <SecurityQueryFilter> <ws> ) | ||
|
|
||
| <FilterList> ::= | ||
| "FILTERLIST:" <ws> ( <SecurityQueryFilter> <ws> )* |
There was a problem hiding this comment.
What is the difference between a filter and a filter list? in you BNF Example in query-lanauge.adoc there are just two $filter objects in the query, so this is a filterlist?
| <FieldIdentifierSM> ::= "$sm#" <FieldsSM> | ||
| <FieldIdentifierSME> ::= "$sme" ( "." <idShortPath> )? "#" <FieldsSME> | ||
| <FieldIdentifierCD> ::= "$cd#" <FieldsCD> | ||
| <FieldIdentifierAasDescriptor> ::= "$aasdesc#" <AasDescriptorClause> |
There was a problem hiding this comment.
why is it here "Clause" and not "FieldsAasDescriptor" ?
Same for SmDescriptorClause
| <SpecificAssetIdsClauseTail> ::= ".name" | ".value" | <ExternalSubjectIdClause> | ||
|
|
||
| <SpecificAssetIdsClauseFragment> ::= <SpecificAssetIdsClauseHead> <SpecificAssetIdsClauseTailFragment>? | ||
| <SpecificAssetIdsClauseTailFragment> ::= <ExternalSubjectIdClauseFragment> |
There was a problem hiding this comment.
is it because the other two are mandatory that it is not identical to SpecificAssetIdsClause ?
Query Language Filterchapter inQuery Languagechapter.FieldIdentifier FragmentDefinition from just plain string to a BNF specification. => allow non leaf paths and disallow required paths.