Skip to content

Commit a542451

Browse files
FriedJannikjannisjungaaronzi
authored
Adds Query Language (#814)
* Adds ElasticSearch and first draft of AASQL to ES QueryDSL Converter * New Converter * Converts Hits * Updates Readme * Updates Readme * Refactors Query Core * Refactor * Adds field-to-field comparison (WIP) * Adds missing dependency for ES Client in Component - Adapts README * Refactors HTTP Controller - Fixes field-to-field comparisons - Enables AASQL for AAS Registry Log MongoDB - [WIP] initiates Search Feature for AasRegistry by Jannis Jung Co-authored-by: Jannis Jung <jannis.jung@iese.fraunhofer.de> Co-authored-by: Aaron Zielstorff <aaron.zi@web.de> * Makes QueryResponse explicit * Revert "Makes QueryResponse explicit" This reverts commit f8e0e5d. * Makes indizes Configurable * WIP - adds unit tests * Adds Unit Tests and implements Casting Operators * WIP Adds QL for other modules * WIP * tries to fix indexing of submodels * WIP Adds QL for other modules * Adds Field Renaming * Changes Recursion to Iterative Method * Cahnges CRLF to LF * Adds Querying for nested SME's * Adapts SM Repo QL * Other Search Components Utilize MongoDB now * Adapts Path Parsing * Adapts Path Parsing * Adapts QL * Adapts QL * Intermediate Commit * IC * AAS Registry IT * Adds missing parameter * Fixes docker-compose.yml * Reverts changes in application.properties * Adds missing .env entries - Removes unused files * Adapts output json to new converter format * Adapts application.properties - Adapts RestControllers of Registries * Adapts Autoconfiguration * Adds missing ConditionalOnExpression * Adapts * Adapts * Adapts * Fixes some issues with spring * Adds Timeout (TODO) * Adds missing line * Removes hard-coded thread.sleep and adds awaitility * Adapts limit * Adapts test * Adapts test * Adds Thread.sleep for the moment * Review remarks --------- Co-authored-by: Jannis Jung <jannis.jung@iese.fraunhofer.de> Co-authored-by: Aaron Zielstorff <aaron.zi@web.de>
1 parent 5525663 commit a542451

File tree

160 files changed

+25166
-46
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

160 files changed

+25166
-46
lines changed

‎.github/workflows/examples_test.yml‎

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -189,3 +189,22 @@ jobs:
189189

190190
- name: Stop BaSyx Node-RED Example
191191
run: docker compose -f examples/BaSyxNodeRED/docker-compose.yml down
192+
193+
test-basyx-query-language:
194+
runs-on: ubuntu-latest
195+
name: Test BaSyx Query Language Example
196+
steps:
197+
- uses: actions/checkout@v5
198+
199+
- name: Set up JDK 17
200+
uses: actions/setup-java@v4
201+
with:
202+
java-version: '17'
203+
distribution: 'adopt'
204+
cache: maven
205+
206+
- name: Start BaSyx Query Language Example
207+
run: docker compose -f examples/BaSyxQueryLanguage/docker-compose.yml up -d
208+
209+
- name: Stop BaSyx Query Language Example
210+
run: docker compose -f examples/BaSyxQueryLanguage/docker-compose.yml down

‎.run/AasRegistryLogMongoDB.run.xml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
<component name="ProjectRunConfigurationManager">
2+
<configuration default="false" name="AasRegistryLogMongoDB" type="SpringBootApplicationConfigurationType" factoryName="Spring Boot">
3+
<option name="ACTIVE_PROFILES" value="logEvents,mongoDbStorage" />
4+
<option name="ALTERNATIVE_JRE_PATH" value="21" />
5+
<option name="ALTERNATIVE_JRE_PATH_ENABLED" value="true" />
6+
<module name="basyx.aasregistry-service-release-log-mongodb" />
7+
<option name="SPRING_BOOT_MAIN_CLASS" value="org.eclipse.digitaltwin.basyx.aasregistry.service.OpenApiGeneratorApplication" />
8+
<method v="2">
9+
<option name="Make" enabled="true" />
10+
</method>
11+
</configuration>
12+
</component>

‎.run/SubmodelRegistryLogMongoDB.run.xml‎

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
<component name="ProjectRunConfigurationManager">
2+
<configuration default="false" name="SubmodelRegistryLogMongoDB" type="SpringBootApplicationConfigurationType" factoryName="Spring Boot">
3+
<option name="ACTIVE_PROFILES" value="logEvents,mongoDbStorage" />
4+
<option name="ALTERNATIVE_JRE_PATH" value="21" />
5+
<option name="ALTERNATIVE_JRE_PATH_ENABLED" value="true" />
6+
<module name="basyx.submodelregistry-service-release-log-mongodb" />
7+
<option name="SPRING_BOOT_MAIN_CLASS" value="org.eclipse.digitaltwin.basyx.submodelregistry.service.OpenApiGeneratorApplication" />
8+
<method v="2">
9+
<option name="Make" enabled="true" />
10+
</method>
11+
</configuration>
12+
</component>

‎basyx.aasenvironment/basyx.aasenvironment.component/pom.xml‎

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -92,6 +92,18 @@
9292
<groupId>org.eclipse.digitaltwin.basyx</groupId>
9393
<artifactId>basyx.aasrepository-feature-kafka</artifactId>
9494
</dependency>
95+
<dependency>
96+
<groupId>org.eclipse.digitaltwin.basyx</groupId>
97+
<artifactId>basyx.aasrepository-feature-search</artifactId>
98+
</dependency>
99+
<dependency>
100+
<groupId>org.eclipse.digitaltwin.basyx</groupId>
101+
<artifactId>basyx.submodelrepository-feature-search</artifactId>
102+
</dependency>
103+
<dependency>
104+
<groupId>org.eclipse.digitaltwin.basyx</groupId>
105+
<artifactId>basyx.conceptdescriptionrepository-feature-search</artifactId>
106+
</dependency>
95107
<dependency>
96108
<groupId>org.eclipse.digitaltwin.basyx</groupId>
97109
<artifactId>basyx.submodelrepository-feature-kafka</artifactId>
@@ -174,6 +186,15 @@
174186
<groupId>org.eclipse.digitaltwin.basyx</groupId>
175187
<artifactId>basyx.mongodbcore</artifactId>
176188
</dependency>
189+
<dependency>
190+
<groupId>org.eclipse.digitaltwin.basyx</groupId>
191+
<artifactId>basyx.querycore</artifactId>
192+
</dependency>
193+
<dependency>
194+
<groupId>co.elastic.clients</groupId>
195+
<artifactId>elasticsearch-java</artifactId>
196+
<version>9.0.1</version>
197+
</dependency>
177198
</dependencies>
178199
<build>
179200
<plugins>

‎basyx.aasenvironment/basyx.aasenvironment.component/src/main/resources/application.properties‎

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -101,3 +101,14 @@ basyx.backend = InMemory
101101
####################################################################################
102102
#springdoc.swagger-ui.enabled=false
103103
#springdoc.api-docs.enabled=false
104+
105+
106+
# basyx.aasrepository.feature.search.enabled=true
107+
# basyx.aasrepository.feature.search.indexname=aas-index-test
108+
# basyx.cdrepository.feature.search.enabled=true
109+
# basyx.cdrepository.feature.search.indexname=cd-index-test
110+
# basyx.submodelrepository.feature.search.enabled=true
111+
# basyx.submodelrepository.feature.search.indexname=sm-index-test
112+
# spring.elasticsearch.uris=http://localhost:9200
113+
# spring.elasticsearch.username=elastic
114+
# spring.elasticsearch.password=vtzJFt1b

‎basyx.aasregistry/basyx.aasregistry-feature-search/pom.xml‎

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
<?xml version="1.0" encoding="UTF-8"?>
2+
<project xmlns="http://maven.apache.org/POM/4.0.0"
3+
xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
4+
xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd">
5+
<modelVersion>4.0.0</modelVersion>
6+
<parent>
7+
<groupId>org.eclipse.digitaltwin.basyx</groupId>
8+
<artifactId>basyx.aasregistry</artifactId>
9+
<version>${revision}</version>
10+
</parent>
11+
<artifactId>basyx.aasregistry-feature-search</artifactId>
12+
<name>BaSyx AAS Registry Feature Search</name>
13+
<description>Feature Search for the BaSyx AAS Registry</description>
14+
15+
<dependencies>
16+
17+
<dependency>
18+
<groupId>org.eclipse.digitaltwin.basyx</groupId>
19+
<artifactId>basyx.aasregistry-service</artifactId>
20+
</dependency>
21+
<dependency>
22+
<groupId>org.eclipse.digitaltwin.basyx</groupId>
23+
<artifactId>basyx.querycore</artifactId>
24+
</dependency>
25+
<dependency>
26+
<groupId>co.elastic.clients</groupId>
27+
<artifactId>elasticsearch-java</artifactId>
28+
<version>9.0.1</version>
29+
</dependency>
30+
<dependency>
31+
<groupId>com.fasterxml.jackson.core</groupId>
32+
<artifactId>jackson-core</artifactId>
33+
<version>2.19.0</version>
34+
</dependency>
35+
<dependency>
36+
<groupId>com.fasterxml.jackson.core</groupId>
37+
<artifactId>jackson-annotations</artifactId>
38+
<version>2.19.0</version>
39+
</dependency>
40+
<dependency>
41+
<groupId>com.fasterxml.jackson.core</groupId>
42+
<artifactId>jackson-databind</artifactId>
43+
<version>2.19.0</version>
44+
</dependency>
45+
<dependency>
46+
<groupId>org.springframework</groupId>
47+
<artifactId>spring-context</artifactId>
48+
</dependency>
49+
<dependency>
50+
<groupId>org.eclipse.digitaltwin.basyx</groupId>
51+
<artifactId>basyx.aasregistry-service-basemodel</artifactId>
52+
</dependency>
53+
<dependency>
54+
<groupId>org.eclipse.digitaltwin.basyx</groupId>
55+
<artifactId>basyx.aasregistry-service-mongodb-storage</artifactId>
56+
</dependency>
57+
<dependency>
58+
<groupId>org.awaitility</groupId>
59+
<artifactId>awaitility</artifactId>
60+
<version>4.3.0</version>
61+
<scope>test</scope>
62+
</dependency>
63+
</dependencies>
64+
65+
</project>

‎basyx.aasregistry/basyx.aasregistry-feature-search/src/main/java/org/eclipse/digitaltwin/basyx/aasregistry/feature/search/DisableSearchAasRegistryConfiguration.java‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
/*******************************************************************************
2+
* Copyright (C) 2025 the Eclipse BaSyx Authors
3+
*
4+
* Permission is hereby granted, free of charge, to any person obtaining
5+
* a copy of this software and associated documentation files (the
6+
* "Software"), to deal in the Software without restriction, including
7+
* without limitation the rights to use, copy, modify, merge, publish,
8+
* distribute, sublicense, and/or sell copies of the Software, and to
9+
* permit persons to whom the Software is furnished to do so, subject to
10+
* the following conditions:
11+
*
12+
* The above copyright notice and this permission notice shall be
13+
* included in all copies or substantial portions of the Software.
14+
*
15+
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
16+
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
17+
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
18+
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
19+
* LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
20+
* OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
21+
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
22+
*
23+
* SPDX-License-Identifier: MIT
24+
******************************************************************************/
25+
26+
package org.eclipse.digitaltwin.basyx.aasregistry.feature.search;
27+
28+
import org.springframework.boot.actuate.autoconfigure.data.elasticsearch.ElasticsearchReactiveHealthContributorAutoConfiguration;
29+
import org.springframework.boot.actuate.autoconfigure.metrics.export.elastic.ElasticMetricsExportAutoConfiguration;
30+
import org.springframework.boot.autoconfigure.EnableAutoConfiguration;
31+
import org.springframework.boot.autoconfigure.condition.ConditionalOnExpression;
32+
import org.springframework.boot.autoconfigure.data.elasticsearch.ElasticsearchDataAutoConfiguration;
33+
import org.springframework.boot.autoconfigure.data.elasticsearch.ElasticsearchRepositoriesAutoConfiguration;
34+
import org.springframework.boot.autoconfigure.data.elasticsearch.ReactiveElasticsearchRepositoriesAutoConfiguration;
35+
import org.springframework.boot.autoconfigure.elasticsearch.ElasticsearchClientAutoConfiguration;
36+
import org.springframework.boot.autoconfigure.elasticsearch.ElasticsearchRestClientAutoConfiguration;
37+
import org.springframework.boot.autoconfigure.elasticsearch.ReactiveElasticsearchClientAutoConfiguration;
38+
import org.springframework.context.annotation.Configuration;
39+
40+
/**
41+
* Configuration to prevent Elasticsearch from being injected by Spring
42+
*
43+
*/
44+
@Configuration
45+
@ConditionalOnExpression("!(${" + SearchAasRegistryFeature.FEATURENAME + ".enabled:false} || ${basyx.feature.search.enabled:false})")
46+
@EnableAutoConfiguration(exclude = {
47+
ElasticsearchClientAutoConfiguration.class,
48+
ElasticsearchRepositoriesAutoConfiguration.class,
49+
ElasticsearchDataAutoConfiguration.class,
50+
ElasticsearchRestClientAutoConfiguration.class,
51+
ElasticsearchReactiveHealthContributorAutoConfiguration.class,
52+
ElasticMetricsExportAutoConfiguration.class,
53+
ReactiveElasticsearchClientAutoConfiguration.class,
54+
ReactiveElasticsearchRepositoriesAutoConfiguration.class
55+
})
56+
public class DisableSearchAasRegistryConfiguration {
57+
}

‎basyx.aasregistry/basyx.aasregistry-feature-search/src/main/java/org/eclipse/digitaltwin/basyx/aasregistry/feature/search/SearchAasRegistryApiHTTPController.java‎

Lines changed: 103 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,103 @@
1+
/*******************************************************************************
2+
* Copyright (C) 2025 the Eclipse BaSyx Authors
3+
*
4+
* Permission is hereby granted, free of charge, to any person obtaining
5+
* a copy of this software and associated documentation files (the
6+
* "Software"), to deal in the Software without restriction, including
7+
* without limitation the rights to use, copy, modify, merge, publish,
8+
* distribute, sublicense, and/or sell copies of the Software, and to
9+
* permit persons to whom the Software is furnished to do so, subject to
10+
* the following conditions:
11+
*
12+
* The above copyright notice and this permission notice shall be
13+
* included in all copies or substantial portions of the Software.
14+
*
15+
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
16+
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
17+
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
18+
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
19+
* LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
20+
* OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
21+
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
22+
*
23+
* SPDX-License-Identifier: MIT
24+
******************************************************************************/
25+
26+
package org.eclipse.digitaltwin.basyx.aasregistry.feature.search;
27+
import co.elastic.clients.elasticsearch.ElasticsearchClient;
28+
import com.fasterxml.jackson.databind.node.ObjectNode;
29+
import org.eclipse.digitaltwin.basyx.aasregistry.model.AssetAdministrationShellDescriptor;
30+
import org.eclipse.digitaltwin.basyx.aasregistry.service.storage.AasRegistryStorage;
31+
import org.eclipse.digitaltwin.basyx.http.pagination.Base64UrlEncodedCursor;
32+
import org.eclipse.digitaltwin.basyx.querycore.query.model.AASQuery;
33+
import org.eclipse.digitaltwin.basyx.querycore.query.model.QueryResponse;
34+
import org.eclipse.digitaltwin.basyx.querycore.query.executor.ESQueryExecutor;
35+
import org.slf4j.Logger;
36+
import org.slf4j.LoggerFactory;
37+
import org.springframework.beans.factory.annotation.Autowired;
38+
import org.springframework.beans.factory.annotation.Value;
39+
import org.springframework.boot.autoconfigure.condition.ConditionalOnExpression;
40+
import org.springframework.boot.autoconfigure.condition.ConditionalOnProperty;
41+
import org.springframework.http.HttpStatus;
42+
import org.springframework.http.ResponseEntity;
43+
import org.springframework.web.bind.annotation.RestController;
44+
45+
import java.io.IOException;
46+
import java.util.ArrayList;
47+
import java.util.List;
48+
49+
@jakarta.annotation.Generated(value = "io.swagger.codegen.v3.generators.java.SpringCodegen", date = "2025-06-18T09:42:17.580283867Z[GMT]")
50+
@ConditionalOnExpression("#{${" + SearchAasRegistryFeature.FEATURENAME + ".enabled:false} or ${basyx.feature.search.enabled:false}}")
51+
@RestController
52+
public class SearchAasRegistryApiHTTPController implements SearchAasRegistryHTTPApi {
53+
54+
private static final Logger log = LoggerFactory.getLogger(SearchAasRegistryApiHTTPController.class);
55+
56+
private final ElasticsearchClient client;
57+
private final AasRegistryStorage backend;
58+
59+
@Value("${" + SearchAasRegistryFeature.FEATURENAME + ".indexname:" + SearchAasRegistryFeature.DEFAULT_INDEX + "}")
60+
private String indexName;
61+
62+
@Autowired
63+
public SearchAasRegistryApiHTTPController(ElasticsearchClient client, AasRegistryStorage backend) {
64+
this.client = client;
65+
this.backend = backend;
66+
}
67+
68+
public ResponseEntity<QueryResponse> queryAssetAdministrationShellDescriptors(Integer limit, Base64UrlEncodedCursor cursor, AASQuery query) {
69+
QueryResponse queryResponse;
70+
try {
71+
if (query.get$select() != null && query.get$select().equals("id")) {
72+
queryResponse = getQueryResponse(query, limit, cursor);
73+
} else {
74+
// Hard Code to only retrieve ids -> Fetching the actual AAS Descs from MongoDB
75+
query.set$select("id");
76+
queryResponse = getQueryResponse(query, limit, cursor);
77+
queryResponse.paging_metadata.resulType = "AssetAdministrationShellDescriptor";
78+
List<AssetAdministrationShellDescriptor> aasDescs = new ArrayList<>();
79+
for (Object id : queryResponse.result) {
80+
String identifier = ((ObjectNode) id).get("id").asText();
81+
AssetAdministrationShellDescriptor aasDesc = backend.getAasDescriptor(identifier);
82+
aasDescs.add(aasDesc);
83+
84+
}
85+
queryResponse.result = aasDescs.stream()
86+
.map(aasDesc -> (Object) aasDesc)
87+
.toList();
88+
}
89+
} catch (IOException e) {
90+
throw new RuntimeException(e);
91+
}
92+
93+
return new ResponseEntity<>(queryResponse, HttpStatus.OK);
94+
}
95+
96+
private QueryResponse getQueryResponse(AASQuery query, Integer limit, Base64UrlEncodedCursor cursor) throws IOException {
97+
QueryResponse queryResponse;
98+
ESQueryExecutor executor = new ESQueryExecutor(client, indexName, "AssetAdministrationShellDescriptor");
99+
queryResponse = executor.executeQueryAndGetResponse(query, limit, cursor);
100+
return queryResponse;
101+
}
102+
103+
}

‎basyx.aasregistry/basyx.aasregistry-feature-search/src/main/java/org/eclipse/digitaltwin/basyx/aasregistry/feature/search/SearchAasRegistryConfiguration.java‎

Lines changed: 35 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,35 @@
1+
/*******************************************************************************
2+
* Copyright (C) 2025 the Eclipse BaSyx Authors
3+
*
4+
* Permission is hereby granted, free of charge, to any person obtaining
5+
* a copy of this software and associated documentation files (the
6+
* "Software"), to deal in the Software without restriction, including
7+
* without limitation the rights to use, copy, modify, merge, publish,
8+
* distribute, sublicense, and/or sell copies of the Software, and to
9+
* permit persons to whom the Software is furnished to do so, subject to
10+
* the following conditions:
11+
*
12+
* The above copyright notice and this permission notice shall be
13+
* included in all copies or substantial portions of the Software.
14+
*
15+
* THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND,
16+
* EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
17+
* MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
18+
* NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
19+
* LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
20+
* OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
21+
* WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE SOFTWARE.
22+
*
23+
* SPDX-License-Identifier: MIT
24+
******************************************************************************/
25+
26+
package org.eclipse.digitaltwin.basyx.aasregistry.feature.search;
27+
28+
import org.springframework.boot.autoconfigure.condition.ConditionalOnExpression;
29+
import org.springframework.context.annotation.Configuration;
30+
31+
@ConditionalOnExpression("#{${" + SearchAasRegistryFeature.FEATURENAME + ".enabled:false} or ${basyx.feature.search.enabled:false}}")
32+
@Configuration
33+
public class SearchAasRegistryConfiguration {
34+
35+
}

0 commit comments

Comments
 (0)