@@ -39,6 +39,9 @@ the docker image.
3939Spark Connect is also running in this container and can be used to easily issue queries to run. The port for
4040Spark Connect is exposed to the host OS.
4141
42+ Spark is configured to have an OpenSearch datastore with the catalog name ` dev ` . Indices on the OpenSearch
43+ server can be queries as ` dev.default.<INDEX_NAME> ` .
44+
4245### Spark Worker
4346
4447The Spark worker node. It is configured to use an external Hive metastore in the container ` metastore ` . The
@@ -68,7 +71,7 @@ It will contain the S3 tables data.
6871
6972This container also has a docker volume used to persist the S3 data.
7073
71- ### Configuration-Updated
74+ ### Configuration-Updater
7275
7376A temporary container that is used to configure the OpenSearch and Minio containers. It is run after both
7477of those have started up. For Minio, it will add the ` integ-test ` bucket and create an access key. For
@@ -109,6 +112,26 @@ spark.sql("CREATE EXTERNAL TABLE foo (id int, name varchar(100)) location 's3a:/
109112spark.sql(" INSERT INTO foo (id, name) VALUES(1, 'Foo')" )
110113```
111114
115+ A REST call to the OpenSearch container can be used to query the table using the Async API.
116+ ``` shell
117+ curl \
118+ -u ' admin:C0rrecthorsebatterystaple.' \
119+ -X POST \
120+ -H ' Content-Type: application/json' \
121+ -d ' {"datasource": "mys3", "lang": "sql", "query": "SELECT * FROM mys3.default.foo"}' \
122+ http://localhost:9200/_plugins/_async_query
123+ ```
124+
125+ When the query is finished, the results can be retrieved with a REST call to the OpenSearch container.
126+ ``` shell
127+ curl \
128+ -u ' admin:C0rrecthorsebatterystaple.' \
129+ -X POST \
130+ -H ' Content-Type: application/json' \
131+ -d ' {}' \
132+ ' http://localhost:9200/query_execution_result_mys3/_search?pretty'
133+ ```
134+
112135## Configuration of the Cluster
113136
114137There are several settings that can be adjusted for the cluster.
0 commit comments