Skip to content

Commit a570e67

Browse files
Updated the integ-test docker README file
Signed-off-by: Norman Jordan <norman.jordan@improving.com>
1 parent 5d40352 commit a570e67

File tree

1 file changed

+24
-1
lines changed

1 file changed

+24
-1
lines changed

docs/docker/integ-test/README.md

Lines changed: 24 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -39,6 +39,9 @@ the docker image.
3939
Spark Connect is also running in this container and can be used to easily issue queries to run. The port for
4040
Spark Connect is exposed to the host OS.
4141

42+
Spark is configured to have an OpenSearch datastore with the catalog name `dev`. Indices on the OpenSearch
43+
server can be queries as `dev.default.<INDEX_NAME>`.
44+
4245
### Spark Worker
4346

4447
The Spark worker node. It is configured to use an external Hive metastore in the container `metastore`. The
@@ -68,7 +71,7 @@ It will contain the S3 tables data.
6871

6972
This container also has a docker volume used to persist the S3 data.
7073

71-
### Configuration-Updated
74+
### Configuration-Updater
7275

7376
A temporary container that is used to configure the OpenSearch and Minio containers. It is run after both
7477
of those have started up. For Minio, it will add the `integ-test` bucket and create an access key. For
@@ -109,6 +112,26 @@ spark.sql("CREATE EXTERNAL TABLE foo (id int, name varchar(100)) location 's3a:/
109112
spark.sql("INSERT INTO foo (id, name) VALUES(1, 'Foo')")
110113
```
111114

115+
A REST call to the OpenSearch container can be used to query the table using the Async API.
116+
```shell
117+
curl \
118+
-u 'admin:C0rrecthorsebatterystaple.' \
119+
-X POST \
120+
-H 'Content-Type: application/json' \
121+
-d '{"datasource": "mys3", "lang": "sql", "query": "SELECT * FROM mys3.default.foo"}' \
122+
http://localhost:9200/_plugins/_async_query
123+
```
124+
125+
When the query is finished, the results can be retrieved with a REST call to the OpenSearch container.
126+
```shell
127+
curl \
128+
-u 'admin:C0rrecthorsebatterystaple.' \
129+
-X POST \
130+
-H 'Content-Type: application/json' \
131+
-d '{}' \
132+
'http://localhost:9200/query_execution_result_mys3/_search?pretty'
133+
```
134+
112135
## Configuration of the Cluster
113136

114137
There are several settings that can be adjusted for the cluster.

0 commit comments

Comments
 (0)