Skip to content

Commit 8aa35ed

Browse files
committed
update trino-iceberg demo documentation
1 parent 9e66a18 commit 8aa35ed

File tree

1 file changed

+22
-106
lines changed

1 file changed

+22
-106
lines changed

docs/modules/demos/pages/trino-iceberg.adoc

Lines changed: 22 additions & 106 deletions
Original file line numberDiff line numberDiff line change
@@ -46,42 +46,43 @@ This demo will
4646

4747
== List the deployed Stackable services
4848

49-
To list the installed installed Stackable services run the following command:
49+
To list the installed Stackable services run the following command:
5050

5151
[source,console]
5252
----
5353
$ stackablectl stacklet list
54-
┌─────────┬──────────────┬───────────┬──────────────────────────────────────────────┐
55-
│ PRODUCT ┆ NAME ┆ NAMESPACE ┆ ENDPOINTS │
56-
╞═════════╪══════════════╪═══════════╪══════════════════════════════════════════════╡
57-
│ hive ┆ hive-iceberg ┆ default ┆ hive 172.18.0.4:30637 │
58-
┆ metrics 172.18.0.4:30176
59-
├╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
60-
│ opa ┆ opa ┆ default ┆ http http://172.18.0.2:32470
61-
├╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
62-
│ trino ┆ trino ┆ default ┆ coordinator-metrics 172.18.0.2:32402
63-
│ ┆ ┆ ┆ coordinator-https https://172.18.0.2:31605
64-
├╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
65-
│ minio ┆ minio ┆ default ┆ http http://172.18.0.2:30357
66-
│ ┆ ┆ ┆ console-http http://172.18.0.2:30310 │
67-
└─────────┴──────────────┴───────────┴──────────────────────────────────────────────┘
54+
55+
┌─────────┬───────────────┬───────────┬───────────────────────────────────────────────┬─────────────────────────────────┐
56+
│ PRODUCT ┆ NAME ┆ NAMESPACE ┆ ENDPOINTS ┆ CONDITIONS │
57+
╞═════════╪═══════════════╪═══════════╪═══════════════════════════════════════════════╪═════════════════════════════════╡
58+
hive ┆ hive-iceberg ┆ default ┆ Available, Reconciling, Running
59+
├╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
60+
│ opa ┆ opa ┆ default ┆ ┆ Available, Reconciling, Running
61+
├╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
62+
│ trino ┆ trino ┆ default ┆ coordinator-metrics 172.18.0.2:32612 ┆ Available, Reconciling, Running
63+
│ ┆ ┆ ┆ coordinator-https https://172.18.0.2:30856 ┆
64+
├╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┼╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌╌┤
65+
│ minio ┆ minio-console ┆ default ┆ http http://172.18.0.2:32489 ┆
66+
└─────────┴───────────────┴───────────┴───────────────────────────────────────────────┴─────────────────────────────────┘
67+
6868
----
6969

7070
include::partial$instance-hint.adoc[]
7171

7272
== MinIO
7373

7474
You can view the available buckets and objects (think of files) described in the
75-
xref:data-lakehouse-iceberg-trino-spark.adoc#_minio[data-lakehouse-iceberg-trino-spark] demo.
75+
xref:data-lakehouse-iceberg-trino-spark.adoc#_minio[data-lakehouse-iceberg-trino-spark] demo. Currently, the bucket `lakehouse`
76+
is still empty, but will be filled during the next steps.
7677

7778
== Connect to Trino
7879

79-
Have a look at the xref:data-lakehouse-iceberg-trino-spark.adoc#_connect_with_dbeaver[documentation] on how to
80+
Have a look at the https://docs.stackable.tech/home/stable/trino/usage-guide/connect_to_trino#_connect_with_dbeaver[documentation] on how to
8081
connect with DBeaver. As an alternative, you can use https://trino.io/download.html[trino-cli] by running:
8182

8283
[source,console]
8384
----
84-
$ java -jar ~/Downloads/trino-cli-396-executable.jar --user admin --insecure --password --server https://172.18.0.3:31250
85+
$ java -jar ~/Downloads/trino-cli-451-executable.jar --user admin --insecure --password --server https://172.18.0.2:30856
8586
----
8687

8788
Make sure to replace the server endpoint with the endpoint listed in the `stackablectl stacklet list` output. When
@@ -131,7 +132,7 @@ show tables in tpch.sf5;
131132
(8 rows)
132133
----
133134
134-
The dataset comes with different scale factors. This demo is intended to run on a Laptop, so it starts with a scale
135+
The dataset comes with different scale factors. This demo is intended to run on a laptop, so it starts with a scale
135136
factor of 5 (hence the `sf5` in the above query). If you have a sufficiently large S3 and Trino deployed, you can easily
136137
re-run the statements below with a different scale factor. This demo has been tested up to a scale factor of 10000, but
137138
you can choose any scale in between or even more if desired.
@@ -498,91 +499,6 @@ try to spin up 8 Trino workers with 16GB RAM each, chances are high that Pods wi
498499
resources required can't be fulfilled.
499500
====
500501

501-
=== Scale S3
502-
503-
That should be the preferred option if you can access a managed S3, for example, from a Cloud provider with an excellent
504-
network interconnection.
505-
506-
You can change the endpoint of the S3 by running `kubectl edit s3connection minio -o yaml` and `kubectl edit secret
507-
minio-s3-credentials`. Please note that the credentials need to be base64 encoded.
508-
509-
.Example IONOS configuration
510-
[%collapsible]
511-
====
512-
[source,sql]
513-
----
514-
apiVersion: s3.stackable.tech/v1alpha1
515-
kind: S3Connection
516-
metadata:
517-
name: ionos-sbernauer
518-
spec:
519-
host: s3-eu-central-1.ionoscloud.com
520-
port: 443
521-
tls:
522-
verification:
523-
server:
524-
caCert:
525-
webPki: {}
526-
credentials:
527-
secretClass: ionos-sbernauer-s3-credentials
528-
---
529-
apiVersion: secrets.stackable.tech/v1alpha1
530-
kind: SecretClass
531-
metadata:
532-
name: ionos-sbernauer-s3-credentials
533-
spec:
534-
backend:
535-
k8sSearch:
536-
searchNamespace:
537-
pod: {}
538-
---
539-
apiVersion: v1
540-
kind: Secret
541-
metadata:
542-
name: ionos-sbernauer-s3-credentials
543-
labels:
544-
secrets.stackable.tech/class: ionos-sbernauer-s3-credentials
545-
stringData:
546-
accessKey: "<username>"
547-
secretKey: "<password>"
548-
----
549-
====
550-
551-
If you don't have access to a managed S3 or don't want to use it, you can also scale up the MinIO cluster. You can see
552-
the available replicas using the following command:
553-
554-
[source,console]
555-
----
556-
$ kubectl get statefulsets.apps minio
557-
558-
NAME READY AGE
559-
minio 2/2 4m16s
560-
----
561-
562-
You can edit the MinIO cluster using `kubectl edit statefulsets.apps minio`.
563-
Especially interesting are the following options:
564-
565-
[source,yaml]
566-
----
567-
apiVersion: apps/v1
568-
kind: StatefulSet
569-
metadata:
570-
name: minio
571-
spec:
572-
replicas: 5 # Number of MinIO nodes
573-
template:
574-
spec:
575-
containers:
576-
- name: minio
577-
resources:
578-
requests:
579-
cpu: 1000m # Guaranteed CPU available (one core in this case)
580-
memory: 4Gi # RAM available
581-
----
582-
583-
For example, set `spec.replicas` to `5` and save the changes. You can re-run `kubectl get statefulsets.apps minio` to see the
584-
effect.
585-
586502
=== Scale Trino
587503

588504
Run `kubectl edit trinocluster trino`. Modify the following settings to your needs:
@@ -603,8 +519,8 @@ spec:
603519
limit: 6Gi # Available RAM
604520
workers:
605521
config:
606-
# This limit can't be to big as otherwise the workers won't start.
607-
# I suggest setting it to half of spec.coordinators.config.resources.memory.limit
522+
# This limit can't be too big as otherwise the workers won't start.
523+
# We suggest setting it to half of spec.coordinators.config.resources.memory.limit
608524
queryMaxMemoryPerNode: 6GB
609525
resources:
610526
cpu:

0 commit comments

Comments
 (0)