Skip to content

Commit f7ba23f

Browse files
catpineapplecatpineapple
authored
Merge pull request #393 from intelligentfu8/dis_helm
Dis helm
2 parents 9563323 + 0e9ca6e commit f7ba23f

File tree

13 files changed

+552
-4
lines changed

13 files changed

+552
-4
lines changed

‎api/disaggregated/v1/types.go‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -153,7 +153,7 @@ type CommonSpec struct {
153153
//+optional
154154
Tolerations []corev1.Toleration `json:"tolerations,omitempty"`
155155

156-
// export metaservice for accessing from outside k8s.
156+
// export service for accessing from outside k8s.
157157
Service *ExportService `json:"service,omitempty"`
158158

159159
// ConfigMaps describe all configmap that need to be mounted.

‎helm-charts/doris-disaggregated/.helmignore‎

Lines changed: 23 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,23 @@
1+
# Patterns to ignore when building packages.
2+
# This supports shell glob matching, relative path matching, and
3+
# negation (prefixed with !). Only one pattern per line.
4+
.DS_Store
5+
# Common VCS dirs
6+
.git/
7+
.gitignore
8+
.bzr/
9+
.bzrignore
10+
.hg/
11+
.hgignore
12+
.svn/
13+
# Common backup files
14+
*.swp
15+
*.bak
16+
*.tmp
17+
*.orig
18+
*~
19+
# Various IDEs
20+
.project
21+
.idea/
22+
*.tmproj
23+
.vscode/

‎helm-charts/doris-disaggregated/Chart.yaml‎

Lines changed: 41 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,41 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
apiVersion: v2
19+
name: doris-disaggregated
20+
description: A Helm chart for deploying Doris Cluster of compute and storage decoupled.
21+
22+
# A chart can be either an 'application' or a 'library' chart.
23+
#
24+
# Application charts are a collection of templates that can be packaged into versioned archives
25+
# to be deployed.
26+
#
27+
# Library charts provide useful utilities or functions for the chart developer. They're included as
28+
# a dependency of application charts to inject those utilities and functions into the rendering
29+
# pipeline. Library charts do not define any templates and therefore cannot be deployed.
30+
type: application
31+
32+
# This is the chart version. This version number should be incremented each time you make changes
33+
# to the chart and its templates, including the app version.
34+
# Versions are expected to follow Semantic Versioning (https://semver.org/)
35+
version: 0.1.0
36+
37+
# This is the version number of the application being deployed. This version number should be
38+
# incremented each time you make changes to the application. Versions are not expected to
39+
# follow Semantic Versioning. They should reflect the version the application is using.
40+
# It is recommended to use it with quotes.
41+
appVersion: "3.4.0"

‎helm-charts/doris-disaggregated/README.md‎

Lines changed: 57 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,57 @@
1+
# Deploy Doris Storage-Compute Decoupled Cluster
2+
[![Artifact Hub](https://img.shields.io/endpoint?url=https://artifacthub.io/badge/repository/doris)](https://artifacthub.io/packages/search?repo=doris)
3+
4+
## Prepare
5+
1. install doris operator
6+
Deploy the doris operator in your kubernetes cluster. deploy doris operator have two methods: [directly yaml](https://github.com/apache/doris-operator); [doris operator helm chart](https://artifacthub.io/packages/helm/doris/doris-operator).
7+
2. install foundationDB
8+
Doris storage-compute decoupled cluster use the FoundationDB as meta storage component, please prepare a foundationdb cluster before deploy doris storage-compute decoupled cluster. please refer the [foundationdb official doc site](https://apple.github.io/foundationdb/administration.html#starting-and-stopping) to deploy on machine, or you can use the [fdb-kubernetes-operator](https://github.com/FoundationDB/fdb-kubernetes-operator) to deploy on kubernetes. doris operator provide a sample [helm chart](https://artifacthub.io/packages/helm/doris/doris-foundationdb) that integrate the official fdb-kubernetes-operator chart to deploy foundationdb on kubernetes.
9+
10+
## Install
11+
### Add helm-chart repo
12+
1. add the selectdb repository
13+
```Bash
14+
helm repo add selectdb https://charts.selectdb.com
15+
```
16+
2. update the helm chart repo to the latest version
17+
```Bash
18+
helm repo udpate selectdb
19+
```
20+
3. check the helm chart repo is the latest version
21+
```Bash
22+
$ helm search repo selectdb
23+
NAME CHART VERSION APP VERSION DESCRIPTION
24+
selectdb/doris-operator 25.4.0 1.3.1 Doris-operator for doris creat ...
25+
selectdb/doris 25.4.0 2.1.7 Apache Doris is an easy-to-use ...
26+
selectdb/doris-foundationdb 0.2.0 v2.3.0 A Helm chart for foundationDB ...
27+
```
28+
### Install the doris storage-compute decoupled cluster
29+
#### Use default config
30+
- fdb deployed on kubernetes
31+
```Bash
32+
helm install doris-disaggregated --set msSpec.fdb.namespace={namespace} --set msSpec.fdb.fdbClusterName={fdbClusterName}
33+
```
34+
please use the real namespace replace the {namespace} as the foundationdb deployed namespace, if you use the [fdb-kubernetes-operator](https://github.com/FoundationDB/fdb-kubernetes-operator) or [doris-foundationdb](https://artifacthub.io/packages/helm/doris/doris-foundationdb) deploy foundationdb.
35+
{fdbClusterName} is the `FoundationDBCluster` resource's name.
36+
- fdb deployed on machine
37+
```Bash
38+
helm install doris-disaggregated --set msSpec.fdb.address={address}
39+
```
40+
{address} is the address of fdb accessed, it is content of [fdb.cluster file](https://apple.github.io/foundationdb/administration.html#cluster-files).
41+
42+
#### Custom deploying
43+
1. use the follow command to download and unpack the chart
44+
```Bash
45+
helm pull --untar selectdb/doris-disaggregated
46+
```
47+
48+
2. helm install doris storage-compute decoupled cluster
49+
when you want to specify resources or different deployment type, please custom the [`values.yaml`](./values.yaml) and use next command for deploying.
50+
```Bash
51+
helm install doris-disaggregated -f values.yaml doris-disaggregated
52+
```
53+
## Uninstall cluster
54+
Please confirm the Doris storage-compute decoupled cluster is not used, when using next command to uninstall `doris-disaggregated`.
55+
```Bash
56+
helm uninstall doris-disaggregated
57+
```

‎helm-charts/doris-disaggregated/templates/NOTES.txt‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1 @@
1+
Thank you for installing {{ .Chart.Name }}-{{ .Chart.Version }}

‎helm-charts/doris-disaggregated/templates/_helpers.tpl‎

Lines changed: 73 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,73 @@
1+
{*
2+
Licensed to the Apache Software Foundation (ASF) under one
3+
or more contributor license agreements. See the NOTICE file
4+
distributed with this work for additional information
5+
regarding copyright ownership. The ASF licenses this file
6+
to you under the Apache License, Version 2.0 (the
7+
"License"); you may not use this file except in compliance
8+
with the License. You may obtain a copy of the License at
9+
10+
http://www.apache.org/licenses/LICENSE-2.0
11+
12+
Unless required by applicable law or agreed to in writing,
13+
software distributed under the License is distributed on an
14+
"AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
15+
KIND, either express or implied. See the License for the
16+
specific language governing permissions and limitations
17+
under the License.
18+
*}
19+
20+
{{/*
21+
cluster config
22+
*/}}
23+
{{- define "doris-disaggregated.name" -}}
24+
{{- default .Chart.Name .Values.clusterName | trunc 63 | trimSuffix "-" }}
25+
{{- end }}
26+
27+
{{/*
28+
ms config part
29+
*/}}
30+
{{- define "ms.fdb.namespace" -}}
31+
{{- default .Release.Namespace .Values.msSpec.fdb.namespace -}}
32+
{{- end }}
33+
34+
{{- define "ms.fdb.configmap.name" -}}
35+
{{ .Values.msSpec.fdb.fdbClusterName }}-config
36+
{{- end }}
37+
38+
{{- define "ms.configmap.name" -}}
39+
{{- print "ms-configmap" }}
40+
{{- end }}
41+
42+
{{- define "ms.configmap.mountpath" -}}
43+
{{- print "/etc/doris" -}}
44+
{{- end }}
45+
46+
47+
{{/*
48+
fe config part
49+
*/}}
50+
{{- define "fe.electionnumber" -}}
51+
{{- default 3 .Values.feSpec.electionNumber -}}
52+
{{- end }}
53+
54+
{{- define "fe.configmap.name" -}}
55+
{{- print "fe-configmap" }}
56+
{{- end }}
57+
58+
{{- define "fe.configmap.mountpath" -}}
59+
{{- print "/etc/doris" -}}
60+
{{- end }}
61+
62+
63+
{{/*
64+
be config part
65+
*/}}
66+
{{- define "be.configmap.name" -}}
67+
{{- print "be-configmap" -}}
68+
{{- end }}
69+
70+
{{- define "be.configmap.mountpath" -}}
71+
{{- print "/etc/doris" -}}
72+
{{- end }}
73+

‎helm-charts/doris-disaggregated/templates/be-configmap.yaml‎

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
apiVersion: v1
19+
kind: ConfigMap
20+
metadata:
21+
name: {{ include "be.configmap.name" . }}
22+
labels:
23+
app.kubernetes.io/component: be
24+
data:
25+
be.conf: |
26+
# For jdk 17, this JAVA_OPTS will be used as default JVM options
27+
JAVA_OPTS_FOR_JDK_17="-Xmx1024m -DlogPath=$LOG_DIR/jni.log -Xlog:gc*:$LOG_DIR/be.gc.log.$CUR_DATE:time,uptime:filecount=10,filesize=50M -Djavax.security.auth.useSubjectCredsOnly=false -Dsun.security.krb5.debug=true -Dsun.java.command=DorisBE -XX:-CriticalJNINatives -XX:+IgnoreUnrecognizedVMOptions --add-opens=java.base/java.lang=ALL-UNNAMED --add-opens=java.base/java.lang.invoke=ALL-UNNAMED --add-opens=java.base/java.lang.reflect=ALL-UNNAMED --add-opens=java.base/java.io=ALL-UNNAMED --add-opens=java.base/java.net=ALL-UNNAMED --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens=java.base/java.util=ALL-UNNAMED --add-opens=java.base/java.util.concurrent=ALL-UNNAMED --add-opens=java.base/java.util.concurrent.atomic=ALL-UNNAMED --add-opens=java.base/sun.nio.ch=ALL-UNNAMED --add-opens=java.base/sun.nio.cs=ALL-UNNAMED --add-opens=java.base/sun.security.action=ALL-UNNAMED --add-opens=java.base/sun.util.calendar=ALL-UNNAMED --add-opens=java.security.jgss/sun.security.krb5=ALL-UNNAMED --add-opens=java.management/sun.management=ALL-UNNAMED"
28+
file_cache_path = [{"path":"/opt/apache-doris/be/file_cache","total_size":107374182400,"query_limit":107374182400}]
29+
storage_root_path = /opt/apache-doris/be/file_cache

‎helm-charts/doris-disaggregated/templates/dorisdisaggregatedcluster.yaml‎

Lines changed: 100 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,100 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
apiVersion: disaggregated.cluster.doris.com/v1
19+
kind: DorisDisaggregatedCluster
20+
metadata:
21+
name: {{ include "doris-disaggregated.name" . }}
22+
{{- if .Values.Labels }}
23+
labels:
24+
{{- .Values.Labels | nindent 4 }}
25+
{{- end }}
26+
spec:
27+
{{- if .Values.adminUser }}
28+
adminUser:
29+
name: {{ .Values.adminUser.name }}
30+
password: {{ .Values.adminUser.password }}
31+
{{- end }}
32+
metaService:
33+
image: {{ .Values.msSpec.image.repository }}:{{ .Values.msSpec.image.tag }}
34+
{{- if .Values.msSpec.imagePullSecrets }}
35+
imagePullSecrets:
36+
{{- toYaml .Values.msSpec.imagePullSecrets | nindent 6 }}
37+
{{- end }}
38+
replicas: {{ .Values.msSpec.replicas }}
39+
fdb:
40+
{{- if .Values.msSpec.fdb.address }}
41+
address: {{ .Values.msSpec.fdb.address }}
42+
{{- else }}
43+
configMapNamespaceName:
44+
name: {{ include "ms.fdb.configmap.name" . }}
45+
namespace: {{ include "ms.fdb.namespace" . }}
46+
{{- end }}
47+
configMaps:
48+
- name: {{ include "ms.configmap.name" . }}
49+
mountPath: {{ include "ms.configmap.mountpath" . }}
50+
{{- if .Values.msSpec.resources }}
51+
{{- toYaml .Values.msSpec.resources | nindent 4 }}
52+
{{- end }}
53+
{{- if .Values.msSpec.nodeSelector }}
54+
nodeSelector:
55+
{{- toYaml .Values.msSpec.nodeSelector | nindent 6 }}
56+
{{- end }}
57+
feSpec:
58+
replicas: {{ .Values.feSpec.replicas }}
59+
electionNumber: {{ include "fe.electionnumber" . }}
60+
image: {{ .Values.feSpec.image.repository }}:{{ .Values.feSpec.image.tag }}
61+
{{- if .Values.feSpec.imagePullSecrets }}
62+
imagePullSecrets:
63+
{{- toYaml .Values.feSpec.imagePullSecrets | nindent 6 }}
64+
{{- end }}
65+
configMaps:
66+
- name: {{ include "fe.configmap.name" . }}
67+
mountPath: {{ include "fe.configmap.mountpath" . }}
68+
logNotStore: {{ .Values.feSpec.logNotStore }}
69+
{{- if .Values.feSpec.resources }}
70+
{{- toYaml .Values.feSpec.resources | nindent 4 }}
71+
{{- end }}
72+
{{- if .Values.feSpec.service }}
73+
{{- toYaml .Values.feSpec.service }}
74+
{{- end }}
75+
{{- if .Values.feSpec.persistentVolumes }}
76+
persistentVolumes:
77+
{{- toYaml .Values.feSpec.persistentVolumes | nindent 4 }}
78+
{{- end }}
79+
computeGroups:
80+
{{- $originalContext := . -}}
81+
{{- range $index, $cgName := .Values.computeGroupNames }}
82+
- uniqueId: {{ $cgName }}
83+
replicas: {{ $originalContext.Values.computeSpec.replicas }}
84+
image: {{ $originalContext.Values.computeSpec.image.repository }}:{{ $originalContext.Values.computeSpec.image.tag }}
85+
{{- if $originalContext.Values.computeSpec.imagePullSecrets }}
86+
imagePullSecrets:
87+
{{- toYaml $originalContext.Values.computeSpec.imagePullSecrets | nindent 6 }}
88+
{{- end }}
89+
{{- if $originalContext.Values.computeSpec.resources }}
90+
{{- toYaml $originalContext.Values.computeSpec.resources | nindent 4 }}
91+
{{- end }}
92+
configMaps:
93+
- name: {{ include "be.configmap.name" . }}
94+
mountPath: {{ include "be.configmap.mountpath" . }}
95+
logNotStore: {{ $originalContext.Values.computeSpec.logNotStore }}
96+
{{- if $originalContext.Values.computeSpec.persistentVolumes }}
97+
persistentVolumes:
98+
{{- toYaml $originalContext.Values.computeSpec.persistentVolumes | nindent 4 }}
99+
{{- end }}
100+
{{- end }}

‎helm-charts/doris-disaggregated/templates/fe-configmap.yaml‎

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,39 @@
1+
# Licensed to the Apache Software Foundation (ASF) under one
2+
# or more contributor license agreements. See the NOTICE file
3+
# distributed with this work for additional information
4+
# regarding copyright ownership. The ASF licenses this file
5+
# to you under the Apache License, Version 2.0 (the
6+
# "License"); you may not use this file except in compliance
7+
# with the License. You may obtain a copy of the License at
8+
#
9+
# http://www.apache.org/licenses/LICENSE-2.0
10+
#
11+
# Unless required by applicable law or agreed to in writing,
12+
# software distributed under the License is distributed on an
13+
# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
14+
# KIND, either express or implied. See the License for the
15+
# specific language governing permissions and limitations
16+
# under the License.
17+
18+
apiVersion: v1
19+
data:
20+
fe.conf: |
21+
CUR_DATE=`date +%Y%m%d-%H%M%S`
22+
# Log dir
23+
LOG_DIR = ${DORIS_HOME}/log
24+
# For jdk 17, this JAVA_OPTS will be used as default JVM options
25+
JAVA_OPTS_FOR_JDK_17="-Djavax.security.auth.useSubjectCredsOnly=false -Xmx8192m -Xms8192m -XX:+HeapDumpOnOutOfMemoryError -XX:HeapDumpPath=$LOG_DIR -Xlog:gc*:$LOG_DIR/fe.gc.log.$CUR_DATE:time,uptime:filecount=10,filesize=50M --add-opens=java.base/java.nio=ALL-UNNAMED --add-opens java.base/jdk.internal.ref=ALL-UNNAMED"
26+
# INFO, WARN, ERROR, FATAL
27+
sys_log_level = INFO
28+
# NORMAL, BRIEF, ASYNC
29+
sys_log_mode = NORMAL
30+
# Default dirs to put jdbc drivers,default value is ${DORIS_HOME}/jdbc_drivers
31+
# jdbc_drivers_dir = ${DORIS_HOME}/jdbc_drivers
32+
http_port = 8030
33+
rpc_port = 9020
34+
query_port = 9030
35+
edit_log_port = 9010
36+
enable_fqdn_mode=true
37+
kind: ConfigMap
38+
metadata:
39+
name: {{ include "fe.configmap.name" . }}

0 commit comments

Comments
 (0)