Skip to content

Commit ac83fb7

Browse files
SlachSlach
committed
Merge branch '0.26.0' of github.com:Altinity/clickhouse-operator into hotfix_go_mod
2 parents f381bc4 + 618e1e2 commit ac83fb7

File tree

85 files changed

+3313
-709
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

85 files changed

+3313
-709
lines changed

‎.github/workflows/release_chart.yaml‎

Lines changed: 30 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -5,12 +5,18 @@ on:
55
types:
66
- published
77
- edited
8+
workflow_dispatch:
9+
inputs:
10+
release_tag:
11+
description: 'release tag'
12+
required: true
813

914
jobs:
1015
release_chart:
1116
name: Release Chart
1217
permissions:
1318
contents: write
19+
packages: write
1420
runs-on: ubuntu-latest
1521
steps:
1622
- name: Checkout
@@ -20,28 +26,43 @@ jobs:
2026

2127
- name: Install chart-releaser
2228
run: |
23-
wget https://github.com/helm/chart-releaser/releases/download/v1.4.1/chart-releaser_1.4.1_linux_amd64.tar.gz
24-
tar -zxf chart-releaser_1.4.1_linux_amd64.tar.gz cr
29+
HELM_CHART_RELEASER_VERSION=$(curl -fsSL https://github.com/helm/chart-releaser/releases/latest -H "Accept: application/json" | jq -r .tag_name)
30+
wget -qO "chart-releaser_${HELM_CHART_RELEASER_VERSION#v}_linux_$(dpkg --print-architecture).tar.gz" "https://github.com/helm/chart-releaser/releases/download/${HELM_CHART_RELEASER_VERSION}/chart-releaser_${HELM_CHART_RELEASER_VERSION#v}_linux_$(dpkg --print-architecture).tar.gz"
31+
tar -zxf "chart-releaser_${HELM_CHART_RELEASER_VERSION#v}_linux_$(dpkg --print-architecture).tar.gz"
2532
sudo install cr /usr/local/bin/
2633
rm -f cr chart-releaser_1.4.1_linux_amd64.tar.gz
2734
2835
- name: Install Helm
2936
run: |
30-
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-3 | bash
37+
curl https://raw.githubusercontent.com/helm/helm/main/scripts/get-helm-4 | bash
3138
3239
- name: Login to GitHub Container Registry
3340
run: echo "${{ secrets.GITHUB_TOKEN }}" | helm registry login ghcr.io -u ${{ github.actor }} --password-stdin
3441

3542
- name: Package Chart
3643
run: cr package deploy/helm/clickhouse-operator
3744

45+
- name: Resolve Helm release ID and NAME
46+
run: |
47+
RELEASE_ID="${{ github.event.release.id }}"
48+
RELEASE_NAME="${{ github.event.release.name }}"
49+
if [[ -z "${RELEASE_ID}" || -z "${RELEASE_NAME}" ]]; then
50+
51+
RELEASE_ID=$(curl -s -H "Accept: application/json" https://github.com/${{ github.repository }}/releases/${{ github.event.inputs.release_tag }} | jq .id)
52+
RELEASE_NAME="${{ github.event.inputs.release_tag }}"
53+
fi
54+
echo "RELEASE_ID=$RELEASE_ID" >> "$GITHUB_ENV"
55+
echo "RELEASE_NAME=$RELEASE_NAME" >> "$GITHUB_ENV"
56+
57+
58+
3859
- name: Get Release Assets
3960
id: get_assets
4061
run: |
4162
CHART_PATH=$(ls .cr-release-packages/altinity-clickhouse-operator-*.tgz)
4263
ASSET_NAME=$(basename ${CHART_PATH})
4364
ASSET_ID=$(curl -s -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
44-
"https://api.github.com/repos/${{ github.repository }}/releases/${{ github.event.release.id }}/assets" | \
65+
"https://api.github.com/repos/${{ github.repository }}/releases/${RELEASE_ID}/assets" | \
4566
jq -r ".[] | select(.name == \"$ASSET_NAME\") | .id")
4667
4768
echo "Asset ID is $ASSET_ID"
@@ -51,7 +72,7 @@ jobs:
5172
if: steps.get_assets.outputs.asset_id != ''
5273
run: |
5374
curl -X DELETE -H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
54-
"https://api.github.com/repos/${{ github.repository }}/releases/${{ github.event.release.id }}/assets/${{ steps.get_assets.outputs.asset_id }}"
75+
"https://api.github.com/repos/${{ github.repository }}/releases/${RELEASE_ID}/assets/${{ steps.get_assets.outputs.asset_id }}"
5576
5677
- name: Upload Release Artifacts
5778
run: |
@@ -61,7 +82,7 @@ jobs:
6182
-H "Authorization: Bearer ${{ secrets.GITHUB_TOKEN }}" \
6283
-H "Content-Type: application/gzip" \
6384
-T "${CHART_PATH}" \
64-
"https://uploads.github.com/repos/${GITHUB_REPOSITORY}/releases/${{ github.event.release.id }}/assets?name=$(basename ${CHART_PATH})"
85+
"https://uploads.github.com/repos/${GITHUB_REPOSITORY}/releases/${RELEASE_ID}/assets?name=$(basename ${CHART_PATH})"
6586
- name: Validate Helm Repository Configuration
6687
run: |
6788
if [ -z "${{ secrets.HELM_GITHUB_TOKEN }}" ]; then
@@ -90,7 +111,7 @@ jobs:
90111
cr upload \
91112
--git-repo=${{ vars.HELM_GITHUB_REPOSITORY }} \
92113
--owner=${GITHUB_REPOSITORY_OWNER} \
93-
--release-name-template=${{ github.event.release.name }} \
114+
--release-name-template=${RELEASE_NAME} \
94115
--token=${{ secrets.HELM_GITHUB_TOKEN }} \
95116
--package-path=.cr-release-packages \
96117
--skip-existing
@@ -105,7 +126,7 @@ jobs:
105126
cr index \
106127
--git-repo=${GITHUB_REPOSITORY#*/} \
107128
--owner=${GITHUB_REPOSITORY_OWNER} \
108-
--release-name-template=${{ github.event.release.name }} \
129+
--release-name-template=${RELEASE_NAME} \
109130
--token=${{ secrets.GITHUB_TOKEN }} \
110131
--index-path=index.yaml \
111132
--remote=httpsorigin \
@@ -174,7 +195,7 @@ jobs:
174195
cr index \
175196
--git-repo=${{ vars.HELM_GITHUB_REPOSITORY }} \
176197
--owner=${GITHUB_REPOSITORY_OWNER} \
177-
--release-name-template=${{ github.event.release.name }} \
198+
--release-name-template=${RELEASE_NAME} \
178199
--token=${{ secrets.HELM_GITHUB_TOKEN }} \
179200
--package-path=.cr-release-packages \
180201
--index-path=index.yaml \

‎.gitignore‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -55,4 +55,6 @@ tests/image/cache
5555
.aider*
5656
CLAUDE.md
5757
AGENTS.md
58-
.claude
58+
.claude
59+
60+
/.cr-release-packages

‎cmd/operator/app/thread_keeper.go‎

Lines changed: 8 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -59,7 +59,10 @@ func initKeeper(ctx context.Context) error {
5959

6060
err = ctrlRuntime.
6161
NewControllerManagedBy(manager).
62-
For(&api.ClickHouseKeeperInstallation{}, builder.WithPredicates(keeperPredicate())).
62+
For(
63+
&api.ClickHouseKeeperInstallation{},
64+
builder.WithPredicates(keeperPredicate()),
65+
).
6366
Owns(&apps.StatefulSet{}).
6467
Complete(
6568
&controller.Controller{
@@ -88,40 +91,30 @@ func runKeeper(ctx context.Context) error {
8891
func keeperPredicate() predicate.Funcs {
8992
return predicate.Funcs{
9093
CreateFunc: func(e event.CreateEvent) bool {
91-
obj, ok := e.Object.(*api.ClickHouseKeeperInstallation)
94+
new, ok := e.Object.(*api.ClickHouseKeeperInstallation)
9295
if !ok {
9396
return false
9497
}
9598

96-
// Check if namespace should be watched (includes deny list check)
97-
if !chop.Config().IsNamespaceWatched(obj.Namespace) {
98-
logger.V(2).Info("chkInformer: skip event, namespace is not watched or is in deny list", "namespace", obj.Namespace)
99+
if !controller.ShouldEnqueue(new) {
99100
return false
100101
}
101102

102-
if obj.Spec.Suspend.Value() {
103-
return false
104-
}
105103
return true
106104
},
107105
DeleteFunc: func(e event.DeleteEvent) bool {
108106
return true
109107
},
110108
UpdateFunc: func(e event.UpdateEvent) bool {
111-
obj, ok := e.ObjectNew.(*api.ClickHouseKeeperInstallation)
109+
new, ok := e.ObjectNew.(*api.ClickHouseKeeperInstallation)
112110
if !ok {
113111
return false
114112
}
115113

116-
// Check if namespace should be watched (includes deny list check)
117-
if !chop.Config().IsNamespaceWatched(obj.Namespace) {
118-
logger.V(2).Info("chkInformer: skip event, namespace is not watched or is in deny list", "namespace", obj.Namespace)
114+
if !controller.ShouldEnqueue(new) {
119115
return false
120116
}
121117

122-
if obj.Spec.Suspend.Value() {
123-
return false
124-
}
125118
return true
126119
},
127120
GenericFunc: func(e event.GenericEvent) bool {

‎config/config-dev.yaml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -247,6 +247,10 @@ clickhouse:
247247
# Upon reaching this timeout metrics collection is aborted and no more metrics are collected in this cycle.
248248
# All collected metrics are returned.
249249
collect: 9
250+
# Regexp to match tables in system database to fetch metrics from.
251+
# Multiple tables can be matched using regexp. Matched tables are merged using merge() table function.
252+
# Default is "^(metrics|custom_metrics)$" which fetches from both system.metrics and system.custom_metrics.
253+
tablesRegexp: "^(metrics|custom_metrics)$"
250254

251255
keeper:
252256
configuration:
@@ -351,6 +355,20 @@ reconcile:
351355
# 3. ignore - ignore an error, pretend nothing happened, continue reconcile and move on to the next StatefulSet.
352356
onFailure: abort
353357

358+
# Recreate StatefulSet scenario
359+
recreate:
360+
# What to do in case operator is in need to recreate StatefulSet?
361+
# Possible options:
362+
# 1. abort - abort the process, do nothing with the problematic StatefulSet, leave it as it is,
363+
# do not try to fix or delete or update it, just abort reconcile cycle.
364+
# Do not proceed to the next StatefulSet(s) and wait for an admin to assist.
365+
# 2. recreate - proceed and recreate StatefulSet.
366+
367+
# Triggered when PVC data loss or missing volumes are detected
368+
onDataLoss: recreate
369+
# Triggered when StatefulSet update fails or StatefulSet is not ready
370+
onUpdateFailure: recreate
371+
354372
# Reconcile Host scenario
355373
host:
356374
# The operator during reconcile procedure should wait for a ClickHouse host to achieve the following conditions:

‎config/config.yaml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -245,6 +245,10 @@ clickhouse:
245245
# Upon reaching this timeout metrics collection is aborted and no more metrics are collected in this cycle.
246246
# All collected metrics are returned.
247247
collect: 9
248+
# Regexp to match tables in system database to fetch metrics from.
249+
# Multiple tables can be matched using regexp. Matched tables are merged using merge() table function.
250+
# Default is "^(metrics|custom_metrics)$" which fetches from both system.metrics and system.custom_metrics.
251+
tablesRegexp: "^(metrics|custom_metrics)$"
248252

249253
keeper:
250254
configuration:
@@ -349,6 +353,20 @@ reconcile:
349353
# 3. ignore - ignore an error, pretend nothing happened, continue reconcile and move on to the next StatefulSet.
350354
onFailure: abort
351355

356+
# Recreate StatefulSet scenario
357+
recreate:
358+
# What to do in case operator is in need to recreate StatefulSet?
359+
# Possible options:
360+
# 1. abort - abort the process, do nothing with the problematic StatefulSet, leave it as it is,
361+
# do not try to fix or delete or update it, just abort reconcile cycle.
362+
# Do not proceed to the next StatefulSet(s) and wait for an admin to assist.
363+
# 2. recreate - proceed and recreate StatefulSet.
364+
365+
# Triggered when PVC data loss or missing volumes are detected
366+
onDataLoss: recreate
367+
# Triggered when StatefulSet update fails or StatefulSet is not ready
368+
onUpdateFailure: recreate
369+
352370
# Reconcile Host scenario
353371
host:
354372
# The operator during reconcile procedure should wait for a ClickHouse host to achieve the following conditions:

‎deploy/builder/templates-config/config.yaml‎

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -239,6 +239,10 @@ clickhouse:
239239
# Upon reaching this timeout metrics collection is aborted and no more metrics are collected in this cycle.
240240
# All collected metrics are returned.
241241
collect: 9
242+
# Regexp to match tables in system database to fetch metrics from.
243+
# Multiple tables can be matched using regexp. Matched tables are merged using merge() table function.
244+
# Default is "^(metrics|custom_metrics)$" which fetches from both system.metrics and system.custom_metrics.
245+
tablesRegexp: "^(metrics|custom_metrics)$"
242246

243247
keeper:
244248
configuration:
@@ -343,6 +347,20 @@ reconcile:
343347
# 3. ignore - ignore an error, pretend nothing happened, continue reconcile and move on to the next StatefulSet.
344348
onFailure: abort
345349

350+
# Recreate StatefulSet scenario
351+
recreate:
352+
# What to do in case operator is in need to recreate StatefulSet?
353+
# Possible options:
354+
# 1. abort - abort the process, do nothing with the problematic StatefulSet, leave it as it is,
355+
# do not try to fix or delete or update it, just abort reconcile cycle.
356+
# Do not proceed to the next StatefulSet(s) and wait for an admin to assist.
357+
# 2. recreate - proceed and recreate StatefulSet.
358+
359+
# Triggered when PVC data loss or missing volumes are detected
360+
onDataLoss: recreate
361+
# Triggered when StatefulSet update fails or StatefulSet is not ready
362+
onUpdateFailure: recreate
363+
346364
# Reconcile Host scenario
347365
host:
348366
# The operator during reconcile procedure should wait for a ClickHouse host to achieve the following conditions:

‎deploy/builder/templates-install-bundle/clickhouse-operator-install-yaml-template-01-section-crd-01-chi-chit.yaml‎

Lines changed: 74 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -497,6 +497,80 @@ spec:
497497
minimum: 0
498498
maximum: 100
499499
description: "The maximum percentage of cluster shards that may be reconciled in parallel, 50 percent by default."
500+
statefulSet: &TypeReconcileStatefulSet
501+
type: object
502+
description: "Optional, StatefulSet reconcile behavior tuning"
503+
properties:
504+
create:
505+
type: object
506+
description: "Behavior during create StatefulSet"
507+
properties:
508+
onFailure:
509+
type: string
510+
description: |
511+
What to do in case created StatefulSet is not in 'Ready' after `reconcile.statefulSet.update.timeout` seconds.
512+
Possible options:
513+
1. abort - abort the process, do nothing with the problematic StatefulSet, leave it as it is.
514+
2. delete - delete newly created problematic StatefulSet and follow 'abort' path afterwards.
515+
3. ignore - ignore an error, pretend nothing happened, continue reconcile and move on to the next StatefulSet.
516+
enum:
517+
- ""
518+
- "abort"
519+
- "delete"
520+
- "ignore"
521+
update:
522+
type: object
523+
description: "Behavior during update StatefulSet"
524+
properties:
525+
timeout:
526+
type: integer
527+
description: "How many seconds to wait for StatefulSet to be 'Ready' during update"
528+
minimum: 0
529+
maximum: 3600
530+
pollInterval:
531+
type: integer
532+
description: "How many seconds to wait between checks for StatefulSet status during update"
533+
minimum: 1
534+
maximum: 600
535+
onFailure:
536+
type: string
537+
description: |
538+
What to do in case updated StatefulSet is not in 'Ready' after `reconcile.statefulSet.update.timeout` seconds.
539+
Possible options:
540+
1. abort - abort the process, do nothing with the problematic StatefulSet, leave it as it is.
541+
2. rollback - delete Pod and rollback StatefulSet to previous Generation. Follow 'abort' path afterwards.
542+
3. ignore - ignore an error, pretend nothing happened, continue reconcile and move on to the next StatefulSet.
543+
enum:
544+
- ""
545+
- "abort"
546+
- "rollback"
547+
- "ignore"
548+
recreate:
549+
type: object
550+
description: "Behavior during recreate StatefulSet"
551+
properties:
552+
onDataLoss:
553+
type: string
554+
description: |
555+
What to do in case operator needs to recreate StatefulSet due to PVC data loss or missing volumes.
556+
Possible options:
557+
1. abort - abort the process, do nothing with the problematic StatefulSet.
558+
2. recreate - proceed and recreate StatefulSet.
559+
enum:
560+
- ""
561+
- "abort"
562+
- "recreate"
563+
onUpdateFailure:
564+
type: string
565+
description: |
566+
What to do in case operator needs to recreate StatefulSet due to update failure or StatefulSet not ready.
567+
Possible options:
568+
1. abort - abort the process, do nothing with the problematic StatefulSet.
569+
2. recreate - proceed and recreate StatefulSet.
570+
enum:
571+
- ""
572+
- "abort"
573+
- "recreate"
500574
host: &TypeReconcileHost
501575
type: object
502576
description: |

‎deploy/builder/templates-install-bundle/clickhouse-operator-install-yaml-template-01-section-crd-02-chopconf.yaml‎

Lines changed: 24 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -240,6 +240,12 @@ spec:
240240
Timeout used to limit metrics collection request. In seconds.
241241
Upon reaching this timeout metrics collection is aborted and no more metrics are collected in this cycle.
242242
All collected metrics are returned.
243+
tablesRegexp:
244+
type: string
245+
description: |
246+
Regexp to match tables in system database to fetch metrics from.
247+
Multiple tables can be matched using regexp. Matched tables are merged using merge() table function.
248+
Default is "^(metrics|custom_metrics)$".
243249
template:
244250
type: object
245251
description: "Parameters which are used if you want to generate ClickHouseInstallationTemplate custom resources from files which are stored inside clickhouse-operator deployment"
@@ -318,6 +324,24 @@ spec:
318324
1. abort - do nothing, just break the process and wait for admin.
319325
2. rollback (default) - delete Pod and rollback StatefulSet to previous Generation. Pod would be recreated by StatefulSet based on rollback-ed configuration.
320326
3. ignore - ignore error, pretend nothing happened and move on to the next StatefulSet.
327+
recreate:
328+
type: object
329+
description: "Behavior during recreate StatefulSet"
330+
properties:
331+
onDataLoss:
332+
type: string
333+
description: |
334+
What to do in case operator needs to recreate StatefulSet due to PVC data loss or missing volumes.
335+
Possible options:
336+
1. abort - abort the process, do nothing with the problematic StatefulSet.
337+
2. recreate (default) - proceed and recreate StatefulSet.
338+
onUpdateFailure:
339+
type: string
340+
description: |
341+
What to do in case operator needs to recreate StatefulSet due to update failure or StatefulSet not ready.
342+
Possible options:
343+
1. abort - abort the process, do nothing with the problematic StatefulSet.
344+
2. recreate (default) - proceed and recreate StatefulSet.
321345
host:
322346
type: object
323347
description: |

0 commit comments

Comments
 (0)