Skip to content

Commit 9d16511

Browse files
pazonemergify[bot]
pazone
authored and
mergify[bot]
committed
[CI] Stable stack version override (#8352)
* [CI] Stable stack version override * Update stable version * Added UpdateCLI config * Added UpdateCLI config * Applied proposed changes * Update branch filter * Update branch filter * Added th action * Improved PR description * improve PR message * Pin version in create_deployment_csp_configuration * Added control plane team label * Remove redundant : * Applied proposed changes * Update test_infra/ess/deployment.tf Co-authored-by: Victor Martinez <[email protected]> * Added fallback to stack_version for docker images * build to .build * Added pipelineid and force * force: false * Enabled automerge --------- Co-authored-by: Victor Martinez <[email protected]> (cherry picked from commit e728b39) # Conflicts: # pkg/testing/ess/create_deployment_csp_configuration.yaml
1 parent 3724499 commit 9d16511

File tree

5 files changed

+159
-5
lines changed

5 files changed

+159
-5
lines changed

‎.ci/updatecli/updatecli-bump-stack-version.yml‎

Lines changed: 51 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,51 @@
1+
# update-cli configuration for automated go updates
2+
---
3+
name: Bump ESS snapshots
4+
5+
pipelineid: 'bump-ess-snapshot-{{ requiredEnv "BRANCH" }}'
6+
7+
scms:
8+
githubConfig:
9+
kind: github
10+
spec:
11+
user: '{{ requiredEnv "GITHUB_ACTOR" }}'
12+
username: '{{ requiredEnv "GITHUB_ACTOR" }}'
13+
owner: "{{ .scm.owner }}"
14+
repository: "{{ .scm.repository }}"
15+
token: '{{ requiredEnv "GITHUB_TOKEN" }}'
16+
branch: '{{ requiredEnv "BRANCH_NAME" }}'
17+
commitusingapi: true
18+
force: false
19+
20+
sources:
21+
lastStableStackSnapshot:
22+
name: "Latest Stable ESS snapshot for current branch"
23+
kind: json
24+
spec:
25+
file: 'https://storage.googleapis.com/artifacts-api/channels/{{ requiredEnv "BRANCH_NAME" }}.json'
26+
key: .build
27+
targets:
28+
updateStackImages:
29+
name: "Update create_deployment_csp_configuration.yaml"
30+
sourceid: lastStableStackSnapshot
31+
scmid: githubConfig
32+
kind: file
33+
spec:
34+
file: "pkg/testing/ess/create_deployment_csp_configuration.yaml"
35+
matchpattern: '(\S*_image: ".*docker.*):(.+)"'
36+
replacepattern: '$1:{{ source "lastStableStackSnapshot" }}-SNAPSHOT"'
37+
actions:
38+
elastic-agent:
39+
kind: github/pullrequest
40+
scmid: githubConfig
41+
sourceid: lastStableStackSnapshot
42+
spec:
43+
automerge: true
44+
labels:
45+
- dependencies
46+
- backport-skip
47+
- skip-changelog
48+
- "Team:Elastic-Agent-Control-Plane"
49+
title: '[{{ requiredEnv "BRANCH_NAME" }}][Automation] Bump stack images versions to {{ source "lastStableStackSnapshot" }}'
50+
description: |
51+
Updates `create_deployment_csp_configuration.yml`. It is used to pin stable images versions for integration tests.

‎.github/workflows/bump-stable-snapshot.yml‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
---
2+
name: update-stable-snapshot
3+
4+
on:
5+
workflow_dispatch:
6+
schedule:
7+
- cron: "0 0 * * *"
8+
9+
permissions:
10+
contents: read
11+
12+
env:
13+
JOB_URL: "${{ github.server_url }}/${{ github.repository }}/actions/runs/${{ github.run_id }}"
14+
15+
jobs:
16+
filter:
17+
runs-on: ubuntu-latest
18+
timeout-minutes: 1
19+
outputs:
20+
matrix: ${{ steps.generator.outputs.matrix }}
21+
permissions:
22+
contents: read
23+
steps:
24+
- id: generator
25+
uses: elastic/oblt-actions/elastic/active-branches@v1
26+
with:
27+
filter-branches: true
28+
29+
update-stable-snapshot:
30+
runs-on: ubuntu-latest
31+
needs: [filter]
32+
permissions:
33+
contents: write
34+
pull-requests: write
35+
strategy:
36+
fail-fast: false
37+
matrix: ${{ fromJson(needs.filter.outputs.matrix) }}
38+
steps:
39+
# We have same behavior for all branches.
40+
- uses: actions/checkout@v4
41+
42+
- name: Run UpdateCLI in Apply mode
43+
uses: elastic/oblt-actions/updatecli/run@v1
44+
with:
45+
command: apply --config .ci/updatecli/updatecli-bump-stack-version.yml --values .ci/updatecli/values.d/scm.yml
46+
env:
47+
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
48+
BRANCH_NAME: ${{ matrix.branch }}
49+
50+
- if: ${{ failure() }}
51+
uses: elastic/oblt-actions/slack/send@v1
52+
with:
53+
bot-token: ${{ secrets.SLACK_BOT_TOKEN }}
54+
channel-id: "#ingest-notifications"
55+
message: ":traffic_cone: Update Stable snapshot: updatecli failed for `${{ github.repository }}@${{ github.ref_name }}`, `@agent-team` and @robots-ci please look what's going on <${{ env.JOB_URL }}|here>"

‎pkg/testing/ess/create_deployment_csp_configuration.yaml‎

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -13,3 +13,11 @@ aws:
1313
elasticsearch_conf_id: "aws.es.datahot.i3"
1414
elasticsearch_deployment_template_id: "aws-storage-optimized"
1515
kibana_instance_configuration_id: "aws.kibana.c5d"
16+
<<<<<<< HEAD
17+
=======
18+
# Automatically updated by updatecli-bump-stack-version.yml. Don't change the structure and key names
19+
docker:
20+
integration_server_image: "docker.elastic.co/cloud-release/elastic-agent-cloud:9.1.0-48398db3-SNAPSHOT"
21+
elasticsearch_image: "docker.elastic.co/cloud-release/elasticsearch-cloud-ess:9.1.0-48398db3-SNAPSHOT"
22+
kibana_image: "docker.elastic.co/cloud-release/kibana-cloud:9.1.0-48398db3-SNAPSHOT"
23+
>>>>>>> e728b39b5 ([CI] Stable stack version override (#8352))

‎test_infra/ess/deployment.tf‎

Lines changed: 17 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -61,7 +61,6 @@ locals {
6161

6262
ess_region = coalesce(var.ess_region, "gcp-us-east1")
6363
deployment_template_id = coalesce(var.deployment_template_id, "gcp-storage-optimized")
64-
6564
ess_properties = merge(
6665
{
6766
docker = {
@@ -73,10 +72,23 @@ locals {
7372
},
7473
yamldecode(file("${path.module}/../../pkg/testing/ess/create_deployment_csp_configuration.yaml")))
7574

76-
77-
integration_server_docker_image = coalesce(var.integration_server_docker_image, local.ess_properties.docker.integration_server_image, "docker.elastic.co/cloud-release/elastic-agent-cloud:${var.stack_version}")
78-
elasticsearch_docker_image = coalesce(var.elasticsearch_docker_image, local.ess_properties.docker.elasticsearch_image, "docker.elastic.co/cloud-release/elasticsearch-cloud-ess:${var.stack_version}")
79-
kibana_docker_image = coalesce(var.kibana_docker_image, local.ess_properties.docker.kibana_image, "docker.elastic.co/cloud-release/kibana-cloud:${var.stack_version}")
75+
integration_server_docker_image = coalesce(
76+
var.integration_server_docker_image,
77+
local.ess_properties.docker.integration_server_image,
78+
"docker.elastic.co/cloud-release/elastic-agent-cloud:${var.stack_version}"
79+
)
80+
81+
elasticsearch_docker_image = coalesce(
82+
var.elasticsearch_docker_image,
83+
local.ess_properties.docker.elasticsearch_image,
84+
"docker.elastic.co/cloud-release/elasticsearch-cloud-ess:${var.stack_version}"
85+
)
86+
87+
kibana_docker_image = coalesce(
88+
var.kibana_docker_image,
89+
local.ess_properties.docker.kibana_image,
90+
"docker.elastic.co/cloud-release/kibana-cloud:${var.stack_version}"
91+
)
8092
}
8193

8294
# If we have defined a stack version, validate that this version exists on that region and return it.

‎test_infra/ess/readme.md‎

Lines changed: 28 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,28 @@
1+
## Terraform-managed Elastic Stack deployment for integration tests
2+
3+
### Required variables
4+
`stack_version` - The version of the Elastic Stack to deploy, e.g., `9.1.0-SNAPSHOT`.
5+
6+
### Stable snapshots and images override
7+
This deployment uses the latest tested snapshots of the Elastic Stack components by default. However, you can override the Docker images used for the integration server, Elasticsearch, and Kibana.
8+
9+
`create_deployment_csp_configuration.yaml` contains the default configuration for the deployment, including the Docker images for the integration server, Elasticsearch, and Kibana. These images are automatically updated to the latest tested snapshots of the Elastic Stack components.
10+
* `integration_server_image` - The Docker image for the integration server
11+
* `elasticsearch_docker_image` - The Docker image for Elasticsearch
12+
* `kibana_docker_image` - The Docker image for Kibana
13+
14+
Alternatively, you can override these images by setting terraform variables by setting the following variables in your local `terraform.tfvars` file:
15+
```hcl
16+
integration_server_docker_image = "docker.elastic.co/cloud-release/elastic-agent-cloud:9.1.0-48398db3-SNAPSHOT"
17+
elasticsearch_docker_image = "docker.elastic.co/cloud-release/elasticsearch-cloud-ess:9.1.0-48398db3-SNAPSHOT"
18+
kibana_docker_image = "docker.elastic.co/cloud-release/kibana-cloud:9.1.0-48398db3-SNAPSHOT"
19+
```
20+
21+
or by setting the environment variables:
22+
```bash
23+
export TF_VAR_integration_server_docker_image="docker.elastic.co/cloud-release/elastic-agent-cloud:9.1.0-48398db3-SNAPSHOT"
24+
export TF_VAR_elasticsearch_docker_image="docker.elastic.co/cloud-release/elasticsearch-cloud-ess:9.1.0-48398db3-SNAPSHOT"
25+
export TF_VAR_kibana_docker_image="docker.elastic.co/cloud-release/kibana-cloud:9.1.0-48398db3-SNAPSHOT"
26+
```
27+
28+
Note: terraform variables take precedence over `create_deployment_csp_configuration.yaml` docker images configuration. Use `terraform.tfvars` and `TF_VAR_` environment variables to override the images if you don't want to modify the `create_deployment_csp_configuration.yaml` file.

0 commit comments

Comments
 (0)