Skip to content

Commit 57d0c6f

Browse files
mrodmmrodm
authored
Add docs to show how to use custom images (#1323)
Add docs to show how to use custom images for Elasticsearch, Kibana or Elastic Agent services when booting up a new Elastic stack.
1 parent e0b94d0 commit 57d0c6f

File tree

3 files changed

+137
-2
lines changed

3 files changed

+137
-2
lines changed

‎README.md‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -362,6 +362,8 @@ _Context: global_
362362

363363
Use this command to spin up a Docker-based Elastic Stack consisting of Elasticsearch, Kibana, and the Package Registry. By default the latest released version of the stack is spun up but it is possible to specify a different version, including SNAPSHOT versions by appending --version <version>.
364364

365+
You can run your own custom images for Elasticsearch, Kibana or Elastic Agent, see [this document](./docs/howto/custom_images.md).
366+
365367
Be aware that a common issue while trying to boot up the stack is that your Docker environments settings are too low in terms of memory threshold.
366368

367369
For details on how to connect the service with the Elastic stack, see the [service command](https://github.com/elastic/elastic-package/blob/main/README.md#elastic-package-service).
@@ -398,9 +400,11 @@ Use this command to boot up the stack locally.
398400

399401
By default the latest released version of the stack is spun up but it is possible to specify a different version, including SNAPSHOT versions by appending --version <version>.
400402

403+
You can run your own custom images for Elasticsearch, Kibana or Elastic Agent, see [this document](./docs/howto/custom_images.md).
404+
401405
Be aware that a common issue while trying to boot up the stack is that your Docker environments settings are too low in terms of memory threshold.
402406

403-
To expose local packages in the Package Registry, build them first and boot up the stack from inside of the Git repository containing the package (e.g. elastic/integrations). They will be copied to the development stack (~/.elastic-package/stack/development) and used to build a custom Docker image of the Package Registry.
407+
To expose local packages in the Package Registry, build them first and boot up the stack from inside of the Git repository containing the package (e.g. elastic/integrations). They will be copied to the development stack (~/.elastic-package/stack/development) and used to build a custom Docker image of the Package Registry. Starting with Elastic stack version >= 8.7.0, it is not mandatory to be available local packages in the Package Registry to run the tests.
404408

405409
For details on how to connect the service with the Elastic stack, see the [service command](https://github.com/elastic/elastic-package/blob/main/README.md#elastic-package-service).
406410

‎cmd/stack.go‎

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -28,6 +28,8 @@ var availableServices = map[string]struct{}{
2828

2929
const stackLongDescription = `Use this command to spin up a Docker-based Elastic Stack consisting of Elasticsearch, Kibana, and the Package Registry. By default the latest released version of the stack is spun up but it is possible to specify a different version, including SNAPSHOT versions by appending --version <version>.
3030
31+
You can run your own custom images for Elasticsearch, Kibana or Elastic Agent, see [this document](./docs/howto/custom_images.md).
32+
3133
Be aware that a common issue while trying to boot up the stack is that your Docker environments settings are too low in terms of memory threshold.
3234
3335
For details on how to connect the service with the Elastic stack, see the [service command](https://github.com/elastic/elastic-package/blob/main/README.md#elastic-package-service).`
@@ -36,9 +38,11 @@ const stackUpLongDescription = `Use this command to boot up the stack locally.
3638
3739
By default the latest released version of the stack is spun up but it is possible to specify a different version, including SNAPSHOT versions by appending --version <version>.
3840
41+
You can run your own custom images for Elasticsearch, Kibana or Elastic Agent, see [this document](./docs/howto/custom_images.md).
42+
3943
Be aware that a common issue while trying to boot up the stack is that your Docker environments settings are too low in terms of memory threshold.
4044
41-
To expose local packages in the Package Registry, build them first and boot up the stack from inside of the Git repository containing the package (e.g. elastic/integrations). They will be copied to the development stack (~/.elastic-package/stack/development) and used to build a custom Docker image of the Package Registry.
45+
To expose local packages in the Package Registry, build them first and boot up the stack from inside of the Git repository containing the package (e.g. elastic/integrations). They will be copied to the development stack (~/.elastic-package/stack/development) and used to build a custom Docker image of the Package Registry. Starting with Elastic stack version >= 8.7.0, it is not mandatory to be available local packages in the Package Registry to run the tests.
4246
4347
For details on how to connect the service with the Elastic stack, see the [service command](https://github.com/elastic/elastic-package/blob/main/README.md#elastic-package-service).`
4448

‎docs/howto/custom_images.md‎

Lines changed: 127 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,127 @@
1+
# Use your own custom images
2+
3+
By default, `elastic-package stack up` command starts the required docker images for each service.
4+
5+
Setting `--version` flag, you can start the Elastic stack using different versions, for instance SNAPSHOT version (e.g. 8.8.2-SNAPSHOT).
6+
If this flag is not set, the default version used is by elastic-package is defined [here](../../internal/install/stack_version.go).
7+
8+
There could be cases where you need to use your own custom images for development or debugging purposes.
9+
If you need to use your own custom docker images for your service (e.g. elastic-agent), this could be
10+
achieved in two different ways:
11+
- [defining environment variables](#using-environment-variables)
12+
- [updating elastic-package configuration file](#using-the-configuration-file)
13+
14+
If both ways are used, environment variables have preference to set the custom image.
15+
16+
The current images that could be overwritten are:
17+
18+
| Service | Environment Variable | Entry configuration file |
19+
| --- | --- | --- |
20+
| Elasticsearch | ELASTICSEARCH_IMAGE_REF_OVERRIDE | elasticsearch |
21+
| Kibana | KIBANA_IMAGE_REF_OVERRIDE | kibana |
22+
| Elastic Agent | ELASTIC_AGENT_IMAGE_REF_OVERRIDE | elastic-agent |
23+
24+
25+
For the following two examples, it will be used as example overwriting elastic-agent image.
26+
You can find here the instructions to create the docker images for the examples ([docs link](https://github.com/elastic/elastic-agent#packaging)):
27+
28+
```bash
29+
# Go to elastic-agent repository and create the packages for docker
30+
git checkout 8.8 # or the required tag like v8.8.*
31+
DEV=true SNAPSHOT=true EXTERNAL=true PLATFORMS=linux/amd64 PACKAGES=docker mage -v package
32+
33+
docker tag docker.elastic.co/beats/elastic-agent-complete:8.8.2-SNAPSHOT docker.elastic.co/beats/elastic-agent-complete:8.8.2-SNAPSHOT-test
34+
35+
git checkout v8.8.1
36+
DEV=true SNAPSHOT=true EXTERNAL=true PLATFORMS=linux/amd64 PACKAGES=docker mage -v package
37+
38+
docker tag docker.elastic.co/beats/elastic-agent-complete:8.8.1-SNAPSHOT docker.elastic.co/beats/elastic-agent-complete:8.8.1-SNAPSHOT-test
39+
docker tag docker.elastic.co/beats/elastic-agent-complete:8.8.1-SNAPSHOT docker.elastic.co/beats/elastic-agent-complete:8.8.1-SNAPSHOT-foo
40+
```
41+
42+
## Using Environment variables
43+
44+
Depending on the service you need to use a custom image, you need to set a different environment variable.
45+
46+
Following the example of elastic-agent, it is required to define the `ELASTICSEARCH_IMAGE_REF_OVERRIDE` environment variable before running `elastic-package stack up`:
47+
```
48+
export ELASTIC_AGENT_IMAGE_REF="docker.elastic.co/beats/elastic-agent-complete:8.8.2-SNAPSHOT-test"
49+
elastic-package stack up -v -d --version 8.8.0-SNAPSHOT
50+
```
51+
52+
Once the Elastic stack is running, you can check that the docker image used for the elastic-agent image is now the one it was built:
53+
54+
```bash
55+
$ docker ps --format "{{.Names}} {{.Image}}"
56+
elastic-package-stack-elastic-agent-1 docker.elastic.co/beats/elastic-agent-complete:8.8.2-SNAPSHOT-test
57+
elastic-package-stack-fleet-server-1 docker.elastic.co/beats/elastic-agent-complete:8.8.2-SNAPSHOT-test
58+
elastic-package-stack-kibana-1 docker.elastic.co/kibana/kibana:8.8.2-SNAPSHOT
59+
elastic-package-stack-elasticsearch-1 docker.elastic.co/elasticsearch/elasticsearch:8.8.2-SNAPSHOT
60+
elastic-package-stack-package-registry-1 elastic-package-stack-package-registry
61+
```
62+
63+
## Using the configuration file
64+
65+
These custom version or images can also be defined in the configuration file that is located at
66+
`~/.elastic-package/config.yml`.
67+
68+
```yaml
69+
stack:
70+
image_ref_overrides: {}
71+
profile:
72+
current: default
73+
```
74+
75+
To define the same elastic-agent image as in the previous section without any environment variable,
76+
it should be added a map for each Elastic stack version (e.g. 8.8.2-SNAPSHOT) under `image_ref_overrides` key.
77+
Each Elastic stack map would contain the required custom images.
78+
79+
It is important to note that Elastic stack version set in this configuration file must match with the version
80+
set in the `--version` flag.
81+
82+
For instance, the following configuration file sets a custom image for Elastic stack versions `8.8.2-SNAPSHOT` and `8.8.1-SNAPSHOT`:
83+
84+
```yaml
85+
stack:
86+
image_ref_overrides:
87+
8.8.2-SNAPSHOT:
88+
elastic-agent: docker.elastic.co/beats/elastic-agnet-complete:8.8-2-SNAPSHOT-test
89+
# elasticsearch: <elasticsearch_image>
90+
# kibana: <kibana_image>
91+
8.8.1-SNAPSHOT:
92+
elastic-agent: docker.elastic.co/beats/elastic-agnet-complete:8.8-1-SNAPSHOT-test
93+
# elasticsearch: <elasticsearch_image>
94+
# kibana: <kibana_image>
95+
profile:
96+
current: default
97+
```
98+
99+
If Elastic stack version `8.8.1` is started, then there would not be any custom image spun up.
100+
101+
Some examples with the given configuration file:
102+
103+
```bash
104+
$ elastic-stack up -v --version 8.8.1-SNAPSHOT
105+
$ docker ps --format "{{.Names}} {{.Image}}"
106+
elastic-package-stack-elastic-agent-1 docker.elastic.co/beats/elastic-agent-complete:8.8.1-SNAPSHOT-test
107+
elastic-package-stack-fleet-server-1 docker.elastic.co/beats/elastic-agent-complete:8.8.1-SNAPSHOT-test
108+
elastic-package-stack-kibana-1 docker.elastic.co/kibana/kibana:8.8.1-SNAPSHOT
109+
elastic-package-stack-elasticsearch-1 docker.elastic.co/elasticsearch/elasticsearch:8.8.1-SNAPSHOT
110+
elastic-package-stack-package-registry-1 elastic-package-stack-package-registry
111+
112+
$ elastic-stack up -v --version 8.8.2-SNAPSHOT
113+
$ docker ps --format "{{.Names}} {{.Image}}"
114+
elastic-package-stack-elastic-agent-1 docker.elastic.co/beats/elastic-agent-complete:8.8.2-SNAPSHOT-test
115+
elastic-package-stack-fleet-server-1 docker.elastic.co/beats/elastic-agent-complete:8.8.2-SNAPSHOT-test
116+
elastic-package-stack-kibana-1 docker.elastic.co/kibana/kibana:8.8.2-SNAPSHOT
117+
elastic-package-stack-elasticsearch-1 docker.elastic.co/elasticsearch/elasticsearch:8.8.2-SNAPSHOT
118+
elastic-package-stack-package-registry-1 elastic-package-stack-package-registry
119+
120+
$ elastic-stack up -v --version 8.8.1
121+
$ docker ps --format "{{.Names}} {{.Image}}"
122+
elastic-package-stack-elastic-agent-1 docker.elastic.co/elastic-agent/elastic-agent-complete:8.8.1
123+
elastic-package-stack-fleet-server-1 docker.elastic.co/elastic-agent/elastic-agent-complete:8.8.1
124+
elastic-package-stack-kibana-1 docker.elastic.co/kibana/kibana:8.8.1
125+
elastic-package-stack-elasticsearch-1 docker.elastic.co/elasticsearch/elasticsearch:8.8.1
126+
elastic-package-stack-package-registry-1 elastic-package-stack-package-registry
127+
```

0 commit comments

Comments
 (0)