Skip to content

Commit 874d716

Browse files
Merge branch 'main' into ml-allow-null-function-name
2 parents f665d9f + 923a8e1 commit 874d716

File tree

160 files changed

+4583
-1422
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

160 files changed

+4583
-1422
lines changed

README.asciidoc

Lines changed: 14 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -97,6 +97,15 @@ From the `elastic-start-local` folder, check the connection to Elasticsearch usi
9797
source .env
9898
curl $ES_LOCAL_URL -H "Authorization: ApiKey ${ES_LOCAL_API_KEY}"
9999
----
100+
101+
To use the password for the `elastic` user, set and export the `ES_LOCAL_PASSWORD` environment variable. For example:
102+
103+
[source,sh]
104+
----
105+
source .env
106+
export ES_LOCAL_PASSWORD
107+
----
108+
100109
// NOTCONSOLE
101110

102111
=== Send requests to Elasticsearch
@@ -112,22 +121,23 @@ Here's an example curl command to create a new Elasticsearch index, using basic
112121

113122
[source,sh]
114123
----
115-
curl -u elastic:$ELASTIC_PASSWORD \
124+
curl -u elastic:$ES_LOCAL_PASSWORD \
116125
-X PUT \
117126
http://localhost:9200/my-new-index \
118127
-H 'Content-Type: application/json'
119128
----
129+
120130
// NOTCONSOLE
121131

122132
==== Using a language client
123133

124-
To connect to your local dev Elasticsearch cluster with a language client, you can use basic authentication with the `elastic` username and the password you set in the environment variable.
134+
To connect to your local dev Elasticsearch cluster with a language client, you can use basic authentication with the `elastic` username and the password stored in the `ES_LOCAL_PASSWORD` environment variable.
125135

126136
You'll use the following connection details:
127137

128138
* **Elasticsearch endpoint**: `http://localhost:9200`
129139
* **Username**: `elastic`
130-
* **Password**: `$ELASTIC_PASSWORD` (Value you set in the environment variable)
140+
* **Password**: `$ES_LOCAL_PASSWORD` (Value you set in the environment variable)
131141

132142
For example, to connect with the Python `elasticsearch` client:
133143

@@ -137,7 +147,7 @@ import os
137147
from elasticsearch import Elasticsearch
138148
139149
username = 'elastic'
140-
password = os.getenv('ELASTIC_PASSWORD') # Value you set in the environment variable
150+
password = os.getenv('ES_LOCAL_PASSWORD') # Value you set in the environment variable
141151
142152
client = Elasticsearch(
143153
"http://localhost:9200",

TESTING.asciidoc

Lines changed: 8 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -46,14 +46,8 @@ run it using Gradle:
4646
==== Launching and debugging from an IDE
4747

4848
If you want to run and debug Elasticsearch from your IDE, the `./gradlew run` task
49-
supports a remote debugging option. Run the following from your terminal:
50-
51-
---------------------------------------------------------------------------
52-
./gradlew run --debug-jvm
53-
---------------------------------------------------------------------------
54-
55-
Next start the "Debug Elasticsearch" run configuration in IntelliJ. This will enable the IDE to connect to the process and allow debug functionality.
56-
49+
supports a remote debugging option. Start the "Debug Elasticsearch" run configuration in IntelliJ. This will enable the
50+
IDE to connect to the process and allow debug functionality.
5751

5852
As such the IDE needs to be instructed to listen for connections on the debug port.
5953
Since we might run multiple JVMs as part of configuring and starting the cluster it's
@@ -64,6 +58,12 @@ NOTE: If you have imported the project into IntelliJ according to the instructio
6458
link:/CONTRIBUTING.md#importing-the-project-into-intellij-idea[CONTRIBUTING.md] then a debug run configuration
6559
named "Debug Elasticsearch" will be created for you and configured appropriately.
6660

61+
Next run the following from your terminal:
62+
63+
---------------------------------------------------------------------------
64+
./gradlew run --debug-jvm
65+
---------------------------------------------------------------------------
66+
6767
===== Debugging the CLI launcher
6868

6969
The gradle task does not start the Elasticsearch server process directly; like in the Elasticsearch distribution,

build-conventions/settings.gradle

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -8,7 +8,7 @@
88
*/
99

1010
plugins {
11-
id "com.gradle.develocity" version "3.19.2"
11+
id "com.gradle.develocity" version "4.0.1"
1212
}
1313

1414
rootProject.name = 'build-conventions'

build-tools-internal/settings.gradle

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -9,7 +9,7 @@ pluginManagement {
99
}
1010

1111
plugins {
12-
id "com.gradle.develocity" version "3.19.2"
12+
id "com.gradle.develocity" version "4.0.1"
1313
}
1414

1515
dependencyResolutionManagement {

build-tools-internal/src/main/java/org/elasticsearch/gradle/internal/DockerBase.java

Lines changed: 5 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -14,30 +14,27 @@
1414
*/
1515
public enum DockerBase {
1616
// "latest" here is intentional, since the image name specifies "9"
17-
DEFAULT("redhat/ubi9-minimal:latest", "", "microdnf", "Dockerfile.default"),
17+
DEFAULT("redhat/ubi9-minimal:latest", "", "microdnf", "dockerfiles/default/Dockerfile"),
1818

1919
// The Iron Bank base image is UBI (albeit hardened), but we are required to parameterize the Docker build
2020
IRON_BANK("${BASE_REGISTRY}/${BASE_IMAGE}:${BASE_TAG}", "-ironbank", "yum", "Dockerfile"),
2121

2222
// Chainguard based wolfi image with latest jdk
23-
// This is usually updated via renovatebot
24-
// spotless:off
2523
WOLFI(
26-
"docker.elastic.co/wolfi/chainguard-base:latest@sha256:29150cd940cc7f69407d978d5a19c86f4d9e67cf44e4d6ded787a497e8f27c9a",
24+
null,
2725
"-wolfi",
2826
"apk",
29-
"Dockerfile"
27+
"dockerfiles/wolfi/Dockerfile"
3028
),
31-
// spotless:on
3229
// Based on WOLFI above, with more extras. We don't set a base image because
3330
// we programmatically extend from the wolfi image.
3431
CLOUD_ESS(null, "-cloud-ess", "apk", "Dockerfile.ess"),
3532

3633
CLOUD_ESS_FIPS(
37-
"docker.elastic.co/wolfi/chainguard-base-fips:sha256-ebfc3f1d7dba992231747a2e05ad1b859843e81b5e676ad342859d7cf9e425a7@sha256:ebfc3f1d7dba992231747a2e05ad1b859843e81b5e676ad342859d7cf9e425a7",
34+
null,
3835
"-cloud-ess-fips",
3936
"apk",
40-
"Dockerfile.ess-fips"
37+
"dockerfiles/cloud_ess_fips/Dockerfile"
4138
);
4239

4340
private final String image;

build-tools/settings.gradle

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -10,7 +10,7 @@ pluginManagement {
1010
includeBuild "../build-conventions"
1111
}
1212
plugins {
13-
id "com.gradle.develocity" version "3.19.2"
13+
id "com.gradle.develocity" version "4.0.1"
1414
}
1515
include 'reaper'
1616

distribution/docker/build.gradle

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -482,8 +482,10 @@ void addBuildDockerImageTask(Architecture architecture, DockerBase base) {
482482

483483
baseImages = [baseImage]
484484
buildArgs = buildArgsMap
485-
} else {
485+
} else if(base.image != null) {
486486
baseImages = [base.image]
487+
} else {
488+
baseImages = []
487489
}
488490

489491
Provider<DockerSupportService> serviceProvider = GradleUtils.getBuildService(

distribution/docker/src/docker/Dockerfile.ess-fips renamed to distribution/docker/src/docker/dockerfiles/cloud_ess_fips/Dockerfile

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,7 +24,7 @@
2424
# Extract Elasticsearch artifact
2525
################################################################################
2626
27-
FROM ${base_image} AS builder
27+
FROM docker.elastic.co/wolfi/chainguard-base-fips:latest@sha256:ebfc3f1d7dba992231747a2e05ad1b859843e81b5e676ad342859d7cf9e425a7 AS builder
2828
2929
# Install required packages to extract the Elasticsearch distribution
3030
RUN <%= retry.loop(package_manager, "export DEBIAN_FRONTEND=noninteractive && ${package_manager} update && ${package_manager} update && ${package_manager} add --no-cache curl") %>
@@ -103,7 +103,7 @@ WORKDIR /usr/share/elasticsearch/config
103103
# Add entrypoint
104104
################################################################################
105105

106-
FROM ${base_image}
106+
FROM docker.elastic.co/wolfi/chainguard-base-fips:latest@sha256:ebfc3f1d7dba992231747a2e05ad1b859843e81b5e676ad342859d7cf9e425a7
107107

108108
RUN <%= retry.loop(package_manager,
109109
"export DEBIAN_FRONTEND=noninteractive && \n" +
Lines changed: 176 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,176 @@
1+
################################################################################
2+
# This Dockerfile was generated from the template at distribution/src/docker/Dockerfile
3+
#
4+
# Beginning of multi stage Dockerfile
5+
################################################################################
6+
7+
<% /*
8+
This file is passed through Groovy's SimpleTemplateEngine, so dollars and backslashes
9+
have to be escaped in order for them to appear in the final Dockerfile. You
10+
can also comment out blocks, like this one. See:
11+
12+
https://docs.groovy-lang.org/latest/html/api/groovy/text/SimpleTemplateEngine.html
13+
14+
We use control-flow tags in this file to conditionally render the content. The
15+
layout/presentation here has been adjusted so that it looks reasonable when rendered,
16+
at the slight expense of how it looks here.
17+
18+
Note that this file is also filtered to squash together newlines, so we can
19+
add as many newlines here as necessary to improve legibility.
20+
*/ %>
21+
22+
################################################################################
23+
# Build stage 1 `builder`:
24+
# Extract Elasticsearch artifact
25+
################################################################################
26+
27+
FROM docker.elastic.co/wolfi/chainguard-base:latest@sha256:29150cd940cc7f69407d978d5a19c86f4d9e67cf44e4d6ded787a497e8f27c9a AS builder
28+
29+
# Install required packages to extract the Elasticsearch distribution
30+
RUN <%= retry.loop(package_manager, "export DEBIAN_FRONTEND=noninteractive && ${package_manager} update && ${package_manager} update && ${package_manager} add --no-cache curl") %>
31+
32+
# `tini` is a tiny but valid init for containers. This is used to cleanly
33+
# control how ES and any child processes are shut down.
34+
# For wolfi we pick it from the blessed wolfi package registry.
35+
#
36+
# The tini GitHub page gives instructions for verifying the binary using
37+
# gpg, but the keyservers are slow to return the key and this can fail the
38+
# build. Instead, we check the binary against the published checksum.
39+
40+
RUN mkdir /usr/share/elasticsearch
41+
WORKDIR /usr/share/elasticsearch
42+
43+
44+
# Fetch the appropriate Elasticsearch distribution for this architecture.
45+
# Keep this command on one line - it is replaced with a `COPY` during local builds.
46+
# It uses the `arch` shell command to fetch the correct distro for the build machine,
47+
RUN curl --retry 10 -S -L --output /tmp/elasticsearch.tar.gz https://artifacts-no-kpi.elastic.co/downloads/elasticsearch/elasticsearch-${version}-linux-\${arch}.tar.gz
48+
49+
RUN tar -zxf /tmp/elasticsearch.tar.gz --strip-components=1
50+
51+
# The distribution includes a `config` directory, no need to create it
52+
COPY ${config_dir}/elasticsearch.yml config/
53+
COPY ${config_dir}/log4j2.properties config/log4j2.docker.properties
54+
55+
# 1. Configure the distribution for Docker
56+
# 2. Create required directory
57+
# 3. Move the distribution's default logging config aside
58+
# 4. Move the generated docker logging config so that it is the default
59+
# 5. Reset permissions on all directories
60+
# 6. Reset permissions on all files
61+
# 7. Make CLI tools executable
62+
# 8. Make some directories writable. `bin` must be writable because
63+
# plugins can install their own CLI utilities.
64+
# 9. Make some files writable
65+
RUN sed -i -e 's/ES_DISTRIBUTION_TYPE=tar/ES_DISTRIBUTION_TYPE=docker/' bin/elasticsearch-env && \\
66+
mkdir data && \\
67+
mv config/log4j2.properties config/log4j2.file.properties && \\
68+
mv config/log4j2.docker.properties config/log4j2.properties && \\
69+
find . -type d -exec chmod 0555 {} + && \\
70+
find . -type f -exec chmod 0444 {} + && \\
71+
chmod 0555 bin/* jdk/bin/* jdk/lib/jspawnhelper modules/x-pack-ml/platform/linux-*/bin/* && \\
72+
chmod 0775 bin config config/jvm.options.d data logs plugins && \\
73+
find config -type f -exec chmod 0664 {} +
74+
75+
################################################################################
76+
# Build stage 2 (the actual Elasticsearch image):
77+
#
78+
# Copy elasticsearch from stage 1
79+
# Add entrypoint
80+
################################################################################
81+
82+
FROM docker.elastic.co/wolfi/chainguard-base:latest@sha256:29150cd940cc7f69407d978d5a19c86f4d9e67cf44e4d6ded787a497e8f27c9a
83+
84+
RUN <%= retry.loop(package_manager,
85+
"export DEBIAN_FRONTEND=noninteractive && \n" +
86+
" ${package_manager} update && \n" +
87+
" ${package_manager} upgrade && \n" +
88+
" ${package_manager} add --no-cache \n" +
89+
" bash java-cacerts curl libstdc++ libsystemd netcat-openbsd p11-kit p11-kit-trust posix-libc-utils shadow tini unzip zip zstd && \n" +
90+
" rm -rf /var/cache/apk/* "
91+
) %>
92+
93+
# Set Bash as the default shell for future commands
94+
SHELL ["/bin/bash", "-c"]
95+
96+
# Optionally set Bash as the default shell in the container at runtime
97+
CMD ["/bin/bash"]
98+
99+
RUN groupadd -g 1000 elasticsearch && \
100+
adduser -G elasticsearch -u 1000 elasticsearch -D --home /usr/share/elasticsearch elasticsearch && \
101+
adduser elasticsearch root && \
102+
chown -R 0:0 /usr/share/elasticsearch
103+
104+
ENV ELASTIC_CONTAINER=true
105+
106+
WORKDIR /usr/share/elasticsearch
107+
108+
COPY --from=builder --chown=0:0 /usr/share/elasticsearch /usr/share/elasticsearch
109+
110+
ENV PATH=/usr/share/elasticsearch/bin:\$PATH
111+
ENV SHELL=/bin/bash
112+
COPY ${bin_dir}/docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
113+
114+
# 1. Sync the user and group permissions of /etc/passwd
115+
# 2. Set correct permissions of the entrypoint
116+
# 3. Ensure that there are no files with setuid or setgid, in order to mitigate "stackclash" attacks.
117+
# We've already run this in previous layers so it ought to be a no-op.
118+
# 4. Replace OpenJDK's built-in CA certificate keystore with the one from the OS
119+
# vendor. The latter is superior in several ways.
120+
# REF: https://github.com/elastic/elasticsearch-docker/issues/171
121+
# 5. Tighten up permissions on the ES home dir (the permissions of the contents are handled earlier)
122+
# 6. You can't install plugins that include configuration when running as `elasticsearch` and the `config`
123+
# dir is owned by `root`, because the installed tries to manipulate the permissions on the plugin's
124+
# config directory.
125+
RUN chmod g=u /etc/passwd && \\
126+
chmod 0555 /usr/local/bin/docker-entrypoint.sh && \\
127+
find / -xdev -perm -4000 -exec chmod ug-s {} + && \\
128+
chmod 0775 /usr/share/elasticsearch && \\
129+
chown elasticsearch bin config config/jvm.options.d data logs plugins
130+
131+
RUN ln -sf /etc/ssl/certs/java/cacerts /usr/share/elasticsearch/jdk/lib/security/cacerts
132+
133+
EXPOSE 9200 9300
134+
135+
136+
LABEL org.label-schema.build-date="${build_date}" \\
137+
org.label-schema.license="${license}" \\
138+
org.label-schema.name="Elasticsearch" \\
139+
org.label-schema.schema-version="1.0" \\
140+
org.label-schema.url="https://www.elastic.co/products/elasticsearch" \\
141+
org.label-schema.usage="https://www.elastic.co/guide/en/elasticsearch/reference/index.html" \\
142+
org.label-schema.vcs-ref="${git_revision}" \\
143+
org.label-schema.vcs-url="https://github.com/elastic/elasticsearch" \\
144+
org.label-schema.vendor="Elastic" \\
145+
org.label-schema.version="${version}" \\
146+
org.opencontainers.image.created="${build_date}" \\
147+
org.opencontainers.image.documentation="https://www.elastic.co/guide/en/elasticsearch/reference/index.html" \\
148+
org.opencontainers.image.licenses="${license}" \\
149+
org.opencontainers.image.revision="${git_revision}" \\
150+
org.opencontainers.image.source="https://github.com/elastic/elasticsearch" \\
151+
org.opencontainers.image.title="Elasticsearch" \\
152+
org.opencontainers.image.url="https://www.elastic.co/products/elasticsearch" \\
153+
org.opencontainers.image.vendor="Elastic" \\
154+
org.opencontainers.image.version="${version}"
155+
156+
LABEL name="Elasticsearch" \\
157+
maintainer="[email protected]" \\
158+
vendor="Elastic" \\
159+
version="${version}" \\
160+
release="1" \\
161+
summary="Elasticsearch" \\
162+
description="You know, for search."
163+
164+
RUN mkdir /licenses && ln LICENSE.txt /licenses/LICENSE
165+
166+
# Our actual entrypoint is `tini`, a minimal but functional init program. It
167+
# calls the entrypoint we provide, while correctly forwarding signals.
168+
ENTRYPOINT ["/sbin/tini", "--", "/usr/local/bin/docker-entrypoint.sh"]
169+
# Dummy overridable parameter parsed by entrypoint
170+
CMD ["eswrapper"]
171+
172+
USER 1000:0
173+
174+
################################################################################
175+
# End of multi-stage Dockerfile
176+
################################################################################

0 commit comments

Comments
 (0)