Skip to content

Commit ba7cd4d

Browse files
ldematteldematte
committed
Merge remote-tracking branch 'upstream/main' into awssdkv2-string-format-patcher
2 parents f9980f2 + 8cb4493 commit ba7cd4d

File tree

1,822 files changed

+36433
-18377
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

1,822 files changed

+36433
-18377
lines changed

‎.buildkite/pipelines/pull-request/build-benchmark.yml‎

Lines changed: 8 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -7,16 +7,19 @@ steps:
77
command: |
88
.ci/scripts/run-gradle.sh :build-tools-internal:bootstrapPerformanceTests
99
.ci/scripts/install-gradle-profiler.sh
10-
.ci/scripts/run-gradle-profiler.sh --benchmark --scenario-file build-tools-internal/build/performanceTests/elasticsearch-build-benchmark-{{matrix.part}}.scenarios --project-dir . --output-dir profile-out
11-
mkdir build
12-
tar -czf build/$BUILDKITE_BUILD_NUMBER.tar.bz2 profile-out
10+
.ci/scripts/run-gradle-profiler.sh --benchmark --scenario-file build-tools-internal/build/performanceTests/elasticsearch-{{matrix.part}}.scenarios --measure-config-time --project-dir . --output-dir profile-out
11+
mkdir -p build
12+
buildkite-agent artifact upload "profile-out/*"
13+
cat << EOF | buildkite-agent annotate --style "info"
14+
Benchmark Html Report <a href="artifact://profile-out/benchmark.html">Benchmark Html Report</a>
15+
EOF
1316
matrix:
1417
setup:
1518
part:
16-
- part1
17-
- part2
19+
- esql-tests
1820
env:
1921
BUILD_PERFORMANCE_TEST: "true"
22+
2023
agents:
2124
provider: gcp
2225
image: family/elasticsearch-ubuntu-2004

‎.buildkite/pipelines/pull-request/part-1-fips.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
config:
2-
allow-labels: "Team:Security"
2+
allow-labels:
3+
- Team:Security
4+
- test-fips
35
steps:
46
- label: part-1-fips
57
command: .ci/scripts/run-gradle.sh -Dignore.tests.seed -Dtests.fips.enabled=true checkPart1

‎.buildkite/pipelines/pull-request/part-2-fips.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
config:
2-
allow-labels: "Team:Security"
2+
allow-labels:
3+
- Team:Security
4+
- test-fips
35
steps:
46
- label: part-2-fips
57
command: .ci/scripts/run-gradle.sh -Dignore.tests.seed -Dtests.fips.enabled=true checkPart2

‎.buildkite/pipelines/pull-request/part-3-fips.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
config:
2-
allow-labels: "Team:Security"
2+
allow-labels:
3+
- Team:Security
4+
- test-fips
35
steps:
46
- label: part-3-fips
57
command: .ci/scripts/run-gradle.sh -Dignore.tests.seed -Dtests.fips.enabled=true checkPart3

‎.buildkite/pipelines/pull-request/part-4-fips.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
config:
2-
allow-labels: "Team:Security"
2+
allow-labels:
3+
- Team:Security
4+
- test-fips
35
steps:
46
- label: part-4-fips
57
command: .ci/scripts/run-gradle.sh -Dignore.tests.seed -Dtests.fips.enabled=true checkPart4

‎.buildkite/pipelines/pull-request/part-5-fips.yml‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,7 @@
11
config:
2-
allow-labels: "Team:Security"
2+
allow-labels:
3+
- Team:Security
4+
- test-fips
35
steps:
46
- label: part-5-fips
57
command: .ci/scripts/run-gradle.sh -Dignore.tests.seed -Dtests.fips.enabled=true checkPart5

‎.buildkite/scripts/dra-workflow.sh‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@ echo --- Building release artifacts
7070
$VERSION_QUALIFIER_ARG \
7171
buildReleaseArtifacts \
7272
exportCompressedDockerImages \
73+
exportDockerContexts \
7374
:distribution:generateDependenciesReport
7475

7576
PATH="$PATH:${JAVA_HOME}/bin" # Required by the following script

‎.ci/scripts/install-gradle-profiler.sh‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -2,7 +2,7 @@
22

33
set -e
44
# profiler version we wanna install
5-
PROFILER_VERSION="0.16.0"
6-
wget https://repo.gradle.org/gradle/ext-releases-local/org/gradle/profiler/gradle-profiler/$PROFILER_VERSION/gradle-profiler-$PROFILER_VERSION.zip -O $WORKSPACE/gradle-profiler-$PROFILER_VERSION.zip
5+
PROFILER_VERSION="0.22.0"
6+
wget https://repo1.maven.org/maven2/org/gradle/profiler/gradle-profiler/$PROFILER_VERSION/gradle-profiler-$PROFILER_VERSION.zip -O $WORKSPACE/gradle-profiler-$PROFILER_VERSION.zip
77
unzip $WORKSPACE/gradle-profiler-$PROFILER_VERSION.zip
88
mv $WORKSPACE/gradle-profiler-$PROFILER_VERSION $WORKSPACE/gradle-profiler

‎.ci/scripts/packaging-test.sh‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,7 +3,7 @@
33
# opensuse 15 has a missing dep for systemd
44

55
if which zypper > /dev/null ; then
6-
sudo zypper install -y insserv-compat
6+
sudo zypper install -y insserv-compat docker-buildx
77
fi
88

99
if [ -e /etc/sysctl.d/99-gce.conf ]; then

‎benchmarks/README.md‎

Lines changed: 31 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -82,19 +82,21 @@ To get realistic results, you should exercise care when running benchmarks. Here
8282
NOTE: Linux only. Sorry Mac and Windows.
8383

8484
Disassembling is fun! Maybe not always useful, but always fun! Generally, you'll want to install `perf` and the JDK's `hsdis`.
85-
`perf` is generally available via `apg-get install perf` or `pacman -S perf`. `hsdis` you'll want to compile from source. is a little more involved. This worked
85+
`perf` is generally available via `apg-get install perf` or `pacman -S perf linux-tools`. `hsdis` you'll want to compile from source. is a little more involved. This worked
8686
on 2020-08-01:
8787

8888
```
8989
git clone [email protected]:openjdk/jdk.git
9090
cd jdk
91-
git checkout jdk-17-ga
92-
cd src/utils/hsdis
91+
git checkout jdk-24-ga
9392
# Get a known good binutils
9493
wget https://ftp.gnu.org/gnu/binutils/binutils-2.35.tar.gz
9594
tar xf binutils-2.35.tar.gz
96-
make BINUTILS=binutils-2.35 ARCH=amd64
97-
sudo cp build/linux-amd64/hsdis-amd64.so /usr/lib/jvm/java-17-openjdk/lib/server/
95+
bash configure --with-hsdis=binutils --with-binutils-src=binutils-2.35 \
96+
--with-boot-jdk=~/.gradle/jdks/oracle_corporation-24-amd64-linux.2
97+
make build-hsdis
98+
cp ./build/linux-x86_64-server-release/jdk/lib/hsdis-amd64.so \
99+
~/.gradle/jdks/oracle_corporation-24-amd64-linux.2/lib/hsdis.so
98100
```
99101

100102
If you want to disassemble a single method do something like this:
@@ -105,6 +107,30 @@ gradlew -p benchmarks run --args ' MemoryStatsBenchmark -jvmArgs "-XX:+UnlockDia
105107

106108
If you want `perf` to find the hot methods for you, then do add `-prof perfasm`.
107109

110+
NOTE: `perfasm` will need more access:
111+
```
112+
sudo bash
113+
echo -1 > /proc/sys/kernel/perf_event_paranoid
114+
exit
115+
```
116+
117+
If you get warnings like:
118+
```
119+
The perf event count is suspiciously low (0).
120+
```
121+
then check if you are bumping into [this](https://man.archlinux.org/man/perf-stat.1.en#INTEL_HYBRID_SUPPORT)
122+
by running:
123+
```
124+
perf stat -B dd if=/dev/zero of=/dev/null count=1000000
125+
```
126+
127+
If you see lines like:
128+
```
129+
765019980 cpu_atom/cycles/ # 1.728 GHz (0.60%)
130+
2258845959 cpu_core/cycles/ # 5.103 GHz (99.18%)
131+
```
132+
then `perf` is just not going to work for you.
133+
108134
## Async Profiler
109135

110136
Note: Linux and Mac only. Sorry Windows.

0 commit comments

Comments
 (0)