Skip to content

Commit 7e1e45e

Browse files
authored
ESQL: Speed up TO_IP (#126338)
Speed up the TO_IP method by converting directly from utf-8 encoded strings to the ip encoding. Previously we did: ``` utf-8 -> String -> INetAddress -> ip encoding ``` In a step towards solving #125460 this creates three IP parsing functions, one the rejects leading zeros, one that interprets leading zeros as decimal numbers, and one the interprets leading zeros as octal numbers. IPs have historically been parsed in all three of those ways. This plugs the "rejects leading zeros" parser into `TO_IP` because that's the behavior it had before. Here is the performance: ``` Benchmark Score Error Units leadingZerosAreDecimal 14.007 ± 0.093 ns/op leadingZerosAreOctal 15.020 ± 0.373 ns/op leadingZerosRejected 14.176 ± 3.861 ns/op original 32.950 ± 1.062 ns/op ``` So this is roughly 45% faster than what we had.
1 parent 72066ea commit 7e1e45e

File tree

10 files changed

+863
-32
lines changed

10 files changed

+863
-32
lines changed

benchmarks/README.md

Lines changed: 31 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -82,19 +82,21 @@ To get realistic results, you should exercise care when running benchmarks. Here
8282
NOTE: Linux only. Sorry Mac and Windows.
8383

8484
Disassembling is fun! Maybe not always useful, but always fun! Generally, you'll want to install `perf` and the JDK's `hsdis`.
85-
`perf` is generally available via `apg-get install perf` or `pacman -S perf`. `hsdis` you'll want to compile from source. is a little more involved. This worked
85+
`perf` is generally available via `apg-get install perf` or `pacman -S perf linux-tools`. `hsdis` you'll want to compile from source. is a little more involved. This worked
8686
on 2020-08-01:
8787

8888
```
8989
git clone [email protected]:openjdk/jdk.git
9090
cd jdk
91-
git checkout jdk-17-ga
92-
cd src/utils/hsdis
91+
git checkout jdk-24-ga
9392
# Get a known good binutils
9493
wget https://ftp.gnu.org/gnu/binutils/binutils-2.35.tar.gz
9594
tar xf binutils-2.35.tar.gz
96-
make BINUTILS=binutils-2.35 ARCH=amd64
97-
sudo cp build/linux-amd64/hsdis-amd64.so /usr/lib/jvm/java-17-openjdk/lib/server/
95+
bash configure --with-hsdis=binutils --with-binutils-src=binutils-2.35 \
96+
--with-boot-jdk=~/.gradle/jdks/oracle_corporation-24-amd64-linux.2
97+
make build-hsdis
98+
cp ./build/linux-x86_64-server-release/jdk/lib/hsdis-amd64.so \
99+
~/.gradle/jdks/oracle_corporation-24-amd64-linux.2/lib/hsdis.so
98100
```
99101

100102
If you want to disassemble a single method do something like this:
@@ -105,6 +107,30 @@ gradlew -p benchmarks run --args ' MemoryStatsBenchmark -jvmArgs "-XX:+UnlockDia
105107

106108
If you want `perf` to find the hot methods for you, then do add `-prof perfasm`.
107109

110+
NOTE: `perfasm` will need more access:
111+
```
112+
sudo bash
113+
echo -1 > /proc/sys/kernel/perf_event_paranoid
114+
exit
115+
```
116+
117+
If you get warnings like:
118+
```
119+
The perf event count is suspiciously low (0).
120+
```
121+
then check if you are bumping into [this](https://man.archlinux.org/man/perf-stat.1.en#INTEL_HYBRID_SUPPORT)
122+
by running:
123+
```
124+
perf stat -B dd if=/dev/zero of=/dev/null count=1000000
125+
```
126+
127+
If you see lines like:
128+
```
129+
765019980 cpu_atom/cycles/ # 1.728 GHz (0.60%)
130+
2258845959 cpu_core/cycles/ # 5.103 GHz (99.18%)
131+
```
132+
then `perf` is just not going to work for you.
133+
108134
## Async Profiler
109135

110136
Note: Linux and Mac only. Sorry Windows.
Lines changed: 61 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,61 @@
1+
/*
2+
* Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
3+
* or more contributor license agreements. Licensed under the "Elastic License
4+
* 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
5+
* Public License v 1"; you may not use this file except in compliance with, at
6+
* your election, the "Elastic License 2.0", the "GNU Affero General Public
7+
* License v3.0 only", or the "Server Side Public License, v 1".
8+
*/
9+
10+
package org.elasticsearch.benchmark.compute.operator;
11+
12+
import org.apache.lucene.document.InetAddressPoint;
13+
import org.apache.lucene.util.BytesRef;
14+
import org.elasticsearch.common.breaker.NoopCircuitBreaker;
15+
import org.elasticsearch.common.network.InetAddresses;
16+
import org.elasticsearch.compute.operator.BreakingBytesRefBuilder;
17+
import org.elasticsearch.xpack.esql.expression.function.scalar.convert.ParseIp;
18+
import org.openjdk.jmh.annotations.Benchmark;
19+
import org.openjdk.jmh.annotations.BenchmarkMode;
20+
import org.openjdk.jmh.annotations.Fork;
21+
import org.openjdk.jmh.annotations.Measurement;
22+
import org.openjdk.jmh.annotations.Mode;
23+
import org.openjdk.jmh.annotations.OutputTimeUnit;
24+
import org.openjdk.jmh.annotations.Scope;
25+
import org.openjdk.jmh.annotations.State;
26+
import org.openjdk.jmh.annotations.Warmup;
27+
28+
import java.net.InetAddress;
29+
import java.util.concurrent.TimeUnit;
30+
31+
@Warmup(iterations = 5)
32+
@Measurement(iterations = 7)
33+
@BenchmarkMode(Mode.AverageTime)
34+
@OutputTimeUnit(TimeUnit.NANOSECONDS)
35+
@State(Scope.Thread)
36+
@Fork(1)
37+
public class ParseIpBenchmark {
38+
private final BytesRef ip = new BytesRef("192.168.0.1");
39+
private final BreakingBytesRefBuilder scratch = ParseIp.buildScratch(new NoopCircuitBreaker("request"));
40+
41+
@Benchmark
42+
public BytesRef leadingZerosRejected() {
43+
return ParseIp.leadingZerosRejected(ip, scratch);
44+
}
45+
46+
@Benchmark
47+
public BytesRef leadingZerosAreDecimal() {
48+
return ParseIp.leadingZerosAreDecimal(ip, scratch);
49+
}
50+
51+
@Benchmark
52+
public BytesRef leadingZerosAreOctal() {
53+
return ParseIp.leadingZerosAreOctal(ip, scratch);
54+
}
55+
56+
@Benchmark
57+
public BytesRef original() {
58+
InetAddress inetAddress = InetAddresses.forString(ip.utf8ToString());
59+
return new BytesRef(InetAddressPoint.encode(inetAddress));
60+
}
61+
}

docs/changelog/126338.yaml

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
pr: 126338
2+
summary: Speed up TO_IP
3+
area: ES|QL
4+
type: enhancement
5+
issues: []

x-pack/plugin/esql/src/main/generated/org/elasticsearch/xpack/esql/expression/function/scalar/convert/ParseIpLeadingZerosAreDecimalEvaluator.java

Lines changed: 149 additions & 0 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.
Lines changed: 26 additions & 16 deletions
Some generated files are not rendered by default. Learn more about customizing how changed files appear on GitHub.

0 commit comments

Comments
 (0)