Skip to content

Commit c7ff46b

Browse files
nik9000nik9000
committed
Merge branch 'main' into esql_heap_attack_1mb
2 parents ad38f05 + d763805 commit c7ff46b

File tree

32 files changed

+660
-135
lines changed

32 files changed

+660
-135
lines changed

‎docs/changelog/120573.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
pr: 120573
2+
summary: Optimize `IngestDocument` `FieldPath` allocation
3+
area: Ingest Node
4+
type: enhancement
5+
issues: []

‎docs/changelog/120807.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
pr: 120807
2+
summary: Remove INDEX_REFRESH_BLOCK after index becomes searchable
3+
area: CRUD
4+
type: enhancement
5+
issues: []

‎docs/changelog/120824.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
pr: 120824
2+
summary: Optimize some per-document hot paths in the geoip processor
3+
area: Ingest Node
4+
type: enhancement
5+
issues: []

‎docs/changelog/120997.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
pr: 120997
2+
summary: Allow `SSHA-256` for API key credential hash
3+
area: Authentication
4+
type: enhancement
5+
issues: []

‎docs/reference/indices/shard-stores.asciidoc‎

Lines changed: 2 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -198,10 +198,8 @@ The API returns the following response:
198198
// TESTRESPONSE[s/"attributes": \{[^}]*\}/"attributes": $body.$_path/]
199199
// TESTRESPONSE[s/"roles": \[[^]]*\]/"roles": $body.$_path/]
200200
// TESTRESPONSE[s/"8.10.0"/\$node_version/]
201-
// TESTRESPONSE[s/"min_index_version": 7000099/"min_index_version": $body.$_path/]
202-
// TESTRESPONSE[s/"max_index_version": 8100099/"max_index_version": $body.$_path/]
203-
204-
201+
// TESTRESPONSE[s/"min_index_version": [0-9]+/"min_index_version": $body.$_path/]
202+
// TESTRESPONSE[s/"max_index_version": [0-9]+/"max_index_version": $body.$_path/]
205203

206204
<1> The key is the corresponding shard id for the store information
207205
<2> A list of store information for all copies of the shard

‎docs/reference/ingest/apis/enrich/execute-enrich-policy.asciidoc‎

Lines changed: 2 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -96,8 +96,8 @@ or index documents to an enrich index.
9696
Instead, update your source indices
9797
and <<execute-enrich-policy-api,execute>> the enrich policy again.
9898
This creates a new enrich index from your updated source indices.
99-
The previous enrich index will deleted with a delayed maintenance job.
100-
By default this is done every 15 minutes.
99+
The previous enrich index will be deleted with a delayed maintenance
100+
job that executes by default every 15 minutes.
101101
// end::update-enrich-index[]
102102

103103
By default, this API is synchronous: It returns when a policy has been executed.

‎docs/reference/settings/security-hash-settings.asciidoc‎

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -124,4 +124,68 @@ following:
124124
initial input with SHA512 first.
125125
|=======================
126126

127+
Furthermore, {es} supports authentication via securely-generated high entropy tokens,
128+
for instance <<security-api-create-api-key,API keys>>.
129+
Analogous to passwords, only the tokens' hashes are stored. Since the tokens are guaranteed
130+
to have sufficiently high entropy to resist offline attacks, secure salted hash functions are supported
131+
in addition to the password-hashing algorithms mentioned above.
127132

133+
You can configure the algorithm for API key stored credential hashing
134+
by setting the <<static-cluster-setting,static>>
135+
`xpack.security.authc.api_key.hashing.algorithm` setting to one of the
136+
following
137+
138+
[[secure-token-hashing-algorithms]]
139+
.Secure token hashing algorithms
140+
|=======================
141+
| Algorithm | | | Description
142+
143+
| `ssha256` | | | Uses a salted `sha-256` algorithm. (default)
144+
| `bcrypt` | | | Uses `bcrypt` algorithm with salt generated in 1024 rounds.
145+
| `bcrypt4` | | | Uses `bcrypt` algorithm with salt generated in 16 rounds.
146+
| `bcrypt5` | | | Uses `bcrypt` algorithm with salt generated in 32 rounds.
147+
| `bcrypt6` | | | Uses `bcrypt` algorithm with salt generated in 64 rounds.
148+
| `bcrypt7` | | | Uses `bcrypt` algorithm with salt generated in 128 rounds.
149+
| `bcrypt8` | | | Uses `bcrypt` algorithm with salt generated in 256 rounds.
150+
| `bcrypt9` | | | Uses `bcrypt` algorithm with salt generated in 512 rounds.
151+
| `bcrypt10` | | | Uses `bcrypt` algorithm with salt generated in 1024 rounds.
152+
| `bcrypt11` | | | Uses `bcrypt` algorithm with salt generated in 2048 rounds.
153+
| `bcrypt12` | | | Uses `bcrypt` algorithm with salt generated in 4096 rounds.
154+
| `bcrypt13` | | | Uses `bcrypt` algorithm with salt generated in 8192 rounds.
155+
| `bcrypt14` | | | Uses `bcrypt` algorithm with salt generated in 16384 rounds.
156+
| `pbkdf2` | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
157+
pseudorandom function using 10000 iterations.
158+
| `pbkdf2_1000` | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
159+
pseudorandom function using 1000 iterations.
160+
| `pbkdf2_10000` | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
161+
pseudorandom function using 10000 iterations.
162+
| `pbkdf2_50000` | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
163+
pseudorandom function using 50000 iterations.
164+
| `pbkdf2_100000` | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
165+
pseudorandom function using 100000 iterations.
166+
| `pbkdf2_500000` | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
167+
pseudorandom function using 500000 iterations.
168+
| `pbkdf2_1000000` | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
169+
pseudorandom function using 1000000 iterations.
170+
| `pbkdf2_stretch` | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
171+
pseudorandom function using 10000 iterations, after hashing the
172+
initial input with SHA512 first.
173+
| `pbkdf2_stretch_1000` | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
174+
pseudorandom function using 1000 iterations, after hashing the
175+
initial input with SHA512 first.
176+
| `pbkdf2_stretch_10000` | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
177+
pseudorandom function using 10000 iterations, after hashing the
178+
initial input with SHA512 first.
179+
| `pbkdf2_stretch_50000` | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
180+
pseudorandom function using 50000 iterations, after hashing the
181+
initial input with SHA512 first.
182+
| `pbkdf2_stretch_100000` | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
183+
pseudorandom function using 100000 iterations, after hashing the
184+
initial input with SHA512 first.
185+
| `pbkdf2_stretch_500000` | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
186+
pseudorandom function using 500000 iterations, after hashing the
187+
initial input with SHA512 first.
188+
| `pbkdf2_stretch_1000000`| | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
189+
pseudorandom function using 1000000 iterations, after hashing the
190+
initial input with SHA512 first.
191+
|=======================

‎docs/reference/settings/security-settings.asciidoc‎

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -23,8 +23,8 @@ For more information about creating and updating the {es} keystore, see
2323
==== General security settings
2424
`xpack.security.enabled`::
2525
(<<static-cluster-setting,Static>>)
26-
Defaults to `true`, which enables {es} {security-features} on the node.
27-
This setting must be enabled to use Elasticsearch's authentication,
26+
Defaults to `true`, which enables {es} {security-features} on the node.
27+
This setting must be enabled to use Elasticsearch's authentication,
2828
authorization and audit features. +
2929
+
3030
--
@@ -229,7 +229,7 @@ Defaults to `7d`.
229229

230230
--
231231
NOTE: Large real-time clock inconsistency across cluster nodes can cause problems
232-
with evaluating the API key retention period. That is, if the clock on the node
232+
with evaluating the API key retention period. That is, if the clock on the node
233233
invalidating the API key is significantly different than the one performing the deletion,
234234
the key may be retained for longer or shorter than the configured retention period.
235235

@@ -252,7 +252,7 @@ Sets the timeout of the internal search and delete call.
252252
`xpack.security.authc.api_key.hashing.algorithm`::
253253
(<<static-cluster-setting,Static>>)
254254
Specifies the hashing algorithm that is used for securing API key credentials.
255-
See <<password-hashing-algorithms>>. Defaults to `pbkdf2`.
255+
See <<secure-token-hashing-algorithms>>. Defaults to `ssha256`.
256256

257257
[discrete]
258258
[[security-domain-settings]]

‎docs/reference/troubleshooting/common-issues/disk-usage-exceeded.asciidoc‎

Lines changed: 25 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -57,7 +57,7 @@ GET _cluster/allocation/explain
5757
[[fix-watermark-errors-temporary]]
5858
==== Temporary Relief
5959

60-
To immediately restore write operations, you can temporarily increase the
60+
To immediately restore write operations, you can temporarily increase
6161
<<disk-based-shard-allocation,disk watermarks>> and remove the
6262
<<index-block-settings,write block>>.
6363

@@ -106,19 +106,33 @@ PUT _cluster/settings
106106
[[fix-watermark-errors-resolve]]
107107
==== Resolve
108108

109-
As a long-term solution, we recommend you do one of the following best suited
110-
to your use case:
109+
To resolve watermark errors permanently, perform one of the following actions:
111110

112-
* add nodes to the affected <<data-tiers,data tiers>>
113-
+
114-
TIP: You should enable <<xpack-autoscaling,autoscaling>> for clusters deployed using our {ess}, {ece}, and {eck} platforms.
111+
* Horizontally scale nodes of the affected <<data-tiers,data tiers>>.
115112

116-
* upgrade existing nodes to increase disk space
117-
+
118-
TIP: On {ess}, https://support.elastic.co[Elastic Support] intervention may
119-
become necessary if <<cluster-health,cluster health>> reaches `status:red`.
113+
* Vertically scale existing nodes to increase disk space.
120114

121-
* delete unneeded indices using the <<indices-delete-index,delete index API>>
115+
* Delete indices using the <<indices-delete-index,delete index API>>, either
116+
permanently if the index isn't needed, or temporarily to later
117+
<<snapshots-restore-snapshot,restore>>.
122118

123119
* update related <<index-lifecycle-management,ILM policy>> to push indices
124120
through to later <<data-tiers,data tiers>>
121+
122+
TIP: On {ess} and {ece}, indices may need to be temporarily deleted via
123+
its {cloud}/ec-api-console.html[Elasticsearch API Console] to later
124+
<<snapshots-restore-snapshot,snapshot restore>> in order to resolve
125+
<<cluster-health,cluster health>> `status:red` which will block
126+
{cloud}/ec-activity-page.html[attempted changes]. If you experience issues
127+
with this resolution flow on {ess}, kindly reach out to
128+
https://support.elastic.co[Elastic Support] for assistance.
129+
130+
== Prevent watermark errors
131+
132+
To avoid watermark errors in future, , perform one of the following actions:
133+
134+
* If you're using {ess}, {ece}, or {eck}: Enable <<xpack-autoscaling,autoscaling>>.
135+
136+
* Set up {kibana-ref}/kibana-alerts.html[stack monitoring alerts] on top of
137+
<<monitor-elasticsearch-cluster,{es} monitoring>> to be notified before
138+
the flood-stage watermark is reached.

‎modules/ingest-geoip/src/internalClusterTest/java/org/elasticsearch/ingest/geoip/GeoIpDownloaderIT.java‎

Lines changed: 10 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -70,6 +70,7 @@
7070
import static org.elasticsearch.ingest.geoip.GeoIpTestUtils.copyDefaultDatabases;
7171
import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertAcked;
7272
import static org.elasticsearch.test.hamcrest.ElasticsearchAssertions.assertResponse;
73+
import static org.hamcrest.Matchers.allOf;
7374
import static org.hamcrest.Matchers.anEmptyMap;
7475
import static org.hamcrest.Matchers.contains;
7576
import static org.hamcrest.Matchers.containsInAnyOrder;
@@ -172,10 +173,15 @@ public void testInvalidTimestamp() throws Exception {
172173
for (Path geoIpTmpDir : geoIpTmpDirs) {
173174
try (Stream<Path> files = Files.list(geoIpTmpDir)) {
174175
Set<String> names = files.map(f -> f.getFileName().toString()).collect(Collectors.toSet());
175-
assertThat(names, not(hasItem("GeoLite2-ASN.mmdb")));
176-
assertThat(names, not(hasItem("GeoLite2-City.mmdb")));
177-
assertThat(names, not(hasItem("GeoLite2-Country.mmdb")));
178-
assertThat(names, not(hasItem("MyCustomGeoLite2-City.mmdb")));
176+
assertThat(
177+
names,
178+
allOf(
179+
not(hasItem("GeoLite2-ASN.mmdb")),
180+
not(hasItem("GeoLite2-City.mmdb")),
181+
not(hasItem("GeoLite2-Country.mmdb")),
182+
not(hasItem("MyCustomGeoLite2-City.mmdb"))
183+
)
184+
);
179185
}
180186
}
181187
});

0 commit comments

Comments
 (0)