Skip to content

Commit 4fb2afb

Browse files
mayya-sharipovamayya-sharipova
committed
Merge remote-tracking branch 'upstream/main' into lucene_snapshot
2 parents 4a31fb8 + 8e3ab37 commit 4fb2afb

File tree

13 files changed

+1076
-105
lines changed

13 files changed

+1076
-105
lines changed

‎docs/changelog/131723.yaml‎

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
pr: 131723
2+
summary: Tests for FORK's evaluation of field names used in `field_caps` resolve calls
3+
area: Search
4+
type: bug
5+
issues: []

‎docs/release-notes/known-issues.md‎

Lines changed: 8 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -5,13 +5,19 @@ mapped_pages:
55
---
66

77
# Elasticsearch known issues [elasticsearch-known-issues]
8+
89
Known issues are significant defects or limitations that may impact your implementation. These issues are actively being worked on and will be addressed in a future release. Review the Elasticsearch known issues to help you make informed decisions, such as upgrading to a new version.
910

1011
## 9.1.0 [elasticsearch-9.1.0-known-issues]
11-
* An error in the configuration of vector indices with type `bbq_hnsw` may lead to significant search performance degradation on 9.1.0. To mitigate this, set the `-Dvector.rescoring.directio=false` JVM option on all search nodes, then restart the nodes.
12-
This option can be removed in 9.1.1.
12+
13+
* The `-Dvector.rescoring.directio` JVM option is enabled (set to `true`) by default. When used with `bbq_hnsw` type vector indices, this can cause significant search performance degradation; particularly when enough memory is available to hold all vector data. In some cases, kNN search latency can increase by as much as 10x. To mitigate this, set the JVM option `-Dvector.rescoring.directio=false` on all search nodes and restart them. This option can be removed in 9.1.1.
14+
15+
**How do I know if my index vector type is `bbq_hnsw`?**
16+
17+
* Prior to 9.1, the vector type had to be explicitly set to `bbq_hnsw`. Starting with 9.1, `bbq_hnsw` is the default vector type for dense vectors with more than 384 dimensions in new indices, unless another type is specified.
1318

1419
## 9.0.3 [elasticsearch-9.0.3-known-issues]
20+
1521
* A bug in the merge scheduler in Elasticsearch 9.0.3 may prevent shards from closing when there isn’t enough disk space to complete a merge. As a result, operations such as closing or relocating an index may hang until sufficient disk space becomes available.
1622
To mitigate this issue, the disk space checker is disabled by default in 9.0.3 by setting `indices.merge.disk.check_interval` to `0` seconds. Manually enabling this setting is not recommended.
1723

‎muted-tests.yml‎

Lines changed: 0 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -461,9 +461,6 @@ tests:
461461
- class: org.elasticsearch.xpack.esql.ccq.MultiClustersIT
462462
method: testLookupJoinAliasesSkipOld
463463
issue: https://github.com/elastic/elasticsearch/issues/131697
464-
- class: org.elasticsearch.indices.cluster.RemoteSearchForceConnectTimeoutIT
465-
method: testTimeoutSetting
466-
issue: https://github.com/elastic/elasticsearch/issues/131656
467464
- class: org.elasticsearch.packaging.test.DockerTests
468465
method: test151MachineDependentHeapWithSizeOverride
469466
issue: https://github.com/elastic/elasticsearch/issues/123437
@@ -479,9 +476,6 @@ tests:
479476
- class: org.elasticsearch.index.engine.MergeWithLowDiskSpaceIT
480477
method: testRelocationWhileForceMerging
481478
issue: https://github.com/elastic/elasticsearch/issues/131789
482-
- class: org.elasticsearch.indices.cluster.FieldCapsForceConnectTimeoutIT
483-
method: testTimeoutSetting
484-
issue: https://github.com/elastic/elasticsearch/issues/132179
485479
- class: org.elasticsearch.test.rest.yaml.MDPYamlTestSuiteIT
486480
method: test {yaml=mdp/10_basic/Index using shared data path}
487481
issue: https://github.com/elastic/elasticsearch/issues/132223

‎server/src/internalClusterTest/java/org/elasticsearch/indices/cluster/FieldCapsForceConnectTimeoutIT.java‎

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -73,21 +73,23 @@ protected Map<String, Boolean> skipUnavailableForRemoteClusters() {
7373

7474
public void testTimeoutSetting() {
7575
var latch = new CountDownLatch(1);
76-
for (String nodeName : cluster(LOCAL_CLUSTER).getNodeNames()) {
77-
MockTransportService mts = (MockTransportService) cluster(LOCAL_CLUSTER).getInstance(TransportService.class, nodeName);
78-
79-
mts.addConnectBehavior(
80-
cluster(LINKED_CLUSTER_1).getInstance(TransportService.class, (String) null),
81-
((transport, discoveryNode, profile, listener) -> {
76+
for (String localNodeName : cluster(LOCAL_CLUSTER).getNodeNames()) {
77+
MockTransportService localMts = (MockTransportService) cluster(LOCAL_CLUSTER).getInstance(
78+
TransportService.class,
79+
localNodeName
80+
);
81+
for (String remoteNodeName : cluster(LINKED_CLUSTER_1).getNodeNames()) {
82+
TransportService remoteTs = cluster(LINKED_CLUSTER_1).getInstance(TransportService.class, remoteNodeName);
83+
localMts.addConnectBehavior(remoteTs, ((transport, discoveryNode, profile, listener) -> {
8284
try {
8385
latch.await();
8486
} catch (InterruptedException e) {
8587
throw new AssertionError(e);
8688
}
8789

8890
transport.openConnection(discoveryNode, profile, listener);
89-
})
90-
);
91+
}));
92+
}
9193
}
9294

9395
// Add some dummy data to prove we are communicating fine with the remote.

‎server/src/internalClusterTest/java/org/elasticsearch/indices/cluster/RemoteSearchForceConnectTimeoutIT.java‎

Lines changed: 10 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -70,21 +70,23 @@ protected Map<String, Boolean> skipUnavailableForRemoteClusters() {
7070

7171
public void testTimeoutSetting() {
7272
var latch = new CountDownLatch(1);
73-
for (String nodeName : cluster(LOCAL_CLUSTER).getNodeNames()) {
74-
MockTransportService mts = (MockTransportService) cluster(LOCAL_CLUSTER).getInstance(TransportService.class, nodeName);
75-
76-
mts.addConnectBehavior(
77-
cluster(REMOTE_CLUSTER_1).getInstance(TransportService.class, (String) null),
78-
((transport, discoveryNode, profile, listener) -> {
73+
for (String localNodeName : cluster(LOCAL_CLUSTER).getNodeNames()) {
74+
MockTransportService localMts = (MockTransportService) cluster(LOCAL_CLUSTER).getInstance(
75+
TransportService.class,
76+
localNodeName
77+
);
78+
for (String remoteNodeName : cluster(REMOTE_CLUSTER_1).getNodeNames()) {
79+
TransportService remoteTs = cluster(REMOTE_CLUSTER_1).getInstance(TransportService.class, remoteNodeName);
80+
localMts.addConnectBehavior(remoteTs, ((transport, discoveryNode, profile, listener) -> {
7981
try {
8082
latch.await();
8183
} catch (InterruptedException e) {
8284
throw new AssertionError(e);
8385
}
8486

8587
transport.openConnection(discoveryNode, profile, listener);
86-
})
87-
);
88+
}));
89+
}
8890
}
8991

9092
// Add some dummy data to prove we are communicating fine with the remote.

‎server/src/main/java/org/elasticsearch/action/admin/cluster/remote/TransportRemoteInfoAction.java‎

Lines changed: 1 addition & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,6 @@
1313
import org.elasticsearch.action.ActionType;
1414
import org.elasticsearch.action.support.ActionFilters;
1515
import org.elasticsearch.action.support.HandledTransportAction;
16-
import org.elasticsearch.cluster.node.DiscoveryNodeRole;
1716
import org.elasticsearch.common.util.concurrent.EsExecutors;
1817
import org.elasticsearch.injection.guice.Inject;
1918
import org.elasticsearch.tasks.Task;
@@ -35,15 +34,7 @@ public TransportRemoteInfoAction(TransportService transportService, ActionFilter
3534

3635
@Override
3736
protected void doExecute(Task task, RemoteInfoRequest remoteInfoRequest, ActionListener<RemoteInfoResponse> listener) {
38-
if (remoteClusterService.isEnabled() == false) {
39-
throw new IllegalArgumentException(
40-
"node ["
41-
+ remoteClusterService.getLocalNode().getName()
42-
+ "] does not have the ["
43-
+ DiscoveryNodeRole.REMOTE_CLUSTER_CLIENT_ROLE.roleName()
44-
+ "] role"
45-
);
46-
}
37+
remoteClusterService.ensureClientIsEnabled();
4738
listener.onResponse(new RemoteInfoResponse(remoteClusterService.getRemoteConnectionInfos().collect(toList())));
4839
}
4940
}

‎server/src/main/java/org/elasticsearch/transport/RemoteClusterAware.java‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,10 @@ protected RemoteClusterAware(Settings settings) {
4949
this.isRemoteClusterClientEnabled = DiscoveryNode.isRemoteClusterClient(settings);
5050
}
5151

52+
protected String getNodeName() {
53+
return nodeName;
54+
}
55+
5256
/**
5357
* Returns remote clusters that are enabled in these settings
5458
*/

‎server/src/main/java/org/elasticsearch/transport/RemoteClusterService.java‎

Lines changed: 37 additions & 25 deletions
Original file line numberDiff line numberDiff line change
@@ -147,13 +147,11 @@ public final class RemoteClusterService extends RemoteClusterAware
147147

148148
public static final String REMOTE_CLUSTER_HANDSHAKE_ACTION_NAME = "cluster:internal/remote_cluster/handshake";
149149

150-
private final boolean enabled;
150+
private final boolean isRemoteClusterClient;
151+
private final boolean isSearchNode;
152+
private final boolean isStateless;
151153
private final boolean remoteClusterServerEnabled;
152154

153-
public boolean isEnabled() {
154-
return enabled;
155-
}
156-
157155
public boolean isRemoteClusterServerEnabled() {
158156
return remoteClusterServerEnabled;
159157
}
@@ -166,7 +164,9 @@ public boolean isRemoteClusterServerEnabled() {
166164
@FixForMultiProject(description = "Inject the ProjectResolver instance.")
167165
RemoteClusterService(Settings settings, TransportService transportService) {
168166
super(settings);
169-
this.enabled = DiscoveryNode.isRemoteClusterClient(settings);
167+
this.isRemoteClusterClient = DiscoveryNode.isRemoteClusterClient(settings);
168+
this.isSearchNode = DiscoveryNode.hasRole(settings, DiscoveryNodeRole.SEARCH_ROLE);
169+
this.isStateless = DiscoveryNode.isStateless(settings);
170170
this.remoteClusterServerEnabled = REMOTE_CLUSTER_SERVER_ENABLED.get(settings);
171171
this.transportService = transportService;
172172
this.projectResolver = DefaultProjectResolver.INSTANCE;
@@ -179,10 +179,6 @@ public boolean isRemoteClusterServerEnabled() {
179179
}
180180
}
181181

182-
public DiscoveryNode getLocalNode() {
183-
return transportService.getLocalNode();
184-
}
185-
186182
/**
187183
* Group indices by cluster alias mapped to OriginalIndices for that cluster.
188184
* @param remoteClusterNames Set of configured remote cluster names.
@@ -335,11 +331,7 @@ public void maybeEnsureConnectedAndGetConnection(
335331
}
336332

337333
public RemoteClusterConnection getRemoteClusterConnection(String cluster) {
338-
if (enabled == false) {
339-
throw new IllegalArgumentException(
340-
"this node does not have the " + DiscoveryNodeRole.REMOTE_CLUSTER_CLIENT_ROLE.roleName() + " role"
341-
);
342-
}
334+
ensureClientIsEnabled();
343335
@FixForMultiProject(description = "Verify all callers will have the proper context set for resolving the origin project ID.")
344336
RemoteClusterConnection connection = getConnectionsMapForCurrentProject().get(cluster);
345337
if (connection == null) {
@@ -595,11 +587,7 @@ public RemoteClusterServerInfo info() {
595587
* function on success.
596588
*/
597589
public void collectNodes(Set<String> clusters, ActionListener<BiFunction<String, String, DiscoveryNode>> listener) {
598-
if (enabled == false) {
599-
throw new IllegalArgumentException(
600-
"this node does not have the " + DiscoveryNodeRole.REMOTE_CLUSTER_CLIENT_ROLE.roleName() + " role"
601-
);
602-
}
590+
ensureClientIsEnabled();
603591
@FixForMultiProject(description = "Analyze usages and determine if the project ID must be provided.")
604592
final var projectConnectionsMap = getConnectionsMapForCurrentProject();
605593
final var connectionsMap = new HashMap<String, RemoteClusterConnection>();
@@ -662,11 +650,7 @@ public RemoteClusterClient getRemoteClusterClient(
662650
Executor responseExecutor,
663651
DisconnectedStrategy disconnectedStrategy
664652
) {
665-
if (transportService.getRemoteClusterService().isEnabled() == false) {
666-
throw new IllegalArgumentException(
667-
"this node does not have the " + DiscoveryNodeRole.REMOTE_CLUSTER_CLIENT_ROLE.roleName() + " role"
668-
);
669-
}
653+
ensureClientIsEnabled();
670654
if (transportService.getRemoteClusterService().getRegisteredRemoteClusterNames().contains(clusterAlias) == false) {
671655
throw new NoSuchRemoteClusterException(clusterAlias);
672656
}
@@ -677,6 +661,34 @@ public RemoteClusterClient getRemoteClusterClient(
677661
});
678662
}
679663

664+
/**
665+
* Verifies this node is configured to support linked project client operations.
666+
* @throws IllegalArgumentException If this node is not configured to support client operations.
667+
*/
668+
public void ensureClientIsEnabled() {
669+
if (isRemoteClusterClient) {
670+
return;
671+
}
672+
if (isStateless == false) {
673+
throw new IllegalArgumentException(
674+
"node [" + getNodeName() + "] does not have the [" + DiscoveryNodeRole.REMOTE_CLUSTER_CLIENT_ROLE.roleName() + "] role"
675+
);
676+
}
677+
// For stateless the remote cluster client is enabled by default for search nodes,
678+
// REMOTE_CLUSTER_CLIENT_ROLE is not explicitly required.
679+
if (isSearchNode == false) {
680+
throw new IllegalArgumentException(
681+
"node ["
682+
+ getNodeName()
683+
+ "] must have the ["
684+
+ DiscoveryNodeRole.REMOTE_CLUSTER_CLIENT_ROLE.roleName()
685+
+ "] role or the ["
686+
+ DiscoveryNodeRole.SEARCH_ROLE.roleName()
687+
+ "] role in stateless environments to use linked project client features"
688+
);
689+
}
690+
}
691+
680692
static void registerRemoteClusterHandshakeRequestHandler(TransportService transportService) {
681693
transportService.registerRequestHandler(
682694
REMOTE_CLUSTER_HANDSHAKE_ACTION_NAME,

‎server/src/test/java/org/elasticsearch/transport/RemoteClusterClientTests.java‎

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@
2424
import org.elasticsearch.common.settings.Settings;
2525
import org.elasticsearch.common.util.concurrent.EsExecutors;
2626
import org.elasticsearch.core.TimeValue;
27+
import org.elasticsearch.node.Node;
2728
import org.elasticsearch.test.ESTestCase;
2829
import org.elasticsearch.test.junit.annotations.TestLogging;
2930
import org.elasticsearch.test.transport.MockTransportService;
@@ -215,7 +216,10 @@ public void testEnsureWeReconnect() throws Exception {
215216
}
216217

217218
public void testRemoteClusterServiceNotEnabled() {
218-
final Settings settings = removeRoles(Set.of(DiscoveryNodeRole.REMOTE_CLUSTER_CLIENT_ROLE));
219+
final Settings settings = Settings.builder()
220+
.put(removeRoles(Set.of(DiscoveryNodeRole.REMOTE_CLUSTER_CLIENT_ROLE)))
221+
.put(Node.NODE_NAME_SETTING.getKey(), "node-1")
222+
.build();
219223
try (
220224
MockTransportService service = MockTransportService.createNewService(
221225
settings,
@@ -236,7 +240,7 @@ public void testRemoteClusterServiceNotEnabled() {
236240
randomFrom(RemoteClusterService.DisconnectedStrategy.values())
237241
)
238242
);
239-
assertThat(e.getMessage(), equalTo("this node does not have the remote_cluster_client role"));
243+
assertThat(e.getMessage(), equalTo("node [node-1] does not have the [remote_cluster_client] role"));
240244
}
241245
}
242246

0 commit comments

Comments
 (0)