From 1b0acbc91bff13cd73a9b2a55042655be8d9af00 Mon Sep 17 00:00:00 2001 From: David Turner Date: Fri, 27 Sep 2024 09:15:09 +0100 Subject: [PATCH 1/2] Misc `ReferenceDocs` improvements - Migrate link defs to YAML, mainly to reduce noise when adding links. - Forbid ephemeral `_auto_gen.html` and `page.html#_auto_gen` links. - Remove dangling/unused `BOOTSTRAP_CHECK_G1GC` link. --- .../setup/bootstrap-checks-xes.asciidoc | 4 ++ .../reference/setup/bootstrap-checks.asciidoc | 12 +++++ .../elasticsearch/common/ReferenceDocs.java | 18 +++++-- .../common/reference-docs-links.json | 47 ----------------- .../common/reference-docs-links.yaml | 45 ++++++++++++++++ .../common/ReferenceDocsTests.java | 51 ++++++++++++++++--- 6 files changed, 118 insertions(+), 59 deletions(-) delete mode 100644 server/src/main/resources/org/elasticsearch/common/reference-docs-links.json create mode 100644 server/src/main/resources/org/elasticsearch/common/reference-docs-links.yaml diff --git a/docs/reference/setup/bootstrap-checks-xes.asciidoc b/docs/reference/setup/bootstrap-checks-xes.asciidoc index bc187e668d0c0..926afe6463f46 100644 --- a/docs/reference/setup/bootstrap-checks-xes.asciidoc +++ b/docs/reference/setup/bootstrap-checks-xes.asciidoc @@ -6,6 +6,7 @@ In addition to the <>, there are checks that are specific to {xpack} features. [discrete] +[[bootstrap-checks-xpack-encrypt-sensitive-data]] === Encrypt sensitive data check //See EncryptSensitiveDAtaBootstrapCheck.java @@ -18,6 +19,7 @@ on each node in the cluster. For more information, see <>. [discrete] +[[bootstrap-checks-xpack-pki-realm]] === PKI realm check //See PkiRealmBootstrapCheckTests.java @@ -31,6 +33,7 @@ To pass this bootstrap check, if a PKI realm is enabled, you must configure TLS and enable client authentication on at least one network communication layer. [discrete] +[[bootstrap-checks-xpack-role-mappings]] === Role mappings check If you authenticate users with realms other than `native` or `file` realms, you @@ -65,6 +68,7 @@ To pass this bootstrap check, you must [discrete] +[[bootstrap-checks-xpack-token-ssl]] === Token SSL check //See TokenSSLBootstrapCheckTests.java diff --git a/docs/reference/setup/bootstrap-checks.asciidoc b/docs/reference/setup/bootstrap-checks.asciidoc index 64977ae4e4611..7793e3e3c3359 100644 --- a/docs/reference/setup/bootstrap-checks.asciidoc +++ b/docs/reference/setup/bootstrap-checks.asciidoc @@ -60,6 +60,7 @@ in the <>. We strongly encourage you to do this if you are in this specific situation. This system property can be used to force execution of the bootstrap checks independent of the node configuration. +[[bootstrap-checks-heap-size]] === Heap size check By default, {es} automatically sizes JVM heap based on a node's @@ -71,6 +72,7 @@ size on startup. If the initial heap size is not equal to the maximum heap size, some JVM heap may not be locked after a resize. To avoid these issues, start the JVM with an initial heap size equal to the maximum heap size. +[[bootstrap-checks-file-descriptor]] === File descriptor check File descriptors are a Unix construct for tracking open "files". In Unix @@ -83,6 +85,7 @@ bootstrap check is enforced on OS X and Linux. To pass the file descriptor check, you might have to configure <>. +[[bootstrap-checks-memory-lock]] === Memory lock check When the JVM does a major garbage collection it touches every page of @@ -116,6 +119,7 @@ least 4096 threads. This can be done via `/etc/security/limits.conf` using the `nproc` setting (note that you might have to increase the limits for the `root` user too). +[[bootstrap-checks-max-file-size]] === Max file size check The segment files that are the components of individual shards and the translog @@ -144,6 +148,7 @@ address space. This can be done via adding ` - as unlimited` to `/etc/security/limits.conf`. This may require you to increase the limits for the `root` user too. +[[bootstrap-checks-max-map-count]] === Maximum map count check Continuing from the previous <>, to @@ -158,6 +163,7 @@ Alternatively, the maximum map count check is only needed if you are using indices. If you <> the use of `mmap` then this bootstrap check will not be enforced. +[[bootstrap-checks-client-jvm]] === Client JVM check There are two different JVMs provided by OpenJDK-derived JVMs: the @@ -171,6 +177,7 @@ JVM check, you must start Elasticsearch with the server VM. On modern systems and operating systems, the server VM is the default. +[[bootstrap-checks-serial-collector]] === Use serial collector check There are various garbage collectors for the OpenJDK-derived JVMs @@ -187,6 +194,7 @@ configuration that ships with Elasticsearch configures Elasticsearch to use the G1GC garbage collector with JDK14 and later versions. For earlier JDK versions, the configuration defaults to the CMS collector. +[[bootstrap-checks-syscall-filter]] === System call filter check Elasticsearch installs system call filters of various flavors depending on the operating system (e.g., seccomp on Linux). These system call @@ -198,6 +206,7 @@ installed. To pass the system call filter check you must fix any configuration errors on your system that prevented system call filters from installing (check your logs). +[[bootstrap-checks-onerror]] === OnError and OnOutOfMemoryError checks The JVM options `OnError` and `OnOutOfMemoryError` enable executing @@ -214,6 +223,7 @@ use the JVM flag `ExitOnOutOfMemoryError`. While this does not have the full capabilities of `OnError` nor `OnOutOfMemoryError`, arbitrary forking will not be supported with seccomp enabled. +[[bootstrap-checks-early-access]] === Early-access check The OpenJDK project provides early-access snapshots of upcoming releases. These @@ -221,12 +231,14 @@ releases are not suitable for production. The early-access check detects these early-access snapshots. To pass this check, you must start Elasticsearch on a release build of the JVM. +[[bootstrap-checks-all-permission]] === All permission check The all permission check ensures that the security policy used during bootstrap does not grant the `java.security.AllPermission` to Elasticsearch. Running with the all permission granted is equivalent to disabling the security manager. +[[bootstrap-checks-discovery-configuration]] === Discovery configuration check By default, when Elasticsearch first starts up it will try and discover other diff --git a/server/src/main/java/org/elasticsearch/common/ReferenceDocs.java b/server/src/main/java/org/elasticsearch/common/ReferenceDocs.java index 37f4a5ce52295..d7fd1a91ab8fd 100644 --- a/server/src/main/java/org/elasticsearch/common/ReferenceDocs.java +++ b/server/src/main/java/org/elasticsearch/common/ReferenceDocs.java @@ -27,11 +27,11 @@ /** * Encapsulates links to pages in the reference docs, so that for example we can include URLs in logs and API outputs. Each instance's * {@link #toString()} yields (a string representation of) a URL for the relevant docs. Links are defined in the resource file - * {@code reference-docs-links.json} which must include definitions for exactly the set of values of this enum. + * {@code reference-docs-links.yaml} which must include definitions for exactly the set of values of this enum. */ public enum ReferenceDocs { /* - * Note that the docs subsystem parses {@code reference-docs-links.json} with regexes, not a JSON parser, so the whitespace in the file + * Note that the docs subsystem parses {@code reference-docs-links.yaml} with regexes, not a YAML parser, so the whitespace in the file * is important too. See {@code sub check_elasticsearch_links} in {@code https://github.com/elastic/docs/blob/master/build_docs.pl} for * more details. * @@ -61,7 +61,6 @@ public enum ReferenceDocs { BOOTSTRAP_CHECK_SYSTEM_CALL_FILTER, BOOTSTRAP_CHECK_ONERROR_AND_ONOUTOFMEMORYERROR, BOOTSTRAP_CHECK_EARLY_ACCESS, - BOOTSTRAP_CHECK_G1GC, BOOTSTRAP_CHECK_ALL_PERMISSION, BOOTSTRAP_CHECK_DISCOVERY_CONFIGURATION, BOOTSTRAP_CHECKS, @@ -90,7 +89,7 @@ public enum ReferenceDocs { private static final Map linksBySymbol; static { - try (var resourceStream = readFromJarResourceUrl(ReferenceDocs.class.getResource("reference-docs-links.json"))) { + try (var resourceStream = readFromJarResourceUrl(ReferenceDocs.class.getResource("reference-docs-links.yaml"))) { linksBySymbol = Map.copyOf(readLinksBySymbol(resourceStream)); } catch (Exception e) { assert false : e; @@ -103,7 +102,7 @@ public enum ReferenceDocs { static final String VERSION_COMPONENT = getVersionComponent(Build.current().version(), Build.current().isSnapshot()); static Map readLinksBySymbol(InputStream inputStream) throws Exception { - try (var parser = XContentFactory.xContent(XContentType.JSON).createParser(XContentParserConfiguration.EMPTY, inputStream)) { + try (var parser = XContentFactory.xContent(XContentType.YAML).createParser(XContentParserConfiguration.EMPTY, inputStream)) { final var result = parser.map(LinkedHashMap::new, XContentParser::text); final var iterator = result.keySet().iterator(); for (int i = 0; i < values().length; i++) { @@ -119,6 +118,15 @@ static Map readLinksBySymbol(InputStream inputStream) throws Exc if (iterator.hasNext()) { throw new IllegalStateException("found unexpected extra value: " + iterator.next()); } + + // We must only link to anchors with fixed IDs (defined by [[fragment-name]] in the docs) because auto-generated fragment IDs + // depend on the heading text and are too easy to break inadvertently. Auto-generated fragment IDs begin with an underscore. + for (final var entry : result.entrySet()) { + if (entry.getValue().startsWith("_") || entry.getValue().contains("#_")) { + throw new IllegalStateException("found auto-generated fragment ID at " + entry.getKey()); + } + } + return result; } } diff --git a/server/src/main/resources/org/elasticsearch/common/reference-docs-links.json b/server/src/main/resources/org/elasticsearch/common/reference-docs-links.json deleted file mode 100644 index 4da6de7f7b561..0000000000000 --- a/server/src/main/resources/org/elasticsearch/common/reference-docs-links.json +++ /dev/null @@ -1,47 +0,0 @@ -{ - "INITIAL_MASTER_NODES": "important-settings.html#initial_master_nodes", - "DISCOVERY_TROUBLESHOOTING": "discovery-troubleshooting.html", - "UNSTABLE_CLUSTER_TROUBLESHOOTING": "troubleshooting-unstable-cluster.html", - "LAGGING_NODE_TROUBLESHOOTING": "troubleshooting-unstable-cluster.html#troubleshooting-unstable-cluster-lagging", - "SHARD_LOCK_TROUBLESHOOTING": "troubleshooting-unstable-cluster.html#troubleshooting-unstable-cluster-shardlockobtainfailedexception", - "NETWORK_DISCONNECT_TROUBLESHOOTING": "troubleshooting-unstable-cluster.html#troubleshooting-unstable-cluster-network", - "CONCURRENT_REPOSITORY_WRITERS": "diagnosing-corrupted-repositories.html", - "ARCHIVE_INDICES": "archive-indices.html", - "HTTP_TRACER": "modules-network.html#http-rest-request-tracer", - "LOGGING": "logging.html", - "BOOTSTRAP_CHECK_HEAP_SIZE": "_heap_size_check.html", - "BOOTSTRAP_CHECK_FILE_DESCRIPTOR": "_file_descriptor_check.html", - "BOOTSTRAP_CHECK_MEMORY_LOCK": "_memory_lock_check.html", - "BOOTSTRAP_CHECK_MAX_NUMBER_THREADS": "max-number-threads-check.html", - "BOOTSTRAP_CHECK_MAX_FILE_SIZE": "_max_file_size_check.html", - "BOOTSTRAP_CHECK_MAX_SIZE_VIRTUAL_MEMORY": "max-size-virtual-memory-check.html", - "BOOTSTRAP_CHECK_MAXIMUM_MAP_COUNT": "_maximum_map_count_check.html", - "BOOTSTRAP_CHECK_CLIENT_JVM": "_client_jvm_check.html", - "BOOTSTRAP_CHECK_USE_SERIAL_COLLECTOR": "_use_serial_collector_check.html", - "BOOTSTRAP_CHECK_SYSTEM_CALL_FILTER": "_system_call_filter_check.html", - "BOOTSTRAP_CHECK_ONERROR_AND_ONOUTOFMEMORYERROR": "_onerror_and_onoutofmemoryerror_checks.html", - "BOOTSTRAP_CHECK_EARLY_ACCESS": "_early_access_check.html", - "BOOTSTRAP_CHECK_G1GC": "_g1gc_check.html", - "BOOTSTRAP_CHECK_ALL_PERMISSION": "_all_permission_check.html", - "BOOTSTRAP_CHECK_DISCOVERY_CONFIGURATION": "_discovery_configuration_check.html", - "BOOTSTRAP_CHECKS": "bootstrap-checks.html", - "BOOTSTRAP_CHECK_ENCRYPT_SENSITIVE_DATA": "bootstrap-checks-xpack.html#_encrypt_sensitive_data_check", - "BOOTSTRAP_CHECK_PKI_REALM": "bootstrap-checks-xpack.html#_pki_realm_check", - "BOOTSTRAP_CHECK_ROLE_MAPPINGS": "bootstrap-checks-xpack.html#_role_mappings_check", - "BOOTSTRAP_CHECK_TLS": "bootstrap-checks-xpack.html#bootstrap-checks-tls", - "BOOTSTRAP_CHECK_TOKEN_SSL": "bootstrap-checks-xpack.html#_token_ssl_check", - "BOOTSTRAP_CHECK_SECURITY_MINIMAL_SETUP": "security-minimal-setup.html", - "CONTACT_SUPPORT": "troubleshooting.html#troubleshooting-contact-support", - "UNASSIGNED_SHARDS": "red-yellow-cluster-status.html", - "EXECUTABLE_JNA_TMPDIR": "executable-jna-tmpdir.html", - "NETWORK_THREADING_MODEL": "modules-network.html#modules-network-threading-model", - "ALLOCATION_EXPLAIN_API": "cluster-allocation-explain.html", - "NETWORK_BINDING_AND_PUBLISHING": "modules-network.html#modules-network-binding-publishing", - "SNAPSHOT_REPOSITORY_ANALYSIS": "repo-analysis-api.html", - "S3_COMPATIBLE_REPOSITORIES": "repository-s3.html#repository-s3-compatible-services", - "LUCENE_MAX_DOCS_LIMIT": "size-your-shards.html#troubleshooting-max-docs-limit", - "MAX_SHARDS_PER_NODE": "size-your-shards.html#troubleshooting-max-shards-open", - "FLOOD_STAGE_WATERMARK": "fix-watermark-errors.html", - "X_OPAQUE_ID": "api-conventions.html#x-opaque-id", - "FORMING_SINGLE_NODE_CLUSTERS": "modules-discovery-bootstrap-cluster.html#modules-discovery-bootstrap-cluster-joining" -} diff --git a/server/src/main/resources/org/elasticsearch/common/reference-docs-links.yaml b/server/src/main/resources/org/elasticsearch/common/reference-docs-links.yaml new file mode 100644 index 0000000000000..3acd1cf1fd461 --- /dev/null +++ b/server/src/main/resources/org/elasticsearch/common/reference-docs-links.yaml @@ -0,0 +1,45 @@ +--- +INITIAL_MASTER_NODES: "important-settings.html#initial_master_nodes" +DISCOVERY_TROUBLESHOOTING: "discovery-troubleshooting.html" +UNSTABLE_CLUSTER_TROUBLESHOOTING: "troubleshooting-unstable-cluster.html" +LAGGING_NODE_TROUBLESHOOTING: "troubleshooting-unstable-cluster.html#troubleshooting-unstable-cluster-lagging" +SHARD_LOCK_TROUBLESHOOTING: "troubleshooting-unstable-cluster.html#troubleshooting-unstable-cluster-shardlockobtainfailedexception" +NETWORK_DISCONNECT_TROUBLESHOOTING: "troubleshooting-unstable-cluster.html#troubleshooting-unstable-cluster-network" +CONCURRENT_REPOSITORY_WRITERS: "diagnosing-corrupted-repositories.html" +ARCHIVE_INDICES: "archive-indices.html" +HTTP_TRACER: "modules-network.html#http-rest-request-tracer" +LOGGING: "logging.html" +BOOTSTRAP_CHECK_HEAP_SIZE: "bootstrap-checks-heap-size.html" +BOOTSTRAP_CHECK_FILE_DESCRIPTOR: "bootstrap-checks-file-descriptor.html" +BOOTSTRAP_CHECK_MEMORY_LOCK: "bootstrap-checks-memory-lock.html" +BOOTSTRAP_CHECK_MAX_NUMBER_THREADS: "max-number-threads-check.html" +BOOTSTRAP_CHECK_MAX_FILE_SIZE: "bootstrap-checks-max-file-size.html" +BOOTSTRAP_CHECK_MAX_SIZE_VIRTUAL_MEMORY: "max-size-virtual-memory-check.html" +BOOTSTRAP_CHECK_MAXIMUM_MAP_COUNT: "bootstrap-checks-max-map-count.html" +BOOTSTRAP_CHECK_CLIENT_JVM: "bootstrap-checks-client-jvm.html" +BOOTSTRAP_CHECK_USE_SERIAL_COLLECTOR: "bootstrap-checks-serial-collector.html" +BOOTSTRAP_CHECK_SYSTEM_CALL_FILTER: "bootstrap-checks-syscall-filter.html" +BOOTSTRAP_CHECK_ONERROR_AND_ONOUTOFMEMORYERROR: "bootstrap-checks-onerror.html" +BOOTSTRAP_CHECK_EARLY_ACCESS: "bootstrap-checks-early-access.html" +BOOTSTRAP_CHECK_ALL_PERMISSION: "bootstrap-checks-all-permission.html" +BOOTSTRAP_CHECK_DISCOVERY_CONFIGURATION: "bootstrap-checks-discovery-configuration.html" +BOOTSTRAP_CHECKS: "bootstrap-checks.html" +BOOTSTRAP_CHECK_ENCRYPT_SENSITIVE_DATA: "bootstrap-checks-xpack.html#bootstrap-checks-xpack-encrypt-sensitive-data" +BOOTSTRAP_CHECK_PKI_REALM: "bootstrap-checks-xpack.html#bootstrap-checks-xpack-pki-realm" +BOOTSTRAP_CHECK_ROLE_MAPPINGS: "bootstrap-checks-xpack.html#bootstrap-checks-xpack-role-mappings" +BOOTSTRAP_CHECK_TLS: "bootstrap-checks-xpack.html#bootstrap-checks-tls" +BOOTSTRAP_CHECK_TOKEN_SSL: "bootstrap-checks-xpack.html#bootstrap-checks-xpack-token-ssl" +BOOTSTRAP_CHECK_SECURITY_MINIMAL_SETUP: "security-minimal-setup.html" +CONTACT_SUPPORT: "troubleshooting.html#troubleshooting-contact-support" +UNASSIGNED_SHARDS: "red-yellow-cluster-status.html" +EXECUTABLE_JNA_TMPDIR: "executable-jna-tmpdir.html" +NETWORK_THREADING_MODEL: "modules-network.html#modules-network-threading-model" +ALLOCATION_EXPLAIN_API: "cluster-allocation-explain.html" +NETWORK_BINDING_AND_PUBLISHING: "modules-network.html#modules-network-binding-publishing" +SNAPSHOT_REPOSITORY_ANALYSIS: "repo-analysis-api.html" +S3_COMPATIBLE_REPOSITORIES: "repository-s3.html#repository-s3-compatible-services" +LUCENE_MAX_DOCS_LIMIT: "size-your-shards.html#troubleshooting-max-docs-limit" +MAX_SHARDS_PER_NODE: "size-your-shards.html#troubleshooting-max-shards-open" +FLOOD_STAGE_WATERMARK: "fix-watermark-errors.html" +X_OPAQUE_ID: "api-conventions.html#x-opaque-id" +FORMING_SINGLE_NODE_CLUSTERS: "modules-discovery-bootstrap-cluster.html#modules-discovery-bootstrap-cluster-joining" diff --git a/server/src/test/java/org/elasticsearch/common/ReferenceDocsTests.java b/server/src/test/java/org/elasticsearch/common/ReferenceDocsTests.java index b383ce7b869ea..36a77fba1116c 100644 --- a/server/src/test/java/org/elasticsearch/common/ReferenceDocsTests.java +++ b/server/src/test/java/org/elasticsearch/common/ReferenceDocsTests.java @@ -39,9 +39,8 @@ public void testVersionComponent() { assertEquals("master", getVersionComponent("ABCDEF", true)); } - public void testResourceValidation() throws Exception { - - try (var builder = XContentFactory.jsonBuilder()) { + public void testReadsValidLinkDefinitions() throws Exception { + try (var builder = XContentFactory.yamlBuilder()) { builder.startObject(); for (ReferenceDocs link : ReferenceDocs.values()) { builder.field(link.name(), "TEST"); @@ -54,12 +53,16 @@ public void testResourceValidation() throws Exception { assertEquals("TEST", map.get(link.name())); } } + } + public void testRejectsInvalidJSON() throws Exception { try (var stream = new ByteArrayInputStream("{\"invalid\":".getBytes(StandardCharsets.UTF_8))) { expectThrows(XContentParseException.class, () -> ReferenceDocs.readLinksBySymbol(stream)); } + } - try (var builder = XContentFactory.jsonBuilder()) { + public void testRejectsBadStructure() throws Exception { + try (var builder = XContentFactory.yamlBuilder()) { builder.startObject(); for (ReferenceDocs link : ReferenceDocs.values()) { builder.field(link.name(), "TEST"); @@ -70,8 +73,10 @@ public void testResourceValidation() throws Exception { expectThrows(IllegalArgumentException.class, () -> ReferenceDocs.readLinksBySymbol(stream)); } } + } - try (var builder = XContentFactory.jsonBuilder()) { + public void testRejectsExtraSymbol() throws Exception { + try (var builder = XContentFactory.yamlBuilder()) { builder.startObject(); for (ReferenceDocs link : ReferenceDocs.values()) { builder.field(link.name(), "TEST"); @@ -82,8 +87,10 @@ public void testResourceValidation() throws Exception { expectThrows(IllegalStateException.class, () -> ReferenceDocs.readLinksBySymbol(stream)); } } + } - try (var builder = XContentFactory.jsonBuilder()) { + public void testRejectsMissingSymbol() throws Exception { + try (var builder = XContentFactory.yamlBuilder()) { builder.startObject(); var skipped = randomFrom(ReferenceDocs.values()); for (ReferenceDocs link : ReferenceDocs.values()) { @@ -97,8 +104,10 @@ public void testResourceValidation() throws Exception { expectThrows(IllegalStateException.class, () -> ReferenceDocs.readLinksBySymbol(stream)); } } + } - try (var builder = XContentFactory.jsonBuilder()) { + public void testRejectsIncorrectOrder() throws Exception { + try (var builder = XContentFactory.yamlBuilder()) { var shuffled = Arrays.copyOf(ReferenceDocs.values(), ReferenceDocs.values().length); var i = between(0, ReferenceDocs.values().length - 1); var j = randomValueOtherThan(i, () -> between(0, ReferenceDocs.values().length - 1)); @@ -117,4 +126,32 @@ public void testResourceValidation() throws Exception { } } } + + public void testRejectsAutoGeneratedFragment() throws Exception { + try (var builder = XContentFactory.yamlBuilder()) { + builder.startObject(); + for (ReferenceDocs link : ReferenceDocs.values()) { + builder.field(link.name(), "test.html#_auto_generated_fragment"); + } + builder.endObject(); + + try (var stream = BytesReference.bytes(builder).streamInput()) { + expectThrows(IllegalStateException.class, () -> ReferenceDocs.readLinksBySymbol(stream)); + } + } + } + + public void testRejectsAutoGeneratedPageName() throws Exception { + try (var builder = XContentFactory.yamlBuilder()) { + builder.startObject(); + for (ReferenceDocs link : ReferenceDocs.values()) { + builder.field(link.name(), "_auto_generated_page.html"); + } + builder.endObject(); + + try (var stream = BytesReference.bytes(builder).streamInput()) { + expectThrows(IllegalStateException.class, () -> ReferenceDocs.readLinksBySymbol(stream)); + } + } + } } From 268eccbe4ab56300ffcb609b1aa6cd9fbd11921f Mon Sep 17 00:00:00 2001 From: David Turner Date: Tue, 1 Oct 2024 07:25:53 +0100 Subject: [PATCH 2/2] Revert to JSON --- .../elasticsearch/common/ReferenceDocs.java | 8 ++-- .../common/reference-docs-links.json | 46 +++++++++++++++++++ .../common/reference-docs-links.yaml | 45 ------------------ .../common/ReferenceDocsTests.java | 14 +++--- 4 files changed, 57 insertions(+), 56 deletions(-) create mode 100644 server/src/main/resources/org/elasticsearch/common/reference-docs-links.json delete mode 100644 server/src/main/resources/org/elasticsearch/common/reference-docs-links.yaml diff --git a/server/src/main/java/org/elasticsearch/common/ReferenceDocs.java b/server/src/main/java/org/elasticsearch/common/ReferenceDocs.java index d7fd1a91ab8fd..e301d786ce5a4 100644 --- a/server/src/main/java/org/elasticsearch/common/ReferenceDocs.java +++ b/server/src/main/java/org/elasticsearch/common/ReferenceDocs.java @@ -27,11 +27,11 @@ /** * Encapsulates links to pages in the reference docs, so that for example we can include URLs in logs and API outputs. Each instance's * {@link #toString()} yields (a string representation of) a URL for the relevant docs. Links are defined in the resource file - * {@code reference-docs-links.yaml} which must include definitions for exactly the set of values of this enum. + * {@code reference-docs-links.json} which must include definitions for exactly the set of values of this enum. */ public enum ReferenceDocs { /* - * Note that the docs subsystem parses {@code reference-docs-links.yaml} with regexes, not a YAML parser, so the whitespace in the file + * Note that the docs subsystem parses {@code reference-docs-links.json} with regexes, not a JSON parser, so the whitespace in the file * is important too. See {@code sub check_elasticsearch_links} in {@code https://github.com/elastic/docs/blob/master/build_docs.pl} for * more details. * @@ -89,7 +89,7 @@ public enum ReferenceDocs { private static final Map linksBySymbol; static { - try (var resourceStream = readFromJarResourceUrl(ReferenceDocs.class.getResource("reference-docs-links.yaml"))) { + try (var resourceStream = readFromJarResourceUrl(ReferenceDocs.class.getResource("reference-docs-links.json"))) { linksBySymbol = Map.copyOf(readLinksBySymbol(resourceStream)); } catch (Exception e) { assert false : e; @@ -102,7 +102,7 @@ public enum ReferenceDocs { static final String VERSION_COMPONENT = getVersionComponent(Build.current().version(), Build.current().isSnapshot()); static Map readLinksBySymbol(InputStream inputStream) throws Exception { - try (var parser = XContentFactory.xContent(XContentType.YAML).createParser(XContentParserConfiguration.EMPTY, inputStream)) { + try (var parser = XContentFactory.xContent(XContentType.JSON).createParser(XContentParserConfiguration.EMPTY, inputStream)) { final var result = parser.map(LinkedHashMap::new, XContentParser::text); final var iterator = result.keySet().iterator(); for (int i = 0; i < values().length; i++) { diff --git a/server/src/main/resources/org/elasticsearch/common/reference-docs-links.json b/server/src/main/resources/org/elasticsearch/common/reference-docs-links.json new file mode 100644 index 0000000000000..1b5dc5b2f31e0 --- /dev/null +++ b/server/src/main/resources/org/elasticsearch/common/reference-docs-links.json @@ -0,0 +1,46 @@ +{ + "INITIAL_MASTER_NODES": "important-settings.html#initial_master_nodes", + "DISCOVERY_TROUBLESHOOTING": "discovery-troubleshooting.html", + "UNSTABLE_CLUSTER_TROUBLESHOOTING": "troubleshooting-unstable-cluster.html", + "LAGGING_NODE_TROUBLESHOOTING": "troubleshooting-unstable-cluster.html#troubleshooting-unstable-cluster-lagging", + "SHARD_LOCK_TROUBLESHOOTING": "troubleshooting-unstable-cluster.html#troubleshooting-unstable-cluster-shardlockobtainfailedexception", + "NETWORK_DISCONNECT_TROUBLESHOOTING": "troubleshooting-unstable-cluster.html#troubleshooting-unstable-cluster-network", + "CONCURRENT_REPOSITORY_WRITERS": "diagnosing-corrupted-repositories.html", + "ARCHIVE_INDICES": "archive-indices.html", + "HTTP_TRACER": "modules-network.html#http-rest-request-tracer", + "LOGGING": "logging.html", + "BOOTSTRAP_CHECK_HEAP_SIZE": "bootstrap-checks-heap-size.html", + "BOOTSTRAP_CHECK_FILE_DESCRIPTOR": "bootstrap-checks-file-descriptor.html", + "BOOTSTRAP_CHECK_MEMORY_LOCK": "bootstrap-checks-memory-lock.html", + "BOOTSTRAP_CHECK_MAX_NUMBER_THREADS": "max-number-threads-check.html", + "BOOTSTRAP_CHECK_MAX_FILE_SIZE": "bootstrap-checks-max-file-size.html", + "BOOTSTRAP_CHECK_MAX_SIZE_VIRTUAL_MEMORY": "max-size-virtual-memory-check.html", + "BOOTSTRAP_CHECK_MAXIMUM_MAP_COUNT": "bootstrap-checks-max-map-count.html", + "BOOTSTRAP_CHECK_CLIENT_JVM": "bootstrap-checks-client-jvm.html", + "BOOTSTRAP_CHECK_USE_SERIAL_COLLECTOR": "bootstrap-checks-serial-collector.html", + "BOOTSTRAP_CHECK_SYSTEM_CALL_FILTER": "bootstrap-checks-syscall-filter.html", + "BOOTSTRAP_CHECK_ONERROR_AND_ONOUTOFMEMORYERROR": "bootstrap-checks-onerror.html", + "BOOTSTRAP_CHECK_EARLY_ACCESS": "bootstrap-checks-early-access.html", + "BOOTSTRAP_CHECK_ALL_PERMISSION": "bootstrap-checks-all-permission.html", + "BOOTSTRAP_CHECK_DISCOVERY_CONFIGURATION": "bootstrap-checks-discovery-configuration.html", + "BOOTSTRAP_CHECKS": "bootstrap-checks.html", + "BOOTSTRAP_CHECK_ENCRYPT_SENSITIVE_DATA": "bootstrap-checks-xpack.html#bootstrap-checks-xpack-encrypt-sensitive-data", + "BOOTSTRAP_CHECK_PKI_REALM": "bootstrap-checks-xpack.html#bootstrap-checks-xpack-pki-realm", + "BOOTSTRAP_CHECK_ROLE_MAPPINGS": "bootstrap-checks-xpack.html#bootstrap-checks-xpack-role-mappings", + "BOOTSTRAP_CHECK_TLS": "bootstrap-checks-xpack.html#bootstrap-checks-tls", + "BOOTSTRAP_CHECK_TOKEN_SSL": "bootstrap-checks-xpack.html#bootstrap-checks-xpack-token-ssl", + "BOOTSTRAP_CHECK_SECURITY_MINIMAL_SETUP": "security-minimal-setup.html", + "CONTACT_SUPPORT": "troubleshooting.html#troubleshooting-contact-support", + "UNASSIGNED_SHARDS": "red-yellow-cluster-status.html", + "EXECUTABLE_JNA_TMPDIR": "executable-jna-tmpdir.html", + "NETWORK_THREADING_MODEL": "modules-network.html#modules-network-threading-model", + "ALLOCATION_EXPLAIN_API": "cluster-allocation-explain.html", + "NETWORK_BINDING_AND_PUBLISHING": "modules-network.html#modules-network-binding-publishing", + "SNAPSHOT_REPOSITORY_ANALYSIS": "repo-analysis-api.html", + "S3_COMPATIBLE_REPOSITORIES": "repository-s3.html#repository-s3-compatible-services", + "LUCENE_MAX_DOCS_LIMIT": "size-your-shards.html#troubleshooting-max-docs-limit", + "MAX_SHARDS_PER_NODE": "size-your-shards.html#troubleshooting-max-shards-open", + "FLOOD_STAGE_WATERMARK": "fix-watermark-errors.html", + "X_OPAQUE_ID": "api-conventions.html#x-opaque-id", + "FORMING_SINGLE_NODE_CLUSTERS": "modules-discovery-bootstrap-cluster.html#modules-discovery-bootstrap-cluster-joining" +} diff --git a/server/src/main/resources/org/elasticsearch/common/reference-docs-links.yaml b/server/src/main/resources/org/elasticsearch/common/reference-docs-links.yaml deleted file mode 100644 index 3acd1cf1fd461..0000000000000 --- a/server/src/main/resources/org/elasticsearch/common/reference-docs-links.yaml +++ /dev/null @@ -1,45 +0,0 @@ ---- -INITIAL_MASTER_NODES: "important-settings.html#initial_master_nodes" -DISCOVERY_TROUBLESHOOTING: "discovery-troubleshooting.html" -UNSTABLE_CLUSTER_TROUBLESHOOTING: "troubleshooting-unstable-cluster.html" -LAGGING_NODE_TROUBLESHOOTING: "troubleshooting-unstable-cluster.html#troubleshooting-unstable-cluster-lagging" -SHARD_LOCK_TROUBLESHOOTING: "troubleshooting-unstable-cluster.html#troubleshooting-unstable-cluster-shardlockobtainfailedexception" -NETWORK_DISCONNECT_TROUBLESHOOTING: "troubleshooting-unstable-cluster.html#troubleshooting-unstable-cluster-network" -CONCURRENT_REPOSITORY_WRITERS: "diagnosing-corrupted-repositories.html" -ARCHIVE_INDICES: "archive-indices.html" -HTTP_TRACER: "modules-network.html#http-rest-request-tracer" -LOGGING: "logging.html" -BOOTSTRAP_CHECK_HEAP_SIZE: "bootstrap-checks-heap-size.html" -BOOTSTRAP_CHECK_FILE_DESCRIPTOR: "bootstrap-checks-file-descriptor.html" -BOOTSTRAP_CHECK_MEMORY_LOCK: "bootstrap-checks-memory-lock.html" -BOOTSTRAP_CHECK_MAX_NUMBER_THREADS: "max-number-threads-check.html" -BOOTSTRAP_CHECK_MAX_FILE_SIZE: "bootstrap-checks-max-file-size.html" -BOOTSTRAP_CHECK_MAX_SIZE_VIRTUAL_MEMORY: "max-size-virtual-memory-check.html" -BOOTSTRAP_CHECK_MAXIMUM_MAP_COUNT: "bootstrap-checks-max-map-count.html" -BOOTSTRAP_CHECK_CLIENT_JVM: "bootstrap-checks-client-jvm.html" -BOOTSTRAP_CHECK_USE_SERIAL_COLLECTOR: "bootstrap-checks-serial-collector.html" -BOOTSTRAP_CHECK_SYSTEM_CALL_FILTER: "bootstrap-checks-syscall-filter.html" -BOOTSTRAP_CHECK_ONERROR_AND_ONOUTOFMEMORYERROR: "bootstrap-checks-onerror.html" -BOOTSTRAP_CHECK_EARLY_ACCESS: "bootstrap-checks-early-access.html" -BOOTSTRAP_CHECK_ALL_PERMISSION: "bootstrap-checks-all-permission.html" -BOOTSTRAP_CHECK_DISCOVERY_CONFIGURATION: "bootstrap-checks-discovery-configuration.html" -BOOTSTRAP_CHECKS: "bootstrap-checks.html" -BOOTSTRAP_CHECK_ENCRYPT_SENSITIVE_DATA: "bootstrap-checks-xpack.html#bootstrap-checks-xpack-encrypt-sensitive-data" -BOOTSTRAP_CHECK_PKI_REALM: "bootstrap-checks-xpack.html#bootstrap-checks-xpack-pki-realm" -BOOTSTRAP_CHECK_ROLE_MAPPINGS: "bootstrap-checks-xpack.html#bootstrap-checks-xpack-role-mappings" -BOOTSTRAP_CHECK_TLS: "bootstrap-checks-xpack.html#bootstrap-checks-tls" -BOOTSTRAP_CHECK_TOKEN_SSL: "bootstrap-checks-xpack.html#bootstrap-checks-xpack-token-ssl" -BOOTSTRAP_CHECK_SECURITY_MINIMAL_SETUP: "security-minimal-setup.html" -CONTACT_SUPPORT: "troubleshooting.html#troubleshooting-contact-support" -UNASSIGNED_SHARDS: "red-yellow-cluster-status.html" -EXECUTABLE_JNA_TMPDIR: "executable-jna-tmpdir.html" -NETWORK_THREADING_MODEL: "modules-network.html#modules-network-threading-model" -ALLOCATION_EXPLAIN_API: "cluster-allocation-explain.html" -NETWORK_BINDING_AND_PUBLISHING: "modules-network.html#modules-network-binding-publishing" -SNAPSHOT_REPOSITORY_ANALYSIS: "repo-analysis-api.html" -S3_COMPATIBLE_REPOSITORIES: "repository-s3.html#repository-s3-compatible-services" -LUCENE_MAX_DOCS_LIMIT: "size-your-shards.html#troubleshooting-max-docs-limit" -MAX_SHARDS_PER_NODE: "size-your-shards.html#troubleshooting-max-shards-open" -FLOOD_STAGE_WATERMARK: "fix-watermark-errors.html" -X_OPAQUE_ID: "api-conventions.html#x-opaque-id" -FORMING_SINGLE_NODE_CLUSTERS: "modules-discovery-bootstrap-cluster.html#modules-discovery-bootstrap-cluster-joining" diff --git a/server/src/test/java/org/elasticsearch/common/ReferenceDocsTests.java b/server/src/test/java/org/elasticsearch/common/ReferenceDocsTests.java index 36a77fba1116c..a5efd578df36c 100644 --- a/server/src/test/java/org/elasticsearch/common/ReferenceDocsTests.java +++ b/server/src/test/java/org/elasticsearch/common/ReferenceDocsTests.java @@ -40,7 +40,7 @@ public void testVersionComponent() { } public void testReadsValidLinkDefinitions() throws Exception { - try (var builder = XContentFactory.yamlBuilder()) { + try (var builder = XContentFactory.jsonBuilder()) { builder.startObject(); for (ReferenceDocs link : ReferenceDocs.values()) { builder.field(link.name(), "TEST"); @@ -62,7 +62,7 @@ public void testRejectsInvalidJSON() throws Exception { } public void testRejectsBadStructure() throws Exception { - try (var builder = XContentFactory.yamlBuilder()) { + try (var builder = XContentFactory.jsonBuilder()) { builder.startObject(); for (ReferenceDocs link : ReferenceDocs.values()) { builder.field(link.name(), "TEST"); @@ -76,7 +76,7 @@ public void testRejectsBadStructure() throws Exception { } public void testRejectsExtraSymbol() throws Exception { - try (var builder = XContentFactory.yamlBuilder()) { + try (var builder = XContentFactory.jsonBuilder()) { builder.startObject(); for (ReferenceDocs link : ReferenceDocs.values()) { builder.field(link.name(), "TEST"); @@ -90,7 +90,7 @@ public void testRejectsExtraSymbol() throws Exception { } public void testRejectsMissingSymbol() throws Exception { - try (var builder = XContentFactory.yamlBuilder()) { + try (var builder = XContentFactory.jsonBuilder()) { builder.startObject(); var skipped = randomFrom(ReferenceDocs.values()); for (ReferenceDocs link : ReferenceDocs.values()) { @@ -107,7 +107,7 @@ public void testRejectsMissingSymbol() throws Exception { } public void testRejectsIncorrectOrder() throws Exception { - try (var builder = XContentFactory.yamlBuilder()) { + try (var builder = XContentFactory.jsonBuilder()) { var shuffled = Arrays.copyOf(ReferenceDocs.values(), ReferenceDocs.values().length); var i = between(0, ReferenceDocs.values().length - 1); var j = randomValueOtherThan(i, () -> between(0, ReferenceDocs.values().length - 1)); @@ -128,7 +128,7 @@ public void testRejectsIncorrectOrder() throws Exception { } public void testRejectsAutoGeneratedFragment() throws Exception { - try (var builder = XContentFactory.yamlBuilder()) { + try (var builder = XContentFactory.jsonBuilder()) { builder.startObject(); for (ReferenceDocs link : ReferenceDocs.values()) { builder.field(link.name(), "test.html#_auto_generated_fragment"); @@ -142,7 +142,7 @@ public void testRejectsAutoGeneratedFragment() throws Exception { } public void testRejectsAutoGeneratedPageName() throws Exception { - try (var builder = XContentFactory.yamlBuilder()) { + try (var builder = XContentFactory.jsonBuilder()) { builder.startObject(); for (ReferenceDocs link : ReferenceDocs.values()) { builder.field(link.name(), "_auto_generated_page.html");