Default to SSHA-256 as API key stored credential hasher

docs/changelog/120997.yaml

@@ -0,0 +1,5 @@
+pr: 120997
+summary: Allow `SSHA-256` for API key credential hash
+area: Authentication
+type: enhancement
+issues: []

docs/reference/settings/security-hash-settings.asciidoc

@@ -124,4 +124,68 @@ following:
                              initial input with SHA512 first.
 |=======================
 
+Furthermore, {es} supports authentication via securely-generated high entropy tokens,
+for instance <<security-api-create-api-key,API keys>>.
+Analogous to passwords, only the tokens' hashes are stored. Since the tokens are guaranteed
+to have sufficiently high entropy to resist offline attacks, secure salted hash functions are supported
+in addition to the password-hashing algorithms mentioned above.
 
+You can configure the algorithm for API key stored credential hashing
+by setting the <<static-cluster-setting,static>>
+`xpack.security.authc.api_key.hashing.algorithm` setting to one of the
+following
+
+[[secure-token-hashing-algorithms]]
+.Secure token hashing algorithms
+|=======================
+| Algorithm               | | | Description
+
+| `ssha256`               | | | Uses a salted `sha-256` algorithm. (default)
+| `bcrypt`                | | | Uses `bcrypt` algorithm with salt generated in 1024 rounds.
+| `bcrypt4`               | | | Uses `bcrypt` algorithm with salt generated in 16 rounds.
+| `bcrypt5`               | | | Uses `bcrypt` algorithm with salt generated in 32 rounds.
+| `bcrypt6`               | | | Uses `bcrypt` algorithm with salt generated in 64 rounds.
+| `bcrypt7`               | | | Uses `bcrypt` algorithm with salt generated in 128 rounds.
+| `bcrypt8`               | | | Uses `bcrypt` algorithm with salt generated in 256 rounds.
+| `bcrypt9`               | | | Uses `bcrypt` algorithm with salt generated in 512 rounds.
+| `bcrypt10`              | | | Uses `bcrypt` algorithm with salt generated in 1024 rounds.
+| `bcrypt11`              | | | Uses `bcrypt` algorithm with salt generated in 2048 rounds.
+| `bcrypt12`              | | | Uses `bcrypt` algorithm with salt generated in 4096 rounds.
+| `bcrypt13`              | | | Uses `bcrypt` algorithm with salt generated in 8192 rounds.
+| `bcrypt14`              | | | Uses `bcrypt` algorithm with salt generated in 16384 rounds.
+| `pbkdf2`                | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
+                             pseudorandom function using 10000 iterations.
+| `pbkdf2_1000`           | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
+                             pseudorandom function using 1000 iterations.
+| `pbkdf2_10000`          | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
+                             pseudorandom function using 10000 iterations.
+| `pbkdf2_50000`          | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
+                             pseudorandom function using 50000 iterations.
+| `pbkdf2_100000`         | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
+                             pseudorandom function using 100000 iterations.
+| `pbkdf2_500000`         | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
+                              pseudorandom function using 500000 iterations.
+| `pbkdf2_1000000`        | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
+                             pseudorandom function using 1000000 iterations.
+| `pbkdf2_stretch`        | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
+                             pseudorandom function using 10000 iterations, after hashing the
+                             initial input with SHA512 first.
+| `pbkdf2_stretch_1000`   | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
+                             pseudorandom function using 1000 iterations, after hashing the
+                             initial input with SHA512 first.
+| `pbkdf2_stretch_10000`  | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
+                             pseudorandom function using 10000 iterations, after hashing the
+                             initial input with SHA512 first.
+| `pbkdf2_stretch_50000`  | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
+                             pseudorandom function using 50000 iterations, after hashing the
+                             initial input with SHA512 first.
+| `pbkdf2_stretch_100000` | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
+                             pseudorandom function using 100000 iterations, after hashing the
+                             initial input with SHA512 first.
+| `pbkdf2_stretch_500000` | | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
+                             pseudorandom function using 500000 iterations, after hashing the
+                             initial input with SHA512 first.
+| `pbkdf2_stretch_1000000`| | | Uses `PBKDF2` key derivation function with `HMAC-SHA512` as a
+                             pseudorandom function using 1000000 iterations, after hashing the
+                             initial input with SHA512 first.
+|=======================

docs/reference/settings/security-settings.asciidoc

@@ -23,8 +23,8 @@ For more information about creating and updating the {es} keystore, see
 ==== General security settings
 `xpack.security.enabled`::
 (<<static-cluster-setting,Static>>)
-Defaults to `true`, which enables {es} {security-features} on the node. 
-This setting must be enabled to use Elasticsearch's authentication, 
+Defaults to `true`, which enables {es} {security-features} on the node.
+This setting must be enabled to use Elasticsearch's authentication,
 authorization and audit features. +
 +
 --
@@ -229,7 +229,7 @@ Defaults to `7d`.
 
 --
 NOTE:  Large real-time clock inconsistency across cluster nodes can cause problems
-with evaluating the API key retention period. That is, if the clock on the node 
+with evaluating the API key retention period. That is, if the clock on the node
 invalidating the API key is significantly different than the one performing the deletion,
 the key may be retained for longer or shorter than the configured retention period.
 
@@ -252,7 +252,7 @@ Sets the timeout of the internal search and delete call.
 `xpack.security.authc.api_key.hashing.algorithm`::
 (<<static-cluster-setting,Static>>)
 Specifies the hashing algorithm that is used for securing API key credentials.
-See <<password-hashing-algorithms>>. Defaults to `pbkdf2`.
+See <<secure-token-hashing-algorithms>>. Defaults to `ssha256`.
 
 [discrete]
 [[security-domain-settings]]

server/src/main/java/org/elasticsearch/common/SecureRandomUtils.java

@@ -0,0 +1,41 @@
+/*
+ * Copyright Elasticsearch B.V. and/or licensed to Elasticsearch B.V. under one
+ * or more contributor license agreements. Licensed under the "Elastic License
+ * 2.0", the "GNU Affero General Public License v3.0 only", and the "Server Side
+ * Public License v 1"; you may not use this file except in compliance with, at
+ * your election, the "Elastic License 2.0", the "GNU Affero General Public
+ * License v3.0 only", or the "Server Side Public License, v 1".
+ */
+
+package org.elasticsearch.common;
+
+import org.elasticsearch.common.settings.SecureString;
+import org.elasticsearch.core.CharArrays;
+
+import java.util.Arrays;
+import java.util.Base64;
+
+public final class SecureRandomUtils {
+    private SecureRandomUtils() {}
+
+    /**
+     * Returns a cryptographically secure Base64 encoded {@link SecureString} of {@code numBytes} random bytes.
+     */
+    public static SecureString getBase64SecureRandomString(int numBytes) {
+        byte[] randomBytes = null;
+        byte[] encodedBytes = null;
+        try {
+            randomBytes = new byte[numBytes];
+            SecureRandomHolder.INSTANCE.nextBytes(randomBytes);
+            encodedBytes = Base64.getUrlEncoder().withoutPadding().encode(randomBytes);
+            return new SecureString(CharArrays.utf8BytesToChars(encodedBytes));
+        } finally {
+            if (randomBytes != null) {
+                Arrays.fill(randomBytes, (byte) 0);
+            }
+            if (encodedBytes != null) {
+                Arrays.fill(encodedBytes, (byte) 0);
+            }
+        }
+    }
+}

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/XPackSettings.java

@@ -240,7 +240,7 @@ public Iterator<Setting<?>> settings() {
 
     public static final List<String> DEFAULT_CIPHERS = JDK12_CIPHERS;
 
-    public static final Setting<String> PASSWORD_HASHING_ALGORITHM = defaultStoredHashAlgorithmSetting(
+    public static final Setting<String> PASSWORD_HASHING_ALGORITHM = defaultStoredPasswordHashAlgorithmSetting(
         "xpack.security.authc.password_hashing.algorithm",
         (s) -> {
             if (XPackSettings.FIPS_MODE_ENABLED.get(s)) {
@@ -251,19 +251,56 @@ public Iterator<Setting<?>> settings() {
         }
     );
 
-    public static final Setting<String> SERVICE_TOKEN_HASHING_ALGORITHM = defaultStoredHashAlgorithmSetting(
+    public static final Setting<String> SERVICE_TOKEN_HASHING_ALGORITHM = defaultStoredPasswordHashAlgorithmSetting(
         "xpack.security.authc.service_token_hashing.algorithm",
         (s) -> Hasher.PBKDF2_STRETCH.name()
     );
 
     /*
      * Do not allow insecure hashing algorithms to be used for password hashing
      */
-    public static Setting<String> defaultStoredHashAlgorithmSetting(String key, Function<Settings, String> defaultHashingAlgorithm) {
+    public static Setting<String> defaultStoredPasswordHashAlgorithmSetting(
+        String key,
+        Function<Settings, String> defaultHashingAlgorithm
+    ) {
         return new Setting<>(key, defaultHashingAlgorithm, Function.identity(), v -> {
-            if (Hasher.getAvailableAlgoStoredHash().contains(v.toLowerCase(Locale.ROOT)) == false) {
+            if (Hasher.getAvailableAlgoStoredPasswordHash().contains(v.toLowerCase(Locale.ROOT)) == false) {
                 throw new IllegalArgumentException(
-                    "Invalid algorithm: " + v + ". Valid values for password hashing are " + Hasher.getAvailableAlgoStoredHash().toString()
+                    "Invalid algorithm: "
+                        + v
+                        + ". Valid values for password hashing are "
+                        + Hasher.getAvailableAlgoStoredPasswordHash().toString()
+                );
+            } else if (v.regionMatches(true, 0, "pbkdf2", 0, "pbkdf2".length())) {
+                try {
+                    SecretKeyFactory.getInstance("PBKDF2withHMACSHA512");
+                } catch (NoSuchAlgorithmException e) {
+                    throw new IllegalArgumentException(
+                        "Support for PBKDF2WithHMACSHA512 must be available in order to use any of the PBKDF2 algorithms for the ["
+                            + key
+                            + "] setting.",
+                        e
+                    );
+                }
+            }
+        }, Property.NodeScope);
+    }
+
+    /**
+     * Similar to {@link #defaultStoredPasswordHashAlgorithmSetting(String, Function)} but for secure, high-entropy tokens so salted secure
+     * hashing algorithms are allowed, in addition to algorithms that are suitable for password hashing.
+     */
+    public static Setting<String> defaultStoredSecureTokenHashAlgorithmSetting(
+        String key,
+        Function<Settings, String> defaultHashingAlgorithm
+    ) {
+        return new Setting<>(key, defaultHashingAlgorithm, Function.identity(), v -> {
+            if (Hasher.getAvailableAlgoStoredSecureTokenHash().contains(v.toLowerCase(Locale.ROOT)) == false) {
+                throw new IllegalArgumentException(
+                    "Invalid algorithm: "
+                        + v
+                        + ". Valid values for secure token hashing are "
+                        + Hasher.getAvailableAlgoStoredPasswordHash().toString()
                 );
             } else if (v.regionMatches(true, 0, "pbkdf2", 0, "pbkdf2".length())) {
                 try {

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/action/user/PutUserRequestBuilder.java

@@ -89,11 +89,11 @@ public PutUserRequestBuilder email(String email) {
     public PutUserRequestBuilder passwordHash(char[] passwordHash, Hasher configuredHasher) {
         final Hasher resolvedHasher = Hasher.resolveFromHash(passwordHash);
         if (resolvedHasher.equals(configuredHasher) == false
-            && Hasher.getAvailableAlgoStoredHash().contains(resolvedHasher.name().toLowerCase(Locale.ROOT)) == false) {
+            && Hasher.getAvailableAlgoStoredPasswordHash().contains(resolvedHasher.name().toLowerCase(Locale.ROOT)) == false) {
             throw new IllegalArgumentException(
                 "The provided password hash is not a hash or it could not be resolved to a supported hash algorithm. "
                     + "The supported password hash algorithms are "
-                    + Hasher.getAvailableAlgoStoredHash().toString()
+                    + Hasher.getAvailableAlgoStoredPasswordHash().toString()
             );
         }
         if (request.passwordHash() != null) {

x-pack/plugin/core/src/main/java/org/elasticsearch/xpack/core/security/authc/support/Hasher.java

@@ -735,14 +735,28 @@ private static boolean verifyBcryptHash(SecureString text, char[] hash) {
      * an instance of the appropriate {@link Hasher} by using {@link #resolve(String) resolve()}
      */
     @SuppressForbidden(reason = "This is the only allowed way to get available values")
-    public static List<String> getAvailableAlgoStoredHash() {
+    public static List<String> getAvailableAlgoStoredPasswordHash() {
         return Arrays.stream(Hasher.values())
             .map(Hasher::name)
             .map(name -> name.toLowerCase(Locale.ROOT))
             .filter(name -> (name.startsWith("pbkdf2") || name.startsWith("bcrypt")))
             .collect(Collectors.toList());
     }
 
+    /**
+     * Returns a list of lower case String identifiers for the Hashing algorithm and parameter
+     * combinations that can be used for password hashing. The identifiers can be used to get
+     * an instance of the appropriate {@link Hasher} by using {@link #resolve(String) resolve()}
+     */
+    @SuppressForbidden(reason = "This is the only allowed way to get available values")
+    public static List<String> getAvailableAlgoStoredSecureTokenHash() {
+        return Arrays.stream(Hasher.values())
+            .map(Hasher::name)
+            .map(name -> name.toLowerCase(Locale.ROOT))
+            .filter(name -> (name.startsWith("pbkdf2") || name.startsWith("bcrypt") || name.equals("ssha256")))
+            .collect(Collectors.toList());
+    }
+
     /**
      * Returns a list of lower case String identifiers for the Hashing algorithm and parameter
      * combinations that can be used for password hashing in the cache. The identifiers can be used to get

x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/Security.java

@@ -1472,7 +1472,7 @@ public static List<Setting<?>> getSettings(List<SecurityExtension> securityExten
         settingsList.add(TokenService.DELETE_INTERVAL);
         settingsList.add(TokenService.DELETE_TIMEOUT);
         settingsList.addAll(SSLConfigurationSettings.getProfileSettings());
-        settingsList.add(ApiKeyService.PASSWORD_HASHING_ALGORITHM);
+        settingsList.add(ApiKeyService.STORED_HASH_ALGO_SETTING);
         settingsList.add(ApiKeyService.DELETE_TIMEOUT);
         settingsList.add(ApiKeyService.DELETE_INTERVAL);
         settingsList.add(ApiKeyService.DELETE_RETENTION_PERIOD);
@@ -1818,17 +1818,30 @@ static void validateForFips(Settings settings) {
                     + " ] setting."
             );
         }
-        Stream.of(ApiKeyService.PASSWORD_HASHING_ALGORITHM, XPackSettings.SERVICE_TOKEN_HASHING_ALGORITHM).forEach((setting) -> {
-            final var storedHashAlgo = setting.get(settings);
-            if (storedHashAlgo.toLowerCase(Locale.ROOT).startsWith("pbkdf2") == false) {
-                // log instead of validation error for backwards compatibility
-                logger.warn(
-                    "Only PBKDF2 is allowed for stored credential hashing in a FIPS 140 JVM. "
-                        + "Please set the appropriate value for [{}] setting.",
-                    setting.getKey()
-                );
-            }
-        });
+
+        final var serviceTokenStoredHashSettings = XPackSettings.SERVICE_TOKEN_HASHING_ALGORITHM;
+        final var serviceTokenStoredHashAlgo = serviceTokenStoredHashSettings.get(settings);
+        if (serviceTokenStoredHashAlgo.toLowerCase(Locale.ROOT).startsWith("pbkdf2") == false) {
+            // log instead of validation error for backwards compatibility
+            logger.warn(
+                "Only PBKDF2 is allowed for stored credential hashing in a FIPS 140 JVM. "
+                    + "Please set the appropriate value for [{}] setting.",
+                serviceTokenStoredHashSettings.getKey()
+            );
+        }
+
+        final var apiKeyStoredHashSettings = ApiKeyService.STORED_HASH_ALGO_SETTING;
+        final var apiKeyStoredHashAlgo = apiKeyStoredHashSettings.get(settings);
+        if (apiKeyStoredHashAlgo.toLowerCase(Locale.ROOT).startsWith("ssha256") == false
+            || apiKeyStoredHashAlgo.toLowerCase(Locale.ROOT).startsWith("pbkdf2") == false) {
+            // log instead of validation error for backwards compatibility
+            logger.warn(
+                "[{}] is not recommended for stored API key hashing in a FIPS 140 JVM. The recommended hasher for [{}] is SSHA256.",
+                apiKeyStoredHashSettings,
+                apiKeyStoredHashSettings.getKey()
+            );
+        }
+
         final var cacheHashAlgoSettings = settings.filter(k -> k.endsWith(".cache.hash_algo"));
         cacheHashAlgoSettings.keySet().forEach((key) -> {
             final var setting = cacheHashAlgoSettings.get(key);

x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/action/user/ChangePasswordRequestBuilder.java

@@ -72,11 +72,11 @@ public ChangePasswordRequestBuilder password(char[] password, Hasher hasher) {
     public ChangePasswordRequestBuilder passwordHash(char[] passwordHashChars, Hasher configuredHasher) {
         final Hasher resolvedHasher = Hasher.resolveFromHash(passwordHashChars);
         if (resolvedHasher.equals(configuredHasher) == false
-            && Hasher.getAvailableAlgoStoredHash().contains(resolvedHasher.name().toLowerCase(Locale.ROOT)) == false) {
+            && Hasher.getAvailableAlgoStoredPasswordHash().contains(resolvedHasher.name().toLowerCase(Locale.ROOT)) == false) {
             throw new IllegalArgumentException(
                 "The provided password hash is not a hash or it could not be resolved to a supported hash algorithm. "
                     + "The supported password hash algorithms are "
-                    + Hasher.getAvailableAlgoStoredHash().toString()
+                    + Hasher.getAvailableAlgoStoredPasswordHash().toString()
             );
         }
         if (request.passwordHash() != null) {

x-pack/plugin/security/src/main/java/org/elasticsearch/xpack/security/action/user/TransportChangePasswordAction.java

@@ -52,12 +52,12 @@ protected void doExecute(Task task, ChangePasswordRequest request, ActionListene
         final Hasher requestPwdHashAlgo = Hasher.resolveFromHash(request.passwordHash());
         final Hasher configPwdHashAlgo = Hasher.resolve(XPackSettings.PASSWORD_HASHING_ALGORITHM.get(settings));
         if (requestPwdHashAlgo.equals(configPwdHashAlgo) == false
-            && Hasher.getAvailableAlgoStoredHash().contains(requestPwdHashAlgo.name().toLowerCase(Locale.ROOT)) == false) {
+            && Hasher.getAvailableAlgoStoredPasswordHash().contains(requestPwdHashAlgo.name().toLowerCase(Locale.ROOT)) == false) {
             listener.onFailure(
                 new IllegalArgumentException(
                     "The provided password hash is not a hash or it could not be resolved to a supported hash algorithm. "
                         + "The supported password hash algorithms are "
-                        + Hasher.getAvailableAlgoStoredHash().toString()
+                        + Hasher.getAvailableAlgoStoredPasswordHash().toString()
                 )
             );
             return;