Skip to content

Commit 0197084

Browse files
beanuwaveiigoninbennygoerzigKarstenSchnitterKai Sternad
authored
Make test-suite runnable under FIPS JVM (#18491)
* Make test-suite runnable under FIPS JVM Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> * in addition, to 'Use TemporaryFolder JUnit @rule to manage temporary folders in tests' Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> * use testFipsRuntimeOnly instead of fipsRuntimeOnly for :client:sniffer Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> * Use reflection inside libs:ssl-config instead of CryptoServicesRegistrar Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> * revert password check for private key on PemKeyConfig; use Map.of() for TYPE_TO_EXTENSION_MAP; revert changes to SecureSettingsHelpers.getSecureSettingsProvider; revert changes to AzureClassic Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> * Separate FIPS-specific tests into dedicated FipsTests classes Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> * fix spotlessCheck Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> * for GCS plugin prioritize custom truststore. With fallback to default google.p12 in non-FIPS mode and an exception in FIPS mode when no truststore is set up. Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> * Add the @after annotation; regenerate all test certs with SANs included Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> * Optimize FIPS test performance by caching keystores and SSL factories Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> * add required FIPS deps Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> * fix forbiddenApisTest & use RSA 2048 alg to generate certs with KeyStoreUtils Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> * replace `CryptoServicesRegistrar` usage with `KeyStoreUtil.IN_FIPS_MODE`. Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> * use SslContext caching for fips-aware client/server tests. Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> * Test improvements and FIPS compliance fixes - Fix System.in cleanup in Bootstrap tests - Add validation for certificate algorithm extraction - Simplify SSLContext to use generic "TLS" protocol - Add null-check for FIPS truststore resource - Refactor exception testing patterns Signed-off-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]> --------- Signed-off-by: Igonin <[email protected]> Co-authored-by: Igonin <[email protected]> Co-authored-by: Benny Goerzig <[email protected]> Co-authored-by: Karsten Schnitter <[email protected]> Co-authored-by: Kai Sternad <[email protected]>
1 parent 1ee30dc commit 0197084

File tree

157 files changed

+3024
-646
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

157 files changed

+3024
-646
lines changed

‎CHANGELOG.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -24,6 +24,7 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/),
2424
- Add cluster defaults for merge autoThrottle, maxMergeThreads, and maxMergeCount; Add segment size filter to the merged segment warmer ([#19629](https://github.com/opensearch-project/OpenSearch/pull/19629))
2525
- Add build-tooling to run in FIPS environment ([#18921](https://github.com/opensearch-project/OpenSearch/pull/18921))
2626
- Add SMILE/CBOR/YAML document format support to Bulk GRPC endpoint ([#19744](https://github.com/opensearch-project/OpenSearch/pull/19744))
27+
- Make test-suite runnable under FIPS compliance support ([#18491](https://github.com/opensearch-project/OpenSearch/pull/18491))
2728
- Implement GRPC Search params `Highlight`and `Sort` ([#19868](https://github.com/opensearch-project/OpenSearch/pull/19868))
2829
- Implement GRPC ConstantScoreQuery, FuzzyQuery, MatchBoolPrefixQuery, MatchPhrasePrefix, PrefixQuery, MatchQuery ([#19854](https://github.com/opensearch-project/OpenSearch/pull/19854))
2930
- Add async periodic flush task support for pull-based ingestion ([#19878](https://github.com/opensearch-project/OpenSearch/pull/19878))

‎buildSrc/src/main/resources/test/ssl/test-node.bcfks‎

3.69 KB
Binary file not shown.

‎client/rest-high-level/build.gradle‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -36,6 +36,7 @@ apply plugin: 'opensearch.build'
3636
apply plugin: 'opensearch.rest-test'
3737
apply plugin: 'opensearch.publish'
3838
apply plugin: 'opensearch.rest-resources'
39+
apply from: "$rootDir/gradle/fips.gradle"
3940

4041
base {
4142
group = 'org.opensearch.client'
@@ -66,6 +67,9 @@ dependencies {
6667
testImplementation "junit:junit:${versions.junit}"
6768
//this is needed to make RestHighLevelClientTests#testApiNamingConventions work from IDEs
6869
testImplementation project(":rest-api-spec")
70+
testFipsRuntimeOnly "org.bouncycastle:bc-fips:${versions.bouncycastle_jce}"
71+
testFipsRuntimeOnly "org.bouncycastle:bctls-fips:${versions.bouncycastle_tls}"
72+
testFipsRuntimeOnly "org.bouncycastle:bcutil-fips:${versions.bouncycastle_util}"
6973
}
7074

7175
tasks.named('forbiddenApisMain').configure {

‎client/rest/src/test/java/org/opensearch/client/RestClientBuilderIntegTests.java‎

Lines changed: 9 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -56,22 +56,22 @@
5656
import java.security.PrivilegedAction;
5757
import java.security.SecureRandom;
5858

59+
import static org.hamcrest.MatcherAssert.assertThat;
5960
import static org.hamcrest.Matchers.instanceOf;
6061
import static org.junit.Assert.assertEquals;
61-
import static org.junit.Assert.assertThat;
6262
import static org.junit.Assert.fail;
6363

6464
/**
6565
* Integration test to validate the builder builds a client with the correct configuration
6666
*/
67-
public class RestClientBuilderIntegTests extends RestClientTestCase {
67+
public class RestClientBuilderIntegTests extends RestClientTestCase implements RestClientFipsAwareTestCase {
6868

6969
private static HttpsServer httpsServer;
7070

7171
@BeforeClass
7272
public static void startHttpServer() throws Exception {
7373
httpsServer = HttpsServer.create(new InetSocketAddress(InetAddress.getLoopbackAddress(), 0), 0);
74-
httpsServer.setHttpsConfigurator(new HttpsConfigurator(getSslContext(true)));
74+
httpsServer.setHttpsConfigurator(new HttpsConfigurator(new RestClientBuilderIntegTests().getSslContext(true)));
7575
httpsServer.createContext("/", new ResponseHandler());
7676
httpsServer.start();
7777
}
@@ -91,7 +91,6 @@ public static void stopHttpServers() throws IOException {
9191
}
9292

9393
public void testBuilderUsesDefaultSSLContext() throws Exception {
94-
assumeFalse("https://github.com/elastic/elasticsearch/issues/49094", inFipsJvm());
9594
final SSLContext defaultSSLContext = SSLContext.getDefault();
9695
try {
9796
try (RestClient client = buildRestClient()) {
@@ -118,24 +117,23 @@ private RestClient buildRestClient() {
118117
return RestClient.builder(new HttpHost("https", address.getHostString(), address.getPort())).build();
119118
}
120119

121-
private static SSLContext getSslContext(boolean server) throws Exception {
120+
@Override
121+
public SSLContext getSslContext(boolean server, String keyStoreType, SecureRandom secureRandom, String fileExtension) throws Exception {
122122
SSLContext sslContext;
123123
char[] password = "password".toCharArray();
124-
SecureRandom secureRandom = SecureRandom.getInstanceStrong();
125-
String fileExtension = ".jks";
126124

127125
try (
128126
InputStream trustStoreFile = RestClientBuilderIntegTests.class.getResourceAsStream("/test_truststore" + fileExtension);
129127
InputStream keyStoreFile = RestClientBuilderIntegTests.class.getResourceAsStream("/testks" + fileExtension)
130128
) {
131-
KeyStore keyStore = KeyStore.getInstance("JKS");
129+
KeyStore keyStore = KeyStore.getInstance(keyStoreType);
132130
keyStore.load(keyStoreFile, password);
133-
KeyManagerFactory kmf = KeyManagerFactory.getInstance("PKIX");
131+
KeyManagerFactory kmf = KeyManagerFactory.getInstance(KeyManagerFactory.getDefaultAlgorithm());
134132
kmf.init(keyStore, password);
135133

136-
KeyStore trustStore = KeyStore.getInstance("JKS");
134+
KeyStore trustStore = KeyStore.getInstance(keyStoreType);
137135
trustStore.load(trustStoreFile, password);
138-
TrustManagerFactory tmf = TrustManagerFactory.getInstance("PKIX");
136+
TrustManagerFactory tmf = TrustManagerFactory.getInstance(TrustManagerFactory.getDefaultAlgorithm());
139137
tmf.init(trustStore);
140138

141139
SSLContextBuilder sslContextBuilder = SSLContextBuilder.create().setProtocol(getProtocol()).setSecureRandom(secureRandom);

‎client/rest/src/test/java/org/opensearch/client/RestClientFipsAwareTestCase.java‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
/*
2+
* SPDX-License-Identifier: Apache-2.0
3+
*
4+
* The OpenSearch Contributors require contributions made to
5+
* this file be licensed under the Apache-2.0 license or a
6+
* compatible open source license.
7+
*/
8+
9+
package org.opensearch.client;
10+
11+
import javax.net.ssl.SSLContext;
12+
13+
import java.security.SecureRandom;
14+
15+
import static org.opensearch.client.RestClientTestCase.inFipsJvm;
16+
17+
interface RestClientFipsAwareTestCase {
18+
19+
default SSLContext getSslContext(boolean server) throws Exception {
20+
if (inFipsJvm()) {
21+
return getSslContext(server, "BCFKS", SecureRandom.getInstance("DEFAULT", "BCFIPS"), ".bcfks");
22+
}
23+
return getSslContext(server, "JKS", new SecureRandom(), ".jks");
24+
}
25+
26+
SSLContext getSslContext(boolean server, String keyStoreType, SecureRandom secureRandom, String fileExtension) throws Exception;
27+
}

‎client/rest/src/test/resources/test_truststore.bcfks‎

1.47 KB
Binary file not shown.

‎client/rest/src/test/resources/testks.bcfks‎

2.82 KB
Binary file not shown.

‎client/sniffer/build.gradle‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -29,6 +29,7 @@
2929
*/
3030
apply plugin: 'opensearch.build'
3131
apply plugin: 'opensearch.publish'
32+
apply from: "$rootDir/gradle/fips.gradle"
3233

3334
java {
3435
targetCompatibility = JavaVersion.VERSION_11
@@ -55,6 +56,9 @@ dependencies {
5556
testImplementation "org.objenesis:objenesis:${versions.objenesis}"
5657
testImplementation "net.bytebuddy:byte-buddy:${versions.bytebuddy}"
5758
testImplementation "net.bytebuddy:byte-buddy-agent:${versions.bytebuddy}"
59+
testFipsRuntimeOnly "org.bouncycastle:bc-fips:${versions.bouncycastle_jce}"
60+
testFipsRuntimeOnly "org.bouncycastle:bctls-fips:${versions.bouncycastle_tls}"
61+
testFipsRuntimeOnly "org.bouncycastle:bcutil-fips:${versions.bouncycastle_util}"
5862
}
5963

6064
tasks.named('forbiddenApisMain').configure {

‎distribution/tools/fips-demo-installer-cli/src/test/java/org/opensearch/tools/cli/fips/truststore/CreateFipsTrustStoreFipsTests.java‎

Lines changed: 96 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,96 @@
1+
/*
2+
* SPDX-License-Identifier: Apache-2.0
3+
*
4+
* The OpenSearch Contributors require contributions made to
5+
* this file be licensed under the Apache-2.0 license or a
6+
* compatible open source license.
7+
*/
8+
9+
package org.opensearch.tools.cli.fips.truststore;
10+
11+
import java.nio.file.Files;
12+
import java.nio.file.Path;
13+
import java.security.KeyStore;
14+
15+
public class CreateFipsTrustStoreFipsTests extends CreateFipsTrustStoreTests {
16+
17+
public void testConvertToBCFKS() throws Exception {
18+
// given
19+
KeyStore sourceKeyStore = CreateFipsTrustStore.loadJvmDefaultTrustStore(spec, JAVA_HOME);
20+
assertTrue("Source keystore should have certificates", sourceKeyStore.size() > 0);
21+
22+
CommonOptions options = new CommonOptions();
23+
options.force = false;
24+
String password = "testPassword123";
25+
26+
// when
27+
Path result = CreateFipsTrustStore.convertToBCFKS(spec, sourceKeyStore, options, password, confDir);
28+
29+
// then
30+
assertNotNull(result);
31+
assertTrue(Files.exists(result));
32+
assertTrue(result.toString().endsWith("opensearch-fips-truststore.bcfks"));
33+
34+
// Verify the converted keystore has the same certificates
35+
KeyStore bcfksStore = KeyStore.getInstance("BCFKS", "BCFIPS");
36+
try (var is = Files.newInputStream(result)) {
37+
bcfksStore.load(is, password.toCharArray());
38+
}
39+
assertEquals("Converted keystore should have same number of certificates", sourceKeyStore.size(), bcfksStore.size());
40+
}
41+
42+
public void testConvertToBCFKSFileExistsWithoutForce() throws Exception {
43+
// given
44+
KeyStore sourceKeyStore = CreateFipsTrustStore.loadJvmDefaultTrustStore(spec, JAVA_HOME);
45+
assertTrue("Source keystore should have certificates", sourceKeyStore.size() > 0);
46+
47+
CommonOptions options = new CommonOptions();
48+
options.force = false;
49+
String password = "testPassword123";
50+
51+
// Create file first to simulate existing truststore
52+
Path trustStorePath = confDir.resolve("opensearch-fips-truststore.bcfks");
53+
Files.createFile(trustStorePath);
54+
55+
assertTrue("Test setup: file should exist", Files.exists(trustStorePath));
56+
57+
// when/then
58+
RuntimeException exception = expectThrows(
59+
RuntimeException.class,
60+
() -> CreateFipsTrustStore.convertToBCFKS(spec, sourceKeyStore, options, password, confDir)
61+
);
62+
assertEquals("Operation cancelled. Trust store file already exists.", exception.getMessage());
63+
}
64+
65+
public void testConvertToBCFKSFileExistsWithForce() throws Exception {
66+
// given
67+
KeyStore sourceKeyStore = CreateFipsTrustStore.loadJvmDefaultTrustStore(spec, JAVA_HOME);
68+
assertTrue("Source keystore should have certificates", sourceKeyStore.size() > 0);
69+
70+
CommonOptions options = new CommonOptions();
71+
options.force = true;
72+
String password = "testPassword123";
73+
74+
// Create file first
75+
Path trustStorePath = confDir.resolve("opensearch-fips-truststore.bcfks");
76+
Files.createFile(trustStorePath);
77+
78+
assertTrue(Files.exists(trustStorePath));
79+
80+
// when
81+
Path result = CreateFipsTrustStore.convertToBCFKS(spec, sourceKeyStore, options, password, confDir);
82+
83+
// then
84+
assertNotNull(result);
85+
assertTrue(Files.exists(result));
86+
87+
// Verify the converted keystore has actual certificates
88+
KeyStore bcfksStore = KeyStore.getInstance("BCFKS", "BCFIPS");
89+
try (var is = Files.newInputStream(result)) {
90+
bcfksStore.load(is, password.toCharArray());
91+
}
92+
assertTrue("Converted keystore should have certificates", bcfksStore.size() > 0);
93+
assertEquals("Converted keystore should have same number of certificates", sourceKeyStore.size(), bcfksStore.size());
94+
}
95+
96+
}

‎distribution/tools/fips-demo-installer-cli/src/test/java/org/opensearch/tools/cli/fips/truststore/CreateFipsTrustStoreTests.java‎

Lines changed: 3 additions & 90 deletions
Original file line numberDiff line numberDiff line change
@@ -20,7 +20,6 @@
2020
import java.io.Writer;
2121
import java.nio.file.Files;
2222
import java.nio.file.Path;
23-
import java.security.KeyStore;
2423
import java.util.function.Consumer;
2524

2625
import picocli.CommandLine;
@@ -29,10 +28,10 @@ public class CreateFipsTrustStoreTests extends OpenSearchTestCase {
2928
@ClassRule
3029
public static TemporaryFolder tempFolder = new TemporaryFolder();
3130

32-
private static final Path JAVA_HOME = Path.of(System.getProperty("java.home"));
33-
private static Path confDir;
31+
protected static final Path JAVA_HOME = Path.of(System.getProperty("java.home"));
32+
protected static Path confDir;
3433

35-
private CommandLine.Model.CommandSpec spec;
34+
protected CommandLine.Model.CommandSpec spec;
3635

3736
@BeforeClass
3837
@SuppressForbidden(reason = "the java.io.File is exposed by TemporaryFolder")
@@ -126,90 +125,4 @@ public void testConfigureBCFKSTrustStore() {
126125
assertEquals("BCFIPS", config.trustStoreProvider());
127126
}
128127

129-
public void testConvertToBCFKS() throws Exception {
130-
assumeTrue("Should only run when BCFIPS provider is installed.", inFipsJvm());
131-
132-
// given
133-
KeyStore sourceKeyStore = CreateFipsTrustStore.loadJvmDefaultTrustStore(spec, JAVA_HOME);
134-
assertTrue("Source keystore should have certificates", sourceKeyStore.size() > 0);
135-
136-
CommonOptions options = new CommonOptions();
137-
options.force = false;
138-
String password = "testPassword123";
139-
140-
// when
141-
Path result = CreateFipsTrustStore.convertToBCFKS(spec, sourceKeyStore, options, password, confDir);
142-
143-
// then
144-
assertNotNull(result);
145-
assertTrue(Files.exists(result));
146-
assertTrue(result.toString().endsWith("opensearch-fips-truststore.bcfks"));
147-
148-
// Verify the converted keystore has the same certificates
149-
KeyStore bcfksStore = KeyStore.getInstance("BCFKS", "BCFIPS");
150-
try (var is = Files.newInputStream(result)) {
151-
bcfksStore.load(is, password.toCharArray());
152-
}
153-
assertEquals("Converted keystore should have same number of certificates", sourceKeyStore.size(), bcfksStore.size());
154-
}
155-
156-
public void testConvertToBCFKSFileExistsWithoutForce() throws Exception {
157-
// Skip if BCFIPS not available since the method needs it to check file handling
158-
assumeTrue("Should only run when BCFIPS provider is installed.", inFipsJvm());
159-
160-
// given
161-
KeyStore sourceKeyStore = CreateFipsTrustStore.loadJvmDefaultTrustStore(spec, JAVA_HOME);
162-
assertTrue("Source keystore should have certificates", sourceKeyStore.size() > 0);
163-
164-
CommonOptions options = new CommonOptions();
165-
options.force = false;
166-
String password = "testPassword123";
167-
168-
// Create file first to simulate existing truststore
169-
Path trustStorePath = confDir.resolve("opensearch-fips-truststore.bcfks");
170-
Files.createFile(trustStorePath);
171-
172-
assertTrue("Test setup: file should exist", Files.exists(trustStorePath));
173-
174-
// when/then
175-
RuntimeException exception = expectThrows(
176-
RuntimeException.class,
177-
() -> CreateFipsTrustStore.convertToBCFKS(spec, sourceKeyStore, options, password, confDir)
178-
);
179-
assertEquals("Operation cancelled. Trust store file already exists.", exception.getMessage());
180-
}
181-
182-
public void testConvertToBCFKSFileExistsWithForce() throws Exception {
183-
assumeTrue("Should only run when BCFIPS provider is installed.", inFipsJvm());
184-
185-
// given
186-
KeyStore sourceKeyStore = CreateFipsTrustStore.loadJvmDefaultTrustStore(spec, JAVA_HOME);
187-
assertTrue("Source keystore should have certificates", sourceKeyStore.size() > 0);
188-
189-
CommonOptions options = new CommonOptions();
190-
options.force = true;
191-
String password = "testPassword123";
192-
193-
// Create file first
194-
Path trustStorePath = confDir.resolve("opensearch-fips-truststore.bcfks");
195-
Files.createFile(trustStorePath);
196-
197-
assertTrue(Files.exists(trustStorePath));
198-
199-
// when
200-
Path result = CreateFipsTrustStore.convertToBCFKS(spec, sourceKeyStore, options, password, confDir);
201-
202-
// then
203-
assertNotNull(result);
204-
assertTrue(Files.exists(result));
205-
206-
// Verify the converted keystore has actual certificates
207-
KeyStore bcfksStore = KeyStore.getInstance("BCFKS", "BCFIPS");
208-
try (var is = Files.newInputStream(result)) {
209-
bcfksStore.load(is, password.toCharArray());
210-
}
211-
assertTrue("Converted keystore should have certificates", bcfksStore.size() > 0);
212-
assertEquals("Converted keystore should have same number of certificates", sourceKeyStore.size(), bcfksStore.size());
213-
}
214-
215128
}

0 commit comments

Comments
 (0)