[improve] Add jdbc common collect e2e code (#3273) #2
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,72 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| package org.apache.hertzbeat.collector.collect.basic.database; | ||
|
|
||
| /** | ||
| * Database type (image mame) and image tag enumeration class | ||
| */ | ||
| public enum DatabaseImagesEnum { | ||
| MYSQL("mysql", "8.0.36"), | ||
| POSTGRESQL("postgresql", "15"); | ||
|
|
||
| private final String imageName; | ||
| private final String defaultTag; | ||
|
|
||
| DatabaseImagesEnum(String imageName, String defaultTag) { | ||
| this.imageName = imageName; | ||
| this.defaultTag = defaultTag; | ||
| } | ||
|
|
||
| public String getImageName() { | ||
| return imageName; | ||
| } | ||
|
|
||
| public String getDefaultTag() { | ||
| return defaultTag; | ||
| } | ||
|
|
||
| public String getFullImageName() { | ||
| return imageName + ":" + defaultTag; | ||
| } | ||
|
|
||
| public static DatabaseImagesEnum fromImageName(String imageName) { | ||
| for (DatabaseImagesEnum value : values()) { | ||
| if (value.getImageName().equalsIgnoreCase(imageName)) { | ||
| return value; | ||
| } | ||
| } | ||
|
Comment on lines
+45
to
+52
There was a problem hiding this comment. Missing Error Handling in TestContainers Database Type ResolutionThe DatabaseImagesEnum.fromImageName method throws an IllegalArgumentException when an unknown database image name is provided, but there's no error handling in the JdbcCommonCollectE2eTest that uses this method with a hardcoded value. If the database type is changed or removed in the future, tests will fail with an uncaught exception rather than gracefully handling the error. This violates error handling best practices and could lead to difficult-to-diagnose test failures. public static DatabaseImagesEnum fromImageName(String imageName) {
if (imageName == null || imageName.trim().isEmpty()) {
return POSTGRESQL; // Default to PostgreSQL if no name provided
}
for (DatabaseImagesEnum value : values()) {
if (value.getImageName().equalsIgnoreCase(imageName)) {
return value;
}
}
throw new IllegalArgumentException("Unknown database image name: " + imageName);
}ReferencesStandard: Design by Contract - Precondition Validation |
||
| throw new IllegalArgumentException("Unknown database image name: " + imageName); | ||
| } | ||
|
|
||
| public static DatabaseImagesEnum fromFullImageName(String fullImageName) { | ||
| for (DatabaseImagesEnum value : values()) { | ||
| if (value.getFullImageName().equalsIgnoreCase(fullImageName)) { | ||
| return value; | ||
| } | ||
| } | ||
| throw new IllegalArgumentException("Unknown full database image name: " + fullImageName); | ||
| } | ||
|
|
||
| @Override | ||
| public String toString() { | ||
| return "DatabaseImageNameEnum{" + | ||
| "imageName='" + imageName + '\'' + | ||
| ", defaultTag='" + defaultTag + '\'' + | ||
| '}'; | ||
| } | ||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,99 @@ | ||
| /* | ||
| * Licensed to the Apache Software Foundation (ASF) under one or more | ||
| * contributor license agreements. See the NOTICE file distributed with | ||
| * this work for additional information regarding copyright ownership. | ||
| * The ASF licenses this file to You under the Apache License, Version 2.0 | ||
| * (the "License"); you may not use this file except in compliance with | ||
| * the License. You may obtain a copy of the License at | ||
| * | ||
| * http://www.apache.org/licenses/LICENSE-2.0 | ||
| * | ||
| * Unless required by applicable law or agreed to in writing, software | ||
| * distributed under the License is distributed on an "AS IS" BASIS, | ||
| * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. | ||
| * See the License for the specific language governing permissions and | ||
| * limitations under the License. | ||
| */ | ||
|
|
||
| package org.apache.hertzbeat.collector.collect.basic.database; | ||
|
|
||
| import lombok.extern.slf4j.Slf4j; | ||
| import org.apache.hertzbeat.collector.collect.AbstractCollectE2eTest; | ||
| import org.apache.hertzbeat.collector.collect.database.JdbcCommonCollect; | ||
| import org.apache.hertzbeat.collector.util.CollectUtil; | ||
| import org.apache.hertzbeat.common.entity.job.Configmap; | ||
| import org.apache.hertzbeat.common.entity.job.Job; | ||
| import org.apache.hertzbeat.common.entity.job.Metrics; | ||
| import org.apache.hertzbeat.common.entity.job.protocol.JdbcProtocol; | ||
| import org.apache.hertzbeat.common.entity.job.protocol.Protocol; | ||
| import org.apache.hertzbeat.common.entity.message.CollectRep; | ||
| import org.junit.jupiter.api.BeforeEach; | ||
| import org.junit.jupiter.api.Test; | ||
| import org.junit.jupiter.api.extension.ExtendWith; | ||
| import org.mockito.junit.jupiter.MockitoExtension; | ||
|
|
||
| import java.util.HashMap; | ||
| import java.util.LinkedList; | ||
| import java.util.List; | ||
| import java.util.Map; | ||
|
|
||
| import static org.junit.jupiter.api.Assertions.assertDoesNotThrow; | ||
|
|
||
| /** | ||
| * E2e test monitored by Jdbc Common | ||
|
Comment on lines
+38
to
+43
There was a problem hiding this comment. Missing Container Cleanup in JdbcCommonCollectE2eTestThe test class uses TestContainers for database testing but doesn't properly manage container lifecycle. TestContainers typically requires explicit container initialization and cleanup. Without proper container management, tests may leave containers running after completion, causing resource leaks, potential port conflicts, and increased resource consumption during test execution. This violates resource management best practices and could lead to unstable test environments. @Testcontainers
public class JdbcCommonCollectE2eTest extends AbstractCollectE2eTest {
private static final DatabaseImagesEnum databaseImage = DatabaseImagesEnum.fromImageName("postgresql");
private static final String DATABASE_IMAGE_NAME = databaseImage.getImageName();
private static final String DATABASE_IMAGE_TAG = databaseImage.getDefaultTag();
@Container
private static final PostgreSQLContainer<?> postgresContainer = new PostgreSQLContainer<>(databaseImage.getFullImageName())
.withDatabaseName("test")
.withUsername("root")
.withPassword("root");
ReferencesStandard: ISO/IEC 25010 Reliability - Resource Utilization |
||
| */ | ||
| @Slf4j | ||
| @ExtendWith(MockitoExtension.class) | ||
| public class JdbcCommonCollectE2eTest extends AbstractCollectE2eTest { | ||
| private static final DatabaseImagesEnum databaseImage = DatabaseImagesEnum.fromImageName("postgresql"); | ||
arvi18 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| private static final String DATABASE_IMAGE_NAME = databaseImage.getImageName(); | ||
arvi18 marked this conversation as resolved.
Show resolved
Hide resolved
arvi18 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
|
|
||
| private static final String DATABASE_IMAGE_TAG = databaseImage.getDefaultTag(); | ||
|
|
||
| @BeforeEach | ||
| public void setUp() throws Exception { | ||
| super.setUp(); | ||
| collect = new JdbcCommonCollect(); | ||
| metrics = new Metrics(); | ||
| } | ||
|
|
||
| @Override | ||
| protected CollectRep.MetricsData.Builder collectMetrics(Metrics metricsDef) { | ||
| JdbcProtocol jdbcProtocol = (JdbcProtocol) buildProtocol(metricsDef); | ||
| metrics.setJdbc(jdbcProtocol); | ||
| CollectRep.MetricsData.Builder metricsData = CollectRep.MetricsData.newBuilder(); | ||
| metricsData.setApp(DATABASE_IMAGE_NAME); | ||
| metrics.setAliasFields(metricsDef.getAliasFields()); | ||
| return collectMetricsData(metrics, metricsDef, metricsData); | ||
| } | ||
|
|
||
| @Override | ||
| protected Protocol buildProtocol(Metrics metricsDef) { | ||
| JdbcProtocol jdbcProtocol = metricsDef.getJdbc(); | ||
| jdbcProtocol.setHost(DATABASE_IMAGE_NAME); | ||
| jdbcProtocol.setPort(DATABASE_IMAGE_TAG); | ||
| jdbcProtocol.setDatabase("test"); | ||
| jdbcProtocol.setPlatform("testcontainers"); | ||
| return jdbcProtocol; | ||
| } | ||
|
Comment on lines
+71
to
+79
There was a problem hiding this comment. 💡 Verification agent 🧩 Analysis chainVerify if port assignment is correct Line 75 assigns 🏁 Script executed: #!/bin/bash
# Check how DATABASE_IMAGE_TAG is used and if it's appropriate for port configuration
echo "Checking DatabaseImagesEnum class for tag definition..."
fd -t f -e java "DatabaseImagesEnum.java" | xargs cat
echo "Checking JdbcProtocol class for port field type..."
fd -t f -e java "JdbcProtocol.java" | xargs cat | grep -A 10 "private.*port"Length of output: 2986 🏁 Script executed: #!/bin/bash
# Display the relevant test file context around the buildProtocol method
fd -t f -e java "JdbcCommonCollectE2eTest.java" | xargs sed -n '1,200p'Length of output: 4427 🏁 Script executed: #!/bin/bash
# Locate and display AbstractCollectE2eTest class implementation context around container setup or protocol handling
fd -t f -e java "AbstractCollectE2eTest.java" | xargs sed -n '1,200p'Length of output: 6135 Confirm port assignment in buildProtocol The test currently does: jdbcProtocol.setHost(DATABASE_IMAGE_NAME);
jdbcProtocol.setPort(DATABASE_IMAGE_TAG);Here, Please verify and update accordingly:
Location:
|
||
|
|
||
| @Test | ||
| public void testWithJdbcTcUrl() { | ||
| Job dockerJob = appService.getAppDefine(DATABASE_IMAGE_NAME); | ||
| List<Map<String, Configmap>> configmapFromPreCollectData = new LinkedList<>(); | ||
| for (Metrics metricsDef : dockerJob.getMetrics()) { | ||
| metricsDef = CollectUtil.replaceCryPlaceholderToMetrics(metricsDef, configmapFromPreCollectData.size() > 0 ? configmapFromPreCollectData.get(0) : new HashMap<>()); | ||
| String metricName = metricsDef.getName(); | ||
| if ("slow_sql".equals(metricName)) { | ||
| Metrics finalMetricsDef = metricsDef; | ||
| assertDoesNotThrow(() -> collectMetrics(finalMetricsDef), | ||
| String.format("%s failed to collect metrics)", metricName)); | ||
| log.info("{} metrics validation passed", metricName); | ||
| continue; // skip slow_sql, extra extensions | ||
| } | ||
| CollectRep.MetricsData metricsData = validateMetricsCollection(metricsDef, metricName, true); | ||
| CollectUtil.getConfigmapFromPreCollectData(metricsData); | ||
| } | ||
|
Comment on lines
+95
to
+97
There was a problem hiding this comment. 🛠️ Refactor suggestion The return value of getConfigmapFromPreCollectData is not used Line 96 calls -CollectUtil.getConfigmapFromPreCollectData(metricsData);
+// Extract configmap data for potential use in subsequent test iterations
+Map<String, Configmap> configmap = CollectUtil.getConfigmapFromPreCollectData(metricsData);
+if (!configmap.isEmpty()) {
+ configmapFromPreCollectData.add(configmap);
+}
|
||
| } | ||
|
Comment on lines
+91
to
+98
There was a problem hiding this comment. Missing Error Handling in Database Test ImplementationThe test method handles the 'slow_sql' metric case differently but doesn't provide sufficient error handling or logging for diagnostic purposes. While the code uses assertDoesNotThrow() to verify execution, it doesn't capture or validate the actual result of the metrics collection. This makes it harder to diagnose test failures and understand what specifically went wrong with slow_sql metric collection. if ("slow_sql".equals(metricName)) {
Metrics finalMetricsDef = metricsDef;
try {
CollectRep.MetricsData metricsData = collectMetrics(finalMetricsDef).build();
assertNotNull(metricsData, String.format("%s metrics data should not be null", metricName));
log.info("{} metrics validation passed with {} rows", metricName,
metricsData.getValuesCount() > 0 ? metricsData.getValuesCount() : "no");
} catch (Exception e) {
log.error("Error collecting {} metrics: {}", metricName, e.getMessage());
fail(String.format("%s metrics collection failed: %s", metricName, e.getMessage()));
}
continue; // skip slow_sql, extra extensions
}ReferencesStandard: JUnit Best Practices - Error Diagnostics, Clean Code - Error Handling |
||
| } | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Hardcoded Database Credentials in JDBC URL
The code adds a new case for testcontainers that includes hardcoded database credentials ('root'/'root') directly in the JDBC URL. While this is only used for testing purposes, hardcoded credentials in source code are generally considered a security risk as they could be inadvertently exposed or reused in non-test environments.
case "testcontainers" -> "jdbc:tc:" + host + ":" + port + "://" + (jdbcProtocol.getDatabase() == null ? "" : jdbcProtocol.getDatabase()) + "?user=" + (jdbcProtocol.getUsername() == null ? "root" : jdbcProtocol.getUsername()) + "&password=" + (jdbcProtocol.getPassword() == null ? "root" : jdbcProtocol.getPassword());References
Standard: CWE-798
Standard: OWASP Top 10 2021: A07 - Identification and Authentication Failures