Skip to content

Update Ameria to 1.34.1 #6343

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
JannikBrand wants to merge 3 commits into
base: main
Choose a base branch
from
+10 −9
Draft

Update Ameria to 1.34.1 #6343

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
c2006cb
Update Ameria to 1.34.2
JannikBrand Dec 9, 2025
c4c1d21
Update awssdk to 2.41.5 and hadoop to 3.4.2
JannikBrand Dec 17, 2025
0f2c6de
Fix Mockito strict stubbing in DynamoDbClientWrapperTest
JannikBrand Jan 14, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
10 changes: 5 additions & 5 deletions build.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,11 +230,11 @@ subprojects {
resolutionStrategy.eachDependency { def details ->
if (details.requested.group == 'io.netty') {
if (details.requested.name == 'netty') {
details.useTarget group: 'io.netty', name: 'netty-all', version: '4.1.124.Final'
details.because 'Fixes CVE-2025-24970, CVE-2022-41881, CVE-2021-21290 and CVE-2022-41915.'
details.useTarget group: 'io.netty', name: 'netty-all', version: '4.2.7.Final'
details.because 'Required for Armeria 1.34.2 compatibility (needs Netty 4.2.x APIs).'
} else if (!details.requested.name.startsWith('netty-tcnative')) {
details.useVersion '4.1.125.Final'
details.because 'Fixes CVE-2025-58057, CVE-2025-58056, CVE-2025-55163, CVE-2025-24970, CVE-2022-41881, CVE-2021-21290 and CVE-2022-41915.'
details.useVersion '4.2.7.Final'
Copy link
Collaborator

@KarstenSchnitter KarstenSchnitter Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

netty dependencies are also contained in the AWS SDK. The currently used version in Data Prepper uses netty 4.1.118.Final. I am not sure what the latest SDK versions use.

Copy link
Member

@dlvenable dlvenable Dec 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I normally run this command to see what the different projects are using:

./gradlew -p data-prepper-core dependencies > deps.core.out

It looks like there are a few possible issues with migrating:

https://netty.io/wiki/netty-4.2-migration-guide.html

It states this:

It is important to ensure that all the components of your system is using the same Netty version, and are all configuring endpoint validation correctly. First off, this avoids any surprises from older client libraries that configure endpoint validation incorrectly, and may as a result stop working after a Netty 4.2 upgrade. Secondly, Netty 4.1 and 4.2 cannot co-exist on the class path, so you need to ensure you do the upgrade in a single step.

So we should check all the dependencies that use Netty to see if they have migrated to 4.2.

Copy link
Contributor Author

@JannikBrand JannikBrand Dec 17, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@dlvenable Thanks for the pointers!

Regarding the (forced) netty version update from 4.1.118.Final to 4.2.7.Final:
I checked the dependencies and saw that the current versions of the awssdk and hadoop are still using netty 4.1.x:

  • software.amazon.awssdk:netty-nio-client => 4.1.118.Final
  • org.apache.hadoop:hadoop-common:3.4.0 => 4.1.100.Final
  • org.apache.hadoop:hadoop-mapreduce-client-core:3.4.0 => 4.1.100.Final

Note that hadoop isn't even compatible with the current netty version 4.1.118.Final (but only the patch version).

I updated them to the latest version (awssdk 2.30.23 -> 2.41.5; hadoop 3.4.0 -> 3.4.2) and could reduce the version gap to netty 4.2.7.Final a little bit:

  • software.amazon.awssdk:netty-nio-client => 4.1.130.Final
  • org.apache.hadoop:hadoop-common => 4.1.118.Final
  • org.apache.hadoop:hadoop-mapreduce-client-core => 4.1.118.Final

My question is if we should still proceed with the current state of the PR and test it thoroughly, since the awssdk might not have officially validated netty 4.2 or if we should wait until both libraries officially support the netty version.

details.because 'Required for Armeria 1.34.2 compatibility (needs Netty 4.2.x APIs).'
}
} else if (details.requested.group == 'log4j' && details.requested.name == 'log4j') {
details.useTarget group: 'org.apache.logging.log4j', name: 'log4j-1.2-api', version: '2.17.1'
Expand Down Expand Up @@ -278,7 +278,7 @@ subprojects {

configure(subprojects.findAll {it.name != 'data-prepper-api'}) {
dependencies {
implementation platform('software.amazon.awssdk:bom:2.30.23')
implementation platform('software.amazon.awssdk:bom:2.41.5')
implementation 'jakarta.validation:jakarta.validation-api:3.0.2'
}
}
Expand Down
2 changes: 1 addition & 1 deletion ...per-core/src/main/java/org/opensearch/dataprepper/core/meter/EMFLoggingMeterRegistry.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,14 @@
import io.micrometer.core.instrument.step.StepMeterRegistry;
import io.micrometer.core.instrument.util.NamedThreadFactory;
import io.micrometer.core.instrument.util.StringUtils;
import io.micrometer.core.lang.Nullable;
import io.micrometer.core.util.internal.logging.WarnThenDebugLogger;
import software.amazon.cloudwatchlogs.emf.environment.Environment;
import software.amazon.cloudwatchlogs.emf.environment.EnvironmentProvider;
import software.amazon.cloudwatchlogs.emf.logger.MetricsLogger;
import software.amazon.cloudwatchlogs.emf.model.DimensionSet;
import software.amazon.cloudwatchlogs.emf.model.Unit;

import javax.annotation.Nullable;
import java.time.Instant;
import java.util.Collections;
import java.util.HashMap;
Expand Down
3 changes: 2 additions & 1 deletion .../opensearch/dataprepper/plugins/sourcecoordinator/dynamodb/DynamoDbClientWrapperTest.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -70,6 +70,7 @@
import static org.mockito.BDDMockito.given;
import static org.mockito.Mockito.doNothing;
import static org.mockito.Mockito.doThrow;
import static org.mockito.Mockito.lenient;
import static org.mockito.Mockito.mock;
import static org.mockito.Mockito.mockStatic;
import static org.mockito.Mockito.never;
Expand Down Expand Up @@ -842,7 +843,7 @@ void tryDeletePartition_item_success_calls_delete_on_correct_item() throws NoSuc
@Test
void tryDeletePartition_with_exception_throws_PartitionUpdateException() throws NoSuchFieldException, IllegalAccessException {
final DynamoDbTable<DynamoDbSourcePartitionItem> table = mock(DynamoDbTable.class);
when(table.deleteItem(any(DeleteItemEnhancedRequest.class))).thenThrow(RuntimeException.class);
lenient().when(table.deleteItem(any(DeleteItemEnhancedRequest.class))).thenThrow(RuntimeException.class);

final DynamoDbClientWrapper objectUnderTest = createObjectUnderTest();

Expand Down
4 changes: 2 additions & 2 deletions settings.gradle
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,7 @@ dependencyResolutionManagement {
libs {
version('slf4j', '2.0.6')
library('slf4j-api', 'org.slf4j', 'slf4j-api').versionRef('slf4j')
version('armeria', '1.32.5')
version('armeria', '1.34.2')
library('armeria-core', 'com.linecorp.armeria', 'armeria').versionRef('armeria')
library('armeria-grpc', 'com.linecorp.armeria', 'armeria-grpc').versionRef('armeria')
library('armeria-junit', 'com.linecorp.armeria', 'armeria-junit5').versionRef('armeria')
Expand Down Expand Up @@ -66,7 +66,7 @@ dependencyResolutionManagement {
library('parquet-avro', 'org.apache.parquet', 'parquet-avro').versionRef('parquet')
library('parquet-column', 'org.apache.parquet', 'parquet-column').versionRef('parquet')
library('parquet-hadoop', 'org.apache.parquet', 'parquet-hadoop').versionRef('parquet')
version('hadoop', '3.4.0')
version('hadoop', '3.4.2')
library('hadoop-common', 'org.apache.hadoop', 'hadoop-common').versionRef('hadoop')
library('hadoop-mapreduce', 'org.apache.hadoop', 'hadoop-mapreduce-client-core').versionRef('hadoop')
version('avro', '1.11.4')
Expand Down
Loading