Skip to content

Commit f119d07

Browse files
authored
Fix potential jetcd-core shading problem (#4526)
### Motivation There is a potential jar shading issue introduced in #4426 that causes `NoClassDefFoundError` when connecting to an etcd metadata store. The `jetcd-core-shaded` module was introduced in #4426 to address the compatibility issues between jetcd-core’s grpc-java dependency and Netty. You can find more details [here][1] and in the [grpc-java documentation][2]. [1]: #4426 (comment) [2]: https://github.com/grpc/grpc-java/blob/master/SECURITY.md#netty Currently, we use `unpack-shaded-jar` execution unpacks the shaded jar produced by `maven-shade-plugin:shade` into the `jetcd-core-shaded/target/classes` directory. However, the classes in this directory conflict with its dependencies. If the `maven-shade-plugin:shade` runs again without cleaning this directory, it can produce an incorrect shaded jar. You can replicate and verify this issue with the following commands: ```shell # Step 1: Clean the build directory mvn clean # Step 2: Perform an install and unpack the shaded jar into a directory. # Verify the import statement for `io.netty.handler.logging.ByteBufFormat` in # `org/apache/pulsar/jetcd/shaded/io/vertx/core/net/NetClientOptions.class`. # The correct import should be: # `import io.grpc.netty.shaded.io.netty.handler.logging.ByteBufFormat;`. mvn install unzip $M2_REPO/org/apache/bookkeeper/metadata/drivers/jetcd-core-shaded/4.18.0-SNAPSHOT/jetcd-core-shaded-4.18.0-SNAPSHOT-shaded.jar \ -d metadata-drivers/jetcd-core-shaded/target/first-classes # Step 3: Run the install command again without cleaning. # The unpacked jar from the previous step will persist in `target/classes`. # Unpack the shaded jar into a different directory (e.g., second-classes) and check the import. # The incorrect import will be: # `import io.grpc.netty.shaded.io.grpc.netty.shaded.io.netty.handler.logging.ByteBufFormat;`. mvn install unzip $M2_REPO/org/apache/bookkeeper/metadata/drivers/jetcd-core-shaded/4.18.0-SNAPSHOT/jetcd-core-shaded-4.18.0-SNAPSHOT-shaded.jar \ -d metadata-drivers/jetcd-core-shaded/target/second-classes # Step 4: Use IntelliJ IDEA's "Compare Directories" tool to compare the `first-classes` # and `second-classes` directories. The differences in imports should become apparent. ``` We can remove the attach and unpack configurations, and it should work fine. This issue has already been [reported][3] in apache/pulsar, and a similar [patch][patch] has addressed it. [3]: apache/pulsar#23513 [patch]: apache/pulsar#23604
1 parent 0df2240 commit f119d07

File tree

3 files changed

+3
-50
lines changed

3 files changed

+3
-50
lines changed

.github/workflows/bk-ci.yml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -242,7 +242,9 @@ jobs:
242242
run: mvn -B -nsu clean install -Pdocker -DskipTests
243243

244244
- name: Run metadata driver tests
245-
run: mvn -B -nsu -f metadata-drivers/pom.xml test -DintegrationTests
245+
# Exclude jetcd-core-shaded from integration tests, as it’s a POM-only project used internally,
246+
# and maven prioritizes workspace artifacts during testing.
247+
run: mvn -B -nsu -f metadata-drivers/pom.xml -pl '!jetcd-core-shaded' test -DintegrationTests
246248

247249
- name: Run all integration tests (except backward compatibility tests)
248250
run: |

metadata-drivers/etcd/pom.xml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,7 +37,6 @@
3737
<groupId>org.apache.bookkeeper.metadata.drivers</groupId>
3838
<artifactId>jetcd-core-shaded</artifactId>
3939
<version>${project.version}</version>
40-
<classifier>shaded</classifier>
4140
<exclusions>
4241
<exclusion>
4342
<groupId>io.etcd</groupId>

metadata-drivers/jetcd-core-shaded/pom.xml

Lines changed: 0 additions & 48 deletions
Original file line numberDiff line numberDiff line change
@@ -132,54 +132,6 @@
132132
<file>${project.basedir}/dependency-reduced-pom.xml</file>
133133
</transformer>
134134
</transformers>
135-
<!-- required for IntelliJ support -->
136-
<shadedArtifactAttached>true</shadedArtifactAttached>
137-
<shadedClassifierName>shaded</shadedClassifierName>
138-
</configuration>
139-
</execution>
140-
</executions>
141-
</plugin>
142-
<!-- required for IntelliJ support, for some reason shadedArtifactAttached isn't sufficient alone -->
143-
<plugin>
144-
<groupId>org.codehaus.mojo</groupId>
145-
<artifactId>build-helper-maven-plugin</artifactId>
146-
<executions>
147-
<execution>
148-
<id>attach-shaded-jar</id>
149-
<phase>package</phase>
150-
<goals>
151-
<goal>attach-artifact</goal>
152-
</goals>
153-
<configuration>
154-
<artifacts>
155-
<artifact>
156-
<file>${project.build.directory}/${project.artifactId}-${project.version}-shaded.jar</file>
157-
<type>jar</type>
158-
<classifier>shaded</classifier>
159-
</artifact>
160-
</artifacts>
161-
</configuration>
162-
</execution>
163-
</executions>
164-
</plugin>
165-
<!-- required for running tests with "mvn -f metadata-drivers/pom.xml test -DintegrationTests" -->
166-
<plugin>
167-
<groupId>org.apache.maven.plugins</groupId>
168-
<artifactId>maven-antrun-plugin</artifactId>
169-
<version>${maven-antrun-plugin.version}</version>
170-
<executions>
171-
<execution>
172-
<id>unpack-shaded-jar</id>
173-
<phase>package</phase>
174-
<goals>
175-
<goal>run</goal>
176-
</goals>
177-
<configuration>
178-
<target>
179-
<unzip src="${project.build.directory}/${project.artifactId}-${project.version}-shaded.jar"
180-
dest="${project.build.outputDirectory}"
181-
overwrite="true" />
182-
</target>
183135
</configuration>
184136
</execution>
185137
</executions>

0 commit comments

Comments
 (0)