org.apache.commons.io.IOUtils included in pax-web-spi-8.0.34.jar broke liquibase in tests

[steinarb]

platform: debian 13.3 "trixie", amd64, java 21.0.4, liquibase 4.33

I use the karaf BOM in my projects to align my dependencies with karaf's dependencies.

When bumping the karaf version in my pom from 4.4.8 to 4.4.9 got the below stack trace: applying a liquibase change list crashes inside a commons-io class way down in the stack trace.

The crash in the tests happened in this project: https://github.com/steinarb/sonar-collector

The reason for the stack trace is that org.apache.commons.io.input.BOMInputStream.read() in commons-io-2.21.0.jar finds org.apache.commons.io.IOUtils in pax-web-spi-8.0.34.jar instead of the same jar it itself resides in (i.e commons-io-2.21.0.jar) which was kind of weird.

Not sure what this means for the runtime, but I haven't seen this errors in my applictions, built with the karaf 4.4.8 BOM, running in karaf 4.4.9 and all using liquibase change sets.

Here is the stack trace:

java.lang.NoSuchMethodError: 'void org.apache.commons.io.IOUtils.checkFromIndexSize(byte[], int, int)'
	at org.apache.commons.io.input.BOMInputStream.read(BOMInputStream.java:453)
	at java.xml/com.sun.org.apache.xerces.internal.impl.XMLEntityManager$RewindableInputStream.read(XMLEntityManager.java:3065)
	at java.xml/com.sun.org.apache.xerces.internal.impl.io.UTF8Reader.read(UTF8Reader.java:303)
	at java.xml/com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.load(XMLEntityScanner.java:1699)
	at java.xml/com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.arrangeCapacity(XMLEntityScanner.java:1565)
	at java.xml/com.sun.org.apache.xerces.internal.impl.XMLEntityScanner.skipString(XMLEntityScanner.java:1603)
	at java.xml/com.sun.org.apache.xerces.internal.impl.XMLVersionDetector.determineDocVersion(XMLVersionDetector.java:158)
	at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:861)
	at java.xml/com.sun.org.apache.xerces.internal.parsers.XML11Configuration.parse(XML11Configuration.java:825)
	at java.xml/com.sun.org.apache.xerces.internal.parsers.XMLParser.parse(XMLParser.java:141)
	at java.xml/com.sun.org.apache.xerces.internal.parsers.AbstractSAXParser.parse(AbstractSAXParser.java:1224)
	at java.xml/com.sun.org.apache.xerces.internal.jaxp.SAXParserImpl$JAXPSAXParser.parse(SAXParserImpl.java:637)
	at liquibase.parser.core.xml.XMLChangeLogSAXParser.parseToNode(XMLChangeLogSAXParser.java:130)
	at liquibase.parser.core.xml.AbstractChangeLogParser.parse(AbstractChangeLogParser.java:15)
	at liquibase.command.core.helpers.DatabaseChangelogCommandStep.lambda$getDatabaseChangeLog$0(DatabaseChangelogCommandStep.java:143)
	at liquibase.Scope.lambda$child$0(Scope.java:216)
	at liquibase.Scope.child(Scope.java:225)
	at liquibase.Scope.child(Scope.java:215)
	at liquibase.Scope.child(Scope.java:194)
	at liquibase.Scope.child(Scope.java:282)
	at liquibase.Scope.child(Scope.java:286)
	at liquibase.command.core.helpers.DatabaseChangelogCommandStep.getDatabaseChangeLog(DatabaseChangelogCommandStep.java:138)
	at liquibase.command.core.helpers.DatabaseChangelogCommandStep.run(DatabaseChangelogCommandStep.java:84)
	at liquibase.command.CommandScope.lambda$execute$6(CommandScope.java:263)
	at liquibase.Scope.child(Scope.java:225)
	at liquibase.Scope.child(Scope.java:201)
	at liquibase.command.CommandScope.execute(CommandScope.java:251)
	at no.priv.bang.karaf.liquibase.runner.LiquibaseClassPathChangeLogRunner.lambda$applyLiquibaseChangelist$0(LiquibaseClassPathChangeLogRunner.java:56)
	at liquibase.Scope.child(Scope.java:225)
	at liquibase.Scope.child(Scope.java:201)
	at no.priv.bang.karaf.liquibase.runner.LiquibaseClassPathChangeLogRunner.applyLiquibaseChangelist(LiquibaseClassPathChangeLogRunner.java:53)
	at no.priv.bang.sonar.collector.webhook.SonarCollectorServlet.createSchemaWithLiquibase(SonarCollectorServlet.java:92)
	at no.priv.bang.sonar.collector.webhook.SonarCollectorServlet.activate(SonarCollectorServlet.java:86)
	at no.priv.bang.sonar.collector.webhook.SonarCollectorServletTest.testMeasuresView(SonarCollectorServletTest.java:265)

[grgrzybek]

This may be a hashmap issues (unordered map of bundles/jars checked), but that's only a hypothesis.

The error is not looking good, but ... some packages of commons-io are used inside pax-web-spi since the refactoring to 8.0.0: https://github.com/ops4j/org.ops4j.pax.web/blob/13b83d4143ea2b857539921147ad213bf507e78b/pax-web-spi/pom.xml

I added an explicit declaration:

<Private-Package>
	<!-- supporting libraries -->
	org.apache.commons.io,
	org.apache.commons.collections4,
	org.apache.commons.collections4.iterators,
	org.apache.commons.collections4.map,
	org.apache.felix.utils.properties
</Private-Package>

These are private, because I didn't want Pax Web SPI library to depend on too many external dependencies, but I needed org.apache.commons.io.FilenameUtils. Sure - I can remove this class and just copy/invent the required code without such dependency. I also needed commons-collections, so I didn't think too much about this.

Now I see that even org.apache.commons.collections4 is not needed, so I created:

• Remove Commons (lang, collections, IO) dependencies #2132

The point is that your stack trace doesn't show any actual Pax Web code and starts with no.priv.bang.sonar.collector.webhook.SonarCollectorServletTest. So looks like you're running under non-OSGi classloader, because Private-Package-d packages should NEVER been available to liquibase.parser.core.xml.XMLChangeLogSAXParser.parseToNode in the first place.

[steinarb]

>>>> Grzegorz Grzybek ***@***.***>: These are _private_, because I didn't want Pax Web SPI library to depend on too many external dependencies, but I needed `org.apache.commons.io.FilenameUtils`. Sure - I can remove this class and just copy/invent the required code without such dependency. I also needed commons-collections, so I didn't think too much about this.

Ah! This means there (probably) will not be a run-time issue. And I haven't seen any issues when running the applications built with the karaf 4.4.8 BOM in karaf 4.4.9. But I will have to test after building everything with the 4.4.9 BOM, to be certain.

Now I see that even `org.apache.commons.collections4` is not needed, so I created: * #2132

+1

The point is that your stack trace doesn't show any actual Pax Web code and starts with `no.priv.bang.sonar.collector.webhook.SonarCollectorServletTest`. So looks like you're running under non-OSGi classloader, because `Private-Package`-d packages should NEVER been available to `liquibase.parser.core.xml.XMLChangeLogSAXParser.parseToNode` in the first place.

Yes, this was in a JUnit 5 test. I was able to work around the test stack trace with this maven dependency: <dependency> <groupId>commons-io</groupId> <artifactId>commons-io</artifactId> <scope>test</scope> </dependency> (but that's just for test so I worried about the run-time, but if the inclusion is private then there should be no issue under OSGi... I hope...?)

[grgrzybek]

If you really use OSGi (and Pax Web proves you do ;), then you can't avoid at least some real smoke OSGi tests - I always add a tests like "Clean Integration Test", which doesn't test anything but checking the infra. See: https://github.com/ops4j/org.ops4j.pax.web/blob/pax-web-11.x/pax-web-itest/pax-web-itest-osgi/src/test/java/org/ops4j/pax/web/itest/osgi/CleanIntegrationTest.java

[steinarb]

Here is the workaround: steinarb/sonar-collector@f1dcfc5

I have to repeat this in all of the projects using test databases, that also has this dependency:

        <dependency>
            <groupId>org.ops4j.pax.web</groupId>
            <artifactId>pax-web-runtime</artifactId>
            <scope>provided</scope>
        </dependency>

(and the commons-io dependency has to come before the pax-web-runtime dependency in <dependencies>)

All of the projects loaded into karaf, except for sonar-collector, which is just an old, simple thing, from 2017 or 2018 or something, have an osgi smoke test, like e.g. this one: https://github.com/steinarb/sampleapp/blob/master/sampleapp.tests/src/test/java/no/priv/bang/sampleapp/tests/SampleappIntegrationTest.java#L31

Most of the smoke tests are just like the one above, but in one I actually run code (the test for my liquibase karaf feature. I test that the liquibase loaded from the feature is actually able to create a database schema that can be used. This is because liquibase is kinda tricky since it uses SPI as its plugin mechanism and I have to make sure Apache Aries SPI fly run right): https://github.com/steinarb/liquibase-karaf-feature/blob/master/liquibase-integration-test/karaf.liquibase.tests/src/test/java/no/priv/bang/karaf/liquibase/tests/LiquibaseKarafFeatureIntegrationTest.java#L36

[steinarb]

Just confirming that I saw no runtime issues (no liquibase errors in starting the apps, which all run a liquibase changelist, and no stack traces in karaf.log) when running in karaf 4.4.9 after building all apps against the karaf 4.4.9 BOM.

This is as expected, but it is always better to confirm than to assume.

(and also: no OSGi smoke tests failed when building with the 4.4.9 BOM)

[grgrzybek]

Thanks - I can close this one, but will fix #2132 anyway.