Don't shade dependencies

[Ribesg]

I don't think this project should shade all of its dependencies. As expected, it produces tons of warnings when used with SparkJava itself:

[WARNING] jetty-http-9.3.6.v20151106.jar, spark-debug-tools-0.5.jar define 53 overlapping classes: 
[WARNING]   - org.eclipse.jetty.http.HttpParser$HttpHandler
[WARNING]   - org.eclipse.jetty.http.BadMessageException
[WARNING]   - org.eclipse.jetty.http.HttpGenerator$PreparedResponse
[WARNING]   - org.eclipse.jetty.http.HttpParser$1
[WARNING]   - org.eclipse.jetty.http.HttpParser$IllegalCharacterException
[WARNING]   - org.eclipse.jetty.http.PathMap$PathSet
[WARNING]   - org.eclipse.jetty.http.HttpField$IntValueHttpField
[WARNING]   - org.eclipse.jetty.http.HttpVersion
[WARNING]   - org.eclipse.jetty.http.HttpFieldPreEncoder
[WARNING]   - org.eclipse.jetty.http.HttpParser$CharState
[WARNING]   - 43 more...
[WARNING] commons-io-2.1.jar, spark-debug-tools-0.5.jar define 104 overlapping classes: 
[WARNING]   - org.apache.commons.io.FileCleaningTracker
[WARNING]   - org.apache.commons.io.comparator.SizeFileComparator
[WARNING]   - org.apache.commons.io.input.CloseShieldInputStream
[WARNING]   - org.apache.commons.io.filefilter.EmptyFileFilter
[WARNING]   - org.apache.commons.io.monitor.FileEntry
[WARNING]   - org.apache.commons.io.output.ThresholdingOutputStream
[WARNING]   - org.apache.commons.io.input.TailerListener
[WARNING]   - org.apache.commons.io.IOExceptionWithCause
[WARNING]   - org.apache.commons.io.comparator.PathFileComparator
[WARNING]   - org.apache.commons.io.filefilter.NotFileFilter
[WARNING]   - 94 more...
[WARNING] spark-debug-tools-0.5.jar, javax.servlet-api-3.1.0.jar define 79 overlapping classes: 
[WARNING]   - javax.servlet.http.Cookie
[WARNING]   - javax.servlet.ServletContext
[WARNING]   - javax.servlet.Registration
[WARNING]   - javax.servlet.http.HttpSessionListener
[WARNING]   - javax.servlet.http.HttpSessionContext
[WARNING]   - javax.servlet.FilterChain
[WARNING]   - javax.servlet.http.WebConnection
[WARNING]   - javax.servlet.http.HttpServletRequestWrapper
[WARNING]   - javax.servlet.http.HttpSessionAttributeListener
[WARNING]   - javax.servlet.http.HttpSessionBindingListener
[WARNING]   - 69 more...
[WARNING] spark-debug-tools-0.5.jar, jetty-server-9.3.6.v20151106.jar define 217 overlapping classes: 
[WARNING]   - org.eclipse.jetty.server.handler.ManagedAttributeListener
[WARNING]   - org.eclipse.jetty.server.session.JDBCSessionIdManager$SessionTableSchema
[WARNING]   - org.eclipse.jetty.server.ServletRequestHttpWrapper
[WARNING]   - org.eclipse.jetty.server.session.HashSessionManager$Saver
[WARNING]   - org.eclipse.jetty.server.ShutdownMonitor$1
[WARNING]   - org.eclipse.jetty.server.HttpInput$1
[WARNING]   - org.eclipse.jetty.server.UserIdentity$Scope
[WARNING]   - org.eclipse.jetty.server.AsyncContextState$1
[WARNING]   - org.eclipse.jetty.server.handler.ScopedHandler
[WARNING]   - org.eclipse.jetty.server.handler.AbstractHandler
[WARNING]   - 207 more...
[WARNING] websocket-common-9.3.6.v20151106.jar, spark-debug-tools-0.5.jar define 111 overlapping classes: 
[WARNING]   - org.eclipse.jetty.websocket.common.events.annotated.InvalidSignatureException
[WARNING]   - org.eclipse.jetty.websocket.common.io.FrameFlusher$1
[WARNING]   - org.eclipse.jetty.websocket.common.extensions.ExtensionStack$Flusher
[WARNING]   - org.eclipse.jetty.websocket.common.Generator
[WARNING]   - org.eclipse.jetty.websocket.common.events.JettyListenerImpl
[WARNING]   - org.eclipse.jetty.websocket.common.events.AbstractEventDriver
[WARNING]   - org.eclipse.jetty.websocket.common.frames.PongFrame
[WARNING]   - org.eclipse.jetty.websocket.common.message.MessageAppender
[WARNING]   - org.eclipse.jetty.websocket.common.WebSocketRemoteEndpoint$MsgType
[WARNING]   - org.eclipse.jetty.websocket.common.extensions.fragment.FragmentExtension$Flusher
[WARNING]   - 101 more...
[WARNING] websocket-servlet-9.3.6.v20151106.jar, spark-debug-tools-0.5.jar define 7 overlapping classes: 
[WARNING]   - org.eclipse.jetty.websocket.servlet.WebSocketCreator
[WARNING]   - org.eclipse.jetty.websocket.servlet.ServletUpgradeResponse
[WARNING]   - org.eclipse.jetty.websocket.servlet.UpgradeHttpServletRequest
[WARNING]   - org.eclipse.jetty.websocket.servlet.WebSocketServlet
[WARNING]   - org.eclipse.jetty.websocket.servlet.ServletUpgradeRequest
[WARNING]   - org.eclipse.jetty.websocket.servlet.WebSocketServletFactory
[WARNING]   - org.eclipse.jetty.websocket.servlet.WebSocketServletFactory$Loader
[WARNING] spark-debug-tools-0.5.jar, websocket-api-9.3.6.v20151106.jar define 44 overlapping classes: 
[WARNING]   - org.eclipse.jetty.websocket.api.WebSocketPolicy
[WARNING]   - org.eclipse.jetty.websocket.api.annotations.OnWebSocketConnect
[WARNING]   - org.eclipse.jetty.websocket.api.util.QuoteUtil$DeQuotingStringIterator
[WARNING]   - org.eclipse.jetty.websocket.api.BatchMode
[WARNING]   - org.eclipse.jetty.websocket.api.SuspendToken
[WARNING]   - org.eclipse.jetty.websocket.api.BadPayloadException
[WARNING]   - org.eclipse.jetty.websocket.api.WebSocketAdapter
[WARNING]   - org.eclipse.jetty.websocket.api.extensions.Extension
[WARNING]   - org.eclipse.jetty.websocket.api.WebSocketException
[WARNING]   - org.eclipse.jetty.websocket.api.CloseException
[WARNING]   - 34 more...
[WARNING] spark-debug-tools-0.5.jar, jetty-xml-9.3.6.v20151106.jar define 13 overlapping classes: 
[WARNING]   - org.eclipse.jetty.xml.ConfigurationProcessorFactory
[WARNING]   - org.eclipse.jetty.xml.XmlParser$Node
[WARNING]   - org.eclipse.jetty.xml.XmlConfiguration$JettyXmlConfiguration
[WARNING]   - org.eclipse.jetty.xml.XmlConfiguration$JettyXmlConfiguration$AttrOrElementNode
[WARNING]   - org.eclipse.jetty.xml.ConfigurationProcessor
[WARNING]   - org.eclipse.jetty.xml.XmlParser$Handler
[WARNING]   - org.eclipse.jetty.xml.XmlConfiguration
[WARNING]   - org.eclipse.jetty.xml.XmlConfiguration$1
[WARNING]   - org.eclipse.jetty.xml.XmlAppendable
[WARNING]   - org.eclipse.jetty.xml.XmlParser
[WARNING]   - 3 more...
[WARNING] commons-lang3-3.1.jar, spark-debug-tools-0.5.jar define 151 overlapping classes: 
[WARNING]   - org.apache.commons.lang3.text.StrMatcher$NoMatcher
[WARNING]   - org.apache.commons.lang3.SerializationUtils
[WARNING]   - org.apache.commons.lang3.text.translate.NumericEntityEscaper
[WARNING]   - org.apache.commons.lang3.text.StrMatcher$StringMatcher
[WARNING]   - org.apache.commons.lang3.LocaleUtils
[WARNING]   - org.apache.commons.lang3.concurrent.ConcurrentRuntimeException
[WARNING]   - org.apache.commons.lang3.concurrent.ConcurrentException
[WARNING]   - org.apache.commons.lang3.CharRange
[WARNING]   - org.apache.commons.lang3.concurrent.AtomicInitializer
[WARNING]   - org.apache.commons.lang3.builder.IDKey
[WARNING]   - 141 more...
[WARNING] spark-debug-tools-0.5.jar, jetty-util-9.3.6.v20151106.jar define 228 overlapping classes: 
[WARNING]   - org.eclipse.jetty.util.MemoryUtils$1
[WARNING]   - org.eclipse.jetty.util.ArrayQueue
[WARNING]   - org.eclipse.jetty.util.resource.URLResource
[WARNING]   - org.eclipse.jetty.util.Scanner$DiscreteListener
[WARNING]   - org.eclipse.jetty.util.log.StacklessLogging
[WARNING]   - org.eclipse.jetty.util.preventers.Java2DLeakPreventer
[WARNING]   - org.eclipse.jetty.util.component.Dumpable
[WARNING]   - org.eclipse.jetty.util.resource.JarFileResource
[WARNING]   - org.eclipse.jetty.util.Promise
[WARNING]   - org.eclipse.jetty.util.Scanner
[WARNING]   - 218 more...
[WARNING] jetty-servlet-9.3.6.v20151106.jar, spark-debug-tools-0.5.jar define 46 overlapping classes: 
[WARNING]   - org.eclipse.jetty.servlet.ServletHandler$Default404Servlet
[WARNING]   - org.eclipse.jetty.servlet.FilterMapping
[WARNING]   - org.eclipse.jetty.servlet.ServletContextHandler
[WARNING]   - org.eclipse.jetty.servlet.Holder$1
[WARNING]   - org.eclipse.jetty.servlet.FilterMapping$1
[WARNING]   - org.eclipse.jetty.servlet.ServletContextHandler$ServletContainerInitializerCaller
[WARNING]   - org.eclipse.jetty.servlet.Invoker
[WARNING]   - org.eclipse.jetty.servlet.FilterHolder
[WARNING]   - org.eclipse.jetty.servlet.Holder
[WARNING]   - org.eclipse.jetty.servlet.jmx.HolderMBean
[WARNING]   - 36 more...
[WARNING] spark-debug-tools-0.5.jar, spark-core-2.5.jar define 62 overlapping classes: 
[WARNING]   - spark.resource.ClassPathResource
[WARNING]   - spark.Routable
[WARNING]   - spark.Route
[WARNING]   - spark.utils.ObjectUtils
[WARNING]   - spark.HaltException
[WARNING]   - spark.ResponseTransformer
[WARNING]   - spark.route.HttpMethod
[WARNING]   - spark.resource.ExternalResource
[WARNING]   - spark.resource.Resource
[WARNING]   - spark.Spark
[WARNING]   - 52 more...
[WARNING] spark-debug-tools-0.5.jar, jetty-webapp-9.3.6.v20151106.jar define 40 overlapping classes: 
[WARNING]   - org.eclipse.jetty.webapp.FragmentDescriptor
[WARNING]   - org.eclipse.jetty.webapp.WebAppContext$1
[WARNING]   - org.eclipse.jetty.webapp.WebAppContext
[WARNING]   - org.eclipse.jetty.webapp.AbstractConfiguration
[WARNING]   - org.eclipse.jetty.webapp.WebAppContext$Context
[WARNING]   - org.eclipse.jetty.webapp.FragmentDescriptor$OtherType
[WARNING]   - org.eclipse.jetty.webapp.WebDescriptor$1
[WARNING]   - org.eclipse.jetty.webapp.WebInfConfiguration$2
[WARNING]   - org.eclipse.jetty.webapp.Descriptor
[WARNING]   - org.eclipse.jetty.webapp.CloneConfiguration
[WARNING]   - 30 more...
[WARNING] spark-debug-tools-0.5.jar, jetty-security-9.3.6.v20151106.jar define 52 overlapping classes: 
[WARNING]   - org.eclipse.jetty.security.MappedLoginService
[WARNING]   - org.eclipse.jetty.security.SpnegoUserPrincipal
[WARNING]   - org.eclipse.jetty.security.authentication.DigestAuthenticator
[WARNING]   - org.eclipse.jetty.security.UserDataConstraint
[WARNING]   - org.eclipse.jetty.security.PropertyUserStore$UserListener
[WARNING]   - org.eclipse.jetty.security.authentication.FormAuthenticator
[WARNING]   - org.eclipse.jetty.security.UserAuthentication
[WARNING]   - org.eclipse.jetty.security.MappedLoginService$Anonymous
[WARNING]   - org.eclipse.jetty.security.DefaultAuthenticatorFactory
[WARNING]   - org.eclipse.jetty.security.SecurityHandler
[WARNING]   - 42 more...
[WARNING] websocket-server-9.3.6.v20151106.jar, spark-debug-tools-0.5.jar define 18 overlapping classes: 
[WARNING]   - org.eclipse.jetty.websocket.server.pathmap.PathSpec
[WARNING]   - org.eclipse.jetty.websocket.server.ServletWebSocketResponse
[WARNING]   - org.eclipse.jetty.websocket.server.pathmap.PathMappings$MappedResource
[WARNING]   - org.eclipse.jetty.websocket.server.WebSocketServerFactory
[WARNING]   - org.eclipse.jetty.websocket.server.WebSocketHandler$Simple
[WARNING]   - org.eclipse.jetty.websocket.server.pathmap.PathSpecGroup
[WARNING]   - org.eclipse.jetty.websocket.server.pathmap.PathMappings
[WARNING]   - org.eclipse.jetty.websocket.server.pathmap.RegexPathSpec
[WARNING]   - org.eclipse.jetty.websocket.server.pathmap.ServletPathSpec$1
[WARNING]   - org.eclipse.jetty.websocket.server.WebSocketHandler
[WARNING]   - 8 more...
[WARNING] spark-debug-tools-0.5.jar, slf4j-api-1.7.13.jar define 28 overlapping classes: 
[WARNING]   - org.slf4j.helpers.SubstituteLogger
[WARNING]   - org.slf4j.helpers.NamedLoggerBase
[WARNING]   - org.slf4j.helpers.NOPMDCAdapter
[WARNING]   - org.slf4j.MarkerFactory
[WARNING]   - org.slf4j.helpers.BasicMarker
[WARNING]   - org.slf4j.spi.LoggerFactoryBinder
[WARNING]   - org.slf4j.MDC$MDCCloseable
[WARNING]   - org.slf4j.spi.LocationAwareLogger
[WARNING]   - org.slf4j.helpers.MessageFormatter
[WARNING]   - org.slf4j.helpers.Util$ClassContextSecurityManager
[WARNING]   - 18 more...
[WARNING] spark-debug-tools-0.5.jar, jetty-io-9.3.6.v20151106.jar define 73 overlapping classes: 
[WARNING]   - org.eclipse.jetty.io.AbstractEndPoint$2
[WARNING]   - org.eclipse.jetty.io.WriteFlusher$FailedState
[WARNING]   - org.eclipse.jetty.io.ManagedSelector$Acceptor
[WARNING]   - org.eclipse.jetty.io.ByteBufferPool$Lease
[WARNING]   - org.eclipse.jetty.io.ManagedSelector$SelectableEndPoint
[WARNING]   - org.eclipse.jetty.io.ManagedSelector$1
[WARNING]   - org.eclipse.jetty.io.ClientConnectionFactory
[WARNING]   - org.eclipse.jetty.io.Connection$UpgradeTo
[WARNING]   - org.eclipse.jetty.io.LeakTrackingByteBufferPool$1
[WARNING]   - org.eclipse.jetty.io.WriterOutputStream
[WARNING]   - 63 more...
[WARNING] websocket-client-9.3.6.v20151106.jar, spark-debug-tools-0.5.jar define 16 overlapping classes: 
[WARNING]   - org.eclipse.jetty.websocket.client.masks.RandomMasker
[WARNING]   - org.eclipse.jetty.websocket.client.io.WebSocketClientSelectorManager
[WARNING]   - org.eclipse.jetty.websocket.client.ClientUpgradeResponse
[WARNING]   - org.eclipse.jetty.websocket.client.WebSocketClient
[WARNING]   - org.eclipse.jetty.websocket.client.ClientUpgradeRequest
[WARNING]   - org.eclipse.jetty.websocket.client.io.UpgradeConnection$State
[WARNING]   - org.eclipse.jetty.websocket.client.io.WebSocketClientConnection
[WARNING]   - org.eclipse.jetty.websocket.client.io.ConnectionManager
[WARNING]   - org.eclipse.jetty.websocket.client.masks.FixedMasker
[WARNING]   - org.eclipse.jetty.websocket.client.io.UpgradeConnection$SendUpgradeRequest
[WARNING]   - 6 more...
[WARNING] maven-shade-plugin has detected that some class files are
[WARNING] present in two or more JARs. When this happens, only one
[WARNING] single version of the class is copied to the uber jar.
[WARNING] Usually this is not harmful and you can skip these warnings,
[WARNING] otherwise try to manually exclude artifacts based on
[WARNING] mvn dependency:tree -Ddetail=true and the above output.
[WARNING] See http://maven.apache.org/plugins/maven-shade-plugin/

This is my second issue on this nice project, I hope it's not dead!

[tipsy]

Not dead, if you make any features I'd happily merge them. What should we do instead of shading? I did that because it wouldn't work when I tried to build a jar otherwise.

[Ribesg]

I'm not sure, I just know libraries should not shade their dependencies, because you will never run the jar itself, it's a library.

Maven projects referencing this one should detect this project's dependencies and get them as needed.

The idea is that right now, I can't remove the shaded dependencies of spark-debug-tools from my own jar, because they're "all in one". From my projects point of view, these are not dependencies of spark-debug-tools, they're part of spark-debug-tools. If you don't shade the dependencies, the warning will still be shown if versions does not match, but I will be able to choose between the possible versions, and not be forced into the version spark-debug-tools choosed.

Until this gets fixed, maven will choose a pseudo-random version between those available in the dependency tree for each dependency... Which isn't great!

[fwgreen]

As it stands now, one can't use this helpful tool with Spark (which surprisingly just works) on Java 9. I'm hoping the Maven folks figure something out.

[argoyb]

It appears that this shading is causing a weird issue when running spark app as java application. I've discovered this issue while using gradle docker plugin which packs app in jar and launches it with java -jar ....

Steps to reproduce:

1. Create "HelloWorld" spark application
2. Add "application" plugin to build.gradle, specify main class
3. Add to build.gradle:

compile 'com.sparkjava:spark-core:2.7.1'
compile 'com.sparkjava:spark-debug-tools:0.5'
compile 'org.slf4j:slf4j-simple:1.7.25' // to see logs.

With this setup if you run gradle run application starts normally and serves requests correctly, however if you run app with java -jar it won't work. Steps:

1. Run gradle installDist
2. cd to build/install/<app-name>
3. Run sh bin/<app-name>

Application will start with stacktraces like

[qtp325419416-13] WARN org.eclipse.jetty.util.thread.QueuedThreadPool -
java.lang.NoSuchMethodError: org.eclipse.jetty.util.thread.ExecutionStrategy.execute()V
	at org.eclipse.jetty.io.ManagedSelector.run(ManagedSelector.java:136)
	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:673)
	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:591)
	at java.lang.Thread.run(Thread.java:748)

and won't serve any requests.

Application start working if compile 'com.sparkjava:spark-debug-tools:0.5' is commented out.

[argoyb]

With more dependencies in project it's even more weird. E.g. for these dependencies:

compile 'com.sparkjava:spark-debug-tools:0.5'

    compile 'com.fasterxml.jackson.core:jackson-core:2.9.4'
    compile 'com.fasterxml.jackson.core:jackson-databind:2.9.4'
    compile 'com.fasterxml.jackson.module:jackson-module-parameter-names:2.9.4'
    compile 'com.fasterxml.jackson.datatype:jackson-datatype-jdk8:2.9.4'
    compile 'com.fasterxml.jackson.datatype:jackson-datatype-jsr310:2.9.4'
    compile 'com.fasterxml.jackson.datatype:jackson-datatype-guava:2.9.4'
    compile 'com.fasterxml.jackson.dataformat:jackson-dataformat-yaml:2.9.4'

    compile 'mysql:mysql-connector-java:6.0.6'
    compile 'com.zaxxer:HikariCP:2.7.7'
    compile 'org.jooq:jooq:3.10.4'

    compile 'com.google.guava:guava:24.0-jre'
    compile 'com.google.inject:guice:4.1.0'
    compile 'com.typesafe:config:1.3.1'

    compile 'com.amazonaws:aws-java-sdk-s3:1.11.278'

    compile 'com.sparkjava:spark-core:2.7.1'
    compile 'org.slf4j:slf4j-simple:1.7.25' // to see logs.
    testCompile group: 'junit', name: 'junit', version: '4.12'

If I run sh bin/<app-name> from build/install/<app-name> there are no stacktraces, but curl localhost:4567/hello returns

curl: (52) Empty reply from server

[jschwartz73]

I just submitted a PR for this issue:

#11

Low tech solution... removed the maven shade plugin, and documented in the readme the dependencies.

[aseure]

Running into the same issue.

I'm no expert but if you want to keep the dependencies shaded, I think it's possible by making sure to relocate them at the same time (with a shaded.XXX group prefix for instance as described here).

This way, they could still be packaged with the library but would not override other imports, is that correct?

[jschwartz73]

IMO a library should not bundle it's dependencies. Additionally, the spark, and other library versions are continually being updated, so bundling them could also cause incompatibility issues with newer versions.

[davutg]

Hmm that it why its size is this much big. It's >5M , spark core is only 155K. This debug page would be useful but I'm trying to reduce installation overall.

Now as an alternative I fancied using JSON transformer for exceptions .
Lemmi know if this size change one day, project is cool indeed :)