-
Notifications
You must be signed in to change notification settings - Fork 804
SOLR-17161: build: remove needless permitTestUnusedDeclared #4048
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
SOLR-17161: build: remove needless permitTestUnusedDeclared #4048
Conversation
And remove: * testImplementation libs.jakarta.servlet.api * testImplementation libs.eclipse.jetty.http2.client
risdenk
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
whoops definitely missed this thanks!
Remove additional needless "permit": solr-core, modules/cuvs, modules/extraction, modules/sql, solrj, solrj-streaming, test-framework
* clarify opentelemetry
dsmiley
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As this has increased in scope to be wholistic house-cleaning, maybe I should unlink to SOLR-17161. Albeit that JIRA roughly doubled the use of permitBlahBlah, which is the thing that should be avoided.
| testImplementation libs.junit.junit | ||
| testImplementation libs.commonsio.commonsio | ||
|
|
||
| // lucene-backward-codecs is a transitive dependency from cuvs-lucene but required in lockfile |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This explanation doesn't make sense IMO; instead it tells me that the author of the comment doesn't know to write locks first. Maybe our build is complicated for many folks :-(
|
|
||
| dependencies { | ||
| implementation platform(project(':platform')) | ||
| permitUnusedDeclared platform(project(":platform")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Very weird -- using permitBlahBlah on a platform. A platform based declaration isn't really a dependency, it just imports constraints on dependencies (not actually depending on those things).
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@risdenk please review this; I'd love to have your input
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This looks great. I went down this rabbit hole for a bit too the other day. I didn't make progress. I went through and ripped out all the "permit" and ended up with test failures that I didn't have time to debug. I agree with what you are trying to do. We should not have permit* anywhere for the most part. There may be a few cases where its needed but it should be very rare.
|
|
||
| dependencies { | ||
| api platform(project(":platform")) | ||
| permitUnusedDeclared platform(project(":platform")) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
this was a bizarre one since it's a "platform"
| // It is not included in the release so exclude from checks. | ||
| compileOnly libs.spotbugs.annotations | ||
| testCompileOnly libs.spotbugs.annotations | ||
| permitUnusedDeclared libs.spotbugs.annotations |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
because recent plugin update considers compileOnly as such (i.e. automatically)
| implementation libs.apache.lucene.backward.codecs | ||
| permitUnusedDeclared libs.apache.lucene.backward.codecs |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
nonetheless, it's still on the runtime classpath (lockfile is truth on such matters).
| permitUsedUndeclared libs.eclipse.jetty.http | ||
| permitUsedUndeclared libs.eclipse.jetty.util | ||
| permitUsedUndeclared libs.eclipse.jetty.io | ||
| implementation libs.eclipse.jetty.http | ||
| implementation libs.eclipse.jetty.util | ||
| implementation libs.eclipse.jetty.io |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@janhoy were you trying to avoid simply using implementation for some reason?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Strange. I must have added libs.eclipse.jetty.client first (which brings in the other three transitively), and then reacted to the error message by permitting them instead of adding explicit. I don't recall exactly what happened in october, but I may have misinterpreted the error message from the plugin during implementation, and then it was never revisited before merge.
| }) | ||
| permitTestUnusedDeclared libs.apache.zookeeper.zookeeper | ||
|
|
||
| permitTestUsedUndeclared project(':solr:solrj-streaming') // duh! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
so weird...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I dont think this is needed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Try for yourself ;-) It's the weirdest inexplicable thing that I've seen this plugin require me to do. There's one more of these -- search for "// duh!". Ideally this gets reported to the plugin author but I'm skeptical anything becomes of it without us doing the legwork
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add it as an testImplementation. similar to the other testImplementation around line 30. I think it was just the wrong choice to do it.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure I could switch to testImplementation, and maybe I should but the reason the line is there at all is because of this plugin's flaw. And the "permit" use is basically always a flaw, so the line as-is shows that this plugin is flawed.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
There is no flaw here. For the tests, the tests require classes from solr:solrj-streaming to compile. You could conceivably write tests without requiring the classes from solr:solrj-streaming.
If you put testImplementation as intended in Gradle, it would show that this module for the tests requires solr:solrj-streaming classes to compile. Thats not the plugin being wrong.
The wrong thing is that we don't have a testImplementation line that references solr:solrj-streaming
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I see that other modules (e.g. SolrJ) have testImplementation with an explicit dependency on the module itself, which you had added back when you added the cutterslade plugin. So we have cutterslade to thank for this verbosity. Yes of course a module's tests depend on the module! This is how Gradle or any build system would be expected to work. This is a bad side of the plugin -- a flaw; requiring needless verbosity to state the obvious.
I'll switch it to for consistency with our other files (thanks to cutterslade), and to remove a sad "permit", and report the bug.
| org.apache.yetus:audience-annotations:0.12.0=permitTestUnusedDeclared | ||
| org.apache.zookeeper:zookeeper-jute:3.9.4=jarValidation,permitTestUnusedDeclared,testCompileClasspath,testRuntimeClasspath | ||
| org.apache.zookeeper:zookeeper:3.9.4=jarValidation,permitTestUnusedDeclared,testCompileClasspath,testRuntimeClasspath | ||
| org.apache.zookeeper:zookeeper-jute:3.9.4=jarValidation,testCompileClasspath,testRuntimeClasspath |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
see; it's still on the test runtime classpath, so may be consulted in tests at runtime
|
|
||
| We use the "cutterslade.analyze" build plugin to help maintain good dependency hygiene. | ||
| It tries to identify when a dependency is needlessly declared, and then fail the build to complain. | ||
| Unfortunately, it also requires us to add declarations that Gradle would otherwise deem unnecessary, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
@risdenk I saw a notification about a comment that I can't find in GitHub. You said:
I don't think this statement is true?
The plugin is requiring that dependencies used in a module are actually declared. The api spec has nothing to do with it. It makes each module self sufficient. It means that even if an api dependency removes a dependency (say commons-io) that the module doesn't break if it uses commons-io. it just makes the usage of dependency explicit. This is only for compile time dependencies where its a compile time dependency to actually build the module in question.
How does an "api dependency" remove a dependency?
RE "api": An illustrative example would be :solrj-jetty having api libs.eclipse.jetty.http2.client (Jetty's HttpClient).
The use of "api" is a convenience to a user of a solrj-jetty to spare them from adding an additional implementation scoped dependencies that they are likely to use in conjunction with the lib. But thanks to cutterslade, within Solr, that distinction is defeated. Do you agree and/or am I missing something?
There's a spectrum of developer prerogative from conservative to liberal, in terms of use of "api". I say "likely to use in conjunction with" but another person's definition might simply be the mere existence of a public signature that includes the target lib. I've increasingly preferred to use it conservatively. At work it's used liberally in a mono-repo and the extensive dependency web that doing so creates is out of control.
And remove:
https://issues.apache.org/jira/browse/SOLR-17161
Background: while we collaborated on cleaning up the permitTestUnusedDeclared mess in solrj-jetty/build.gradle, we neglected to do similarly in solrj/build.gradle.
Process: (loop until done; took me 2 iterations; went super fast)
check