-
Notifications
You must be signed in to change notification settings - Fork 804
SOLR-18072: Refactor CollectionApiCommands to add expandable context #4046
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SOLR-18072: Refactor CollectionApiCommands to add expandable context #4046
Conversation
dsmiley
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I overall like the proposal of an Admin API context class for these APIs.
I've been looking in this PR for where SubResponseAccumulatingJerseyResponse is defined but can't find it; maybe because Safari/GitHub is completely choking on this massive PR. I don't love the name of that thing. Can you please self-review to identify it?
solr/core/src/java/org/apache/solr/handler/admin/CollectionsHandler.java
Show resolved
Hide resolved
solr/solrj/src/test/org/apache/solr/client/solrj/impl/CloudHttp2SolrClientTest.java
Show resolved
Hide resolved
solr/test-framework/src/java/org/apache/solr/cloud/SolrCloudTestCase.java
Outdated
Show resolved
Hide resolved
solr/test-framework/src/java/org/apache/solr/cloud/SolrCloudTestCase.java
Outdated
Show resolved
Hide resolved
solr/core/src/test/org/apache/solr/handler/admin/api/MockAPITest.java
Outdated
Show resolved
Hide resolved
This response class already exists, and the method that populated it already existed too. I just expanded the use of it, so that we weren't doing the same thing across each API independently. That's why you can't find it in the PR. We can change the name independently if you dislike it, but not really in the scope of this PR. |
dsmiley
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Thanks.
solr/test-framework/src/java/org/apache/solr/cloud/SolrCloudTestCase.java
Outdated
Show resolved
Hide resolved
gerlowskija
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Publishing a partial review to make a few in-line comments visible. Hoping to have the second half of my review available shortly...
solr/api/src/java/org/apache/solr/client/api/endpoint/DeleteAliasApi.java
Outdated
Show resolved
Hide resolved
solr/core/src/java/org/apache/solr/cloud/api/collections/CreateCollectionCmd.java
Outdated
Show resolved
Hide resolved
.../java/org/apache/solr/cloud/api/collections/DistributedCollectionConfigSetCommandRunner.java
Show resolved
Hide resolved
| String internalAsyncId = asyncId + Math.abs(System.nanoTime()); | ||
| props.put(ASYNC, internalAsyncId); | ||
| String internalAsyncId = null; | ||
| if (adminCmdContext.getAsyncId() != null) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[Q] Wdyt of having some sort of getInternalAsyncId() or getSubrequestAsyncId() method in AdminCmdContext that would encapsulate this logic?
Seems like it's something a number of cmd implementations have to figure out for themselves, and would definitely fit under the general umbrella of "admin command context" conceptually.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So since this is mainly handled by the ShardRequestTracker, and very few Cmds actually create their own internal or sub request AsyncId directly, I don't think we really need a utility. Honestly I'm not sure these commands actually need to do it themselves at all. But not really in scope for this PR.
gerlowskija
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great overall!
I left a few minor comments here and there - I think the most significant of which are around some of the new SubResponseAccumulatingJerseyResponse usages. But this looks great, even with those quibbles.
✅
| @Override | ||
| @PermissionName(COLL_EDIT_PERM) | ||
| public SolrJerseyResponse addReplicaProperty( | ||
| public SubResponseAccumulatingJerseyResponse addReplicaProperty( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[+1] Good catch
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This one I actually reverted, since it doesn't have any success or failure being returned (or an async option)
| @PermissionName(COLL_EDIT_PERM) | ||
| public SolrJerseyResponse deleteAliasProperty(String aliasName, String propName) | ||
| throws Exception { | ||
| public SubResponseAccumulatingJerseyResponse deleteAliasProperty( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[-0] All of the "update" methods in this class shouldn't return SubResponseAccumulatingJerseyResponse, afaict, because the API response never uses the "successes" or "failures" fields that are the main reason for SubResponse... to exist.
If we aim of this change is to support async alias-prop modification, we can probably use AsyncJerseyResponse instead?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
So weirdly only updateAliasProperties supports async currently, not even createOrUpdateAliasProperty. So for now only updateAliasProperties will use the AsyncJerseyResponse. The other two will use SolrJerseyResponse.
I will go through the other files and make sure I'm not changing any of these when I shouldn't.
| public SolrJerseyResponse createAlias(CreateAliasRequestBody requestBody) throws Exception { | ||
| final SubResponseAccumulatingJerseyResponse response = | ||
| instantiateJerseyResponse(SubResponseAccumulatingJerseyResponse.class); | ||
| public SubResponseAccumulatingJerseyResponse createAlias(CreateAliasRequestBody requestBody) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[-0] Idk why L88 was instantiating a "SubResponseAccumulatingJerseyResponse" prior to this PR, maybe a copy-paste error on my part?
But in reality, this API never sets the "successes" or "failures" fields that are the main reason for SubResponse... to exist. So rather than switching the return type here to SubResponse, we should probably make it "SolrJerseyResponse" in both places? Or maybe "AsyncJerseyResponse" if we want to support asynchronous alias creation?
solr/core/src/java/org/apache/solr/handler/admin/api/CreateCollection.java
Show resolved
Hide resolved
solr/core/src/java/org/apache/solr/handler/admin/api/DeleteAlias.java
Outdated
Show resolved
Hide resolved
solr/core/src/test/org/apache/solr/handler/admin/api/DeleteNodeAPITest.java
Show resolved
Hide resolved
solr/core/src/test/org/apache/solr/handler/admin/api/MockAPITest.java
Outdated
Show resolved
Hide resolved
| assertEquals("someRepository", remoteMessage.get(BACKUP_REPOSITORY)); | ||
| assertEquals("someBackupName", remoteMessage.get(NAME)); | ||
|
|
||
| AdminCmdContext context = contextCapturer.getValue(); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[0] Most (all?) of these mock-based v2 API tests have some version of this little snippet where we're grabbing the AdminCmdContext out of a capturer and asserting the action and async ID. Just flagging it as there might be a way to pull it into a helper method in MockAPITest.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ok, I created a common validation method validateRunCommand that makes this really clean for all of the tests that do it. The method does everything after calling the api.
validateRunCommand(
CollectionParams.CollectionAction.DELETE,
message -> {
assertEquals(1, message.size());
assertThat(message, hasEntry(NAME, "someCollName"));
});and
validateRunCommand(
CollectionParams.CollectionAction.DELETE,
"someAsyncId",
message -> {
assertEquals(1, message.size());
assertThat(message, hasEntry(NAME, "someCollName"));
});| } | ||
|
|
||
| /** Returns whether the request status should be refreshed given the current state. */ | ||
| public boolean shouldRefresh() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
[0] Idk if "refresh" means anything to most users. Maybe isOperationFinished or something similar? Or maybe ignore me and "refresh" is the best option as anything else would risk causing confusion with the "COMPLETED" enum value.
Ignore me, just thinking aloud I guess...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree on "refresh" not being the most clear word. Maybe isFinal or isFinished, which implies you may want to get the state again.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Makes sense, I chose isFinal
solr/test-framework/src/java/org/apache/solr/cloud/SolrCloudTestCase.java
Show resolved
Hide resolved
| } | ||
|
|
||
| /** Returns whether the request status should be refreshed given the current state. */ | ||
| public boolean shouldRefresh() { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I agree on "refresh" not being the most clear word. Maybe isFinal or isFinished, which implies you may want to get the state again.
|
Ok, will run the tests a few times. But all review comments have been addressed so I'll be merging after confirming the tests are passing! Thanks for the help @gerlowskija and @dsmiley ! |
https://issues.apache.org/jira/browse/SOLR-18072
There are no functionality changes here. Just test improvements, and refactoring of the APIs.
I'm also happy to call
AdminCmdContextsomething else, that's a very easy change.