Skip to content

Conversation

@Gautam-aman
Copy link

@Gautam-aman Gautam-aman commented Dec 3, 2025

Description

This PR registers the s3.client.<client>.disable_chunked_encoding setting in the repository-s3 plugin.

Currently, configuring the setting in opensearch.yml such as:

s3.client.default.disable_chunked_encoding: true

results in a startup failure : unknown setting [s3.client.default.disable_chunked_encoding]

even though DISABLE_CHUNKED_ENCODING is defined in S3ClientSettings and supported at repository level.

Root Cause : 
DISABLE_CHUNKED_ENCODING was not included in the list returned by S3RepositoryPlugin#getSettings(), so the setting was never registered as a valid node setting.

Fix:
Added S3ClientSettings.DISABLE_CHUNKED_ENCODING to S3RepositoryPlugin#getSettings().
Added a test assertion in S3RepositoryPluginTests#testGetExecutorBuilders to ensure the setting is registered.

Testing
./gradlew :plugins:repository-s3:check passes successfully.

Impact : 
Users can now safely configure s3.client.default.disable_chunked_encoding in opensearch.yml for S3-compatible storage that does not support AWS chunked encoding (e.g., Oracle Object Storage, MinIO, Ceph, Cloudflare R2, etc.).


<!-- This is an auto-generated comment: release notes by coderabbit.ai -->
## Summary by CodeRabbit

* **New Features**
  * Exposes new S3 client settings: max sync connections, connection acquisition timeout, max pending connection acquires, and a chunked-encoding toggle.

* **Removals**
  * Removes several previously exposed S3 client settings (request/connection retry and some legacy/parallel upload options).

* **Tests**
  * Test suite updated to assert the revised set of S3 configuration settings.

* **Other**
  * Internal equality/hash behavior adjusted; no public API signatures changed.

<sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub>
<!-- end of auto-generated comment: release notes by coderabbit.ai -->

@Gautam-aman Gautam-aman requested a review from a team as a code owner December 3, 2025 18:27
@coderabbitai
Copy link

coderabbitai bot commented Dec 3, 2025

Walkthrough

Updated S3 repository plugin settings list (removed a subset of older settings and added new connection-related settings), expanded unit tests to assert these settings, and adjusted S3ClientSettings equality/hashCode and refine logic to include maxSyncConnections and use primitive comparison for disableChunkedEncoding.

Changes

Cohort / File(s) Summary
Plugin settings list
plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java
Replaced a large subset of previously exposed S3 settings with a new/updated set focused on connection management. Removed settings such as READ_TIMEOUT_SETTING, MAX_RETRIES_SETTING, REGION, ROLE_ARN_SETTING, IDENTITY_TOKEN_FILE_SETTING, ROLE_SESSION_NAME_SETTING, LEGACY_MD5_CHECKSUM_CALCULATION, PARALLEL_MULTIPART_UPLOAD_*, REDIRECT_LARGE_S3_UPLOAD, UPLOAD_RETRY_ENABLED, S3_PRIORITY_PERMIT_ALLOCATION_PERCENT, PERMIT_BACKED_TRANSFER_ENABLED, S3_ASYNC_HTTP_CLIENT_TYPE. Added/now exposing MAX_SYNC_CONNECTIONS_SETTING, CONNECTION_ACQUISITION_TIMEOUT, MAX_PENDING_CONNECTION_ACQUIRES, DISABLE_CHUNKED_ENCODING, plus existing connection/timeout settings (REQUEST_TIMEOUT_SETTING, CONNECTION_TIMEOUT_SETTING, CONNECTION_TTL_SETTING, MAX_CONNECTIONS_SETTING). Inline comment updated to "s3 client configuration settings".
Tests
plugins/repository-s3/src/test/java/org/opensearch/repositories/s3/S3RepositoryPluginTests.java
Expanded testGetExecutorBuilders to assert the plugin exposes additional client settings, including S3ClientSettings.DISABLE_CHUNKED_ENCODING, MAX_SYNC_CONNECTIONS_SETTING, CONNECTION_ACQUISITION_TIMEOUT, MAX_PENDING_CONNECTION_ACQUIRES, and other connection/timeout related settings.
Client settings equality/hash
plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3ClientSettings.java
refine() now considers maxSyncConnections when deciding reuse; equals() updated to compare maxSyncConnections and uses primitive boolean equality for disableChunkedEncoding; hashCode() updated to include maxSyncConnections.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~20 minutes

  • Verify the exact final list of settings exposed by S3RepositoryPlugin.getSettings() matches intended additions/removals.
  • Confirm S3RepositoryPluginTests assertions align to the new exposed settings and that test expectations aren't relying on removed settings.
  • Inspect S3ClientSettings.refine()/equals()/hashCode() changes for correctness and consistency (especially hash contract with equals).
  • Check any callers or consumers of the removed settings elsewhere in the codebase (not present in this diff) to avoid silent regressions.

Poem

🐇 I hopped through code where settings change,
New connections lined the meadow range,
Tests sniffed each flag with bright delight,
Hashes hummed true beneath the night,
A carrot grin — the config's in sight. 🥕

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)
Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 16.67% which is insufficient. The required threshold is 80.00%. You can run @coderabbitai generate docstrings to improve docstring coverage.
✅ Passed checks (2 passed)
Check name Status Explanation
Title check ✅ Passed The title accurately summarizes the main change: registering the s3.client.<client>.disable_chunked_encoding setting in repository-s3 node settings.
Description check ✅ Passed The PR description fully covers the required template sections: clear description of changes, impact explanation, and testing validation. Related issues section is not filled, but that appears optional.
✨ Finishing touches
  • 📝 Generate docstrings
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Post copyable unit tests in a comment

Comment @coderabbitai help to get the list of available commands and usage tips.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (2)
plugins/repository-s3/src/test/java/org/opensearch/repositories/s3/S3RepositoryPluginTests.java (1)

78-79: Keep the assertion, but avoid PR-specific comment wording

The new assertion correctly verifies that DISABLE_CHUNKED_ENCODING is registered in the plugin settings. The inline comment (“New assertion for your bug fix”) is tied to this PR and doesn’t convey long-term intent; consider either removing it or replacing it with something descriptive like “ensure DISABLE_CHUNKED_ENCODING is registered as a node setting”.

plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1)

366-367: Setting registration is correct; clean up the inline bug-fix comment

Including S3ClientSettings.DISABLE_CHUNKED_ENCODING in getSettings() is the right fix to register this node setting. The comment “Fixed the bug in this line” is tied to this PR and doesn’t describe behavior; consider removing it or replacing it with a brief description of what the setting does (e.g., that it controls disabling HTTP chunked encoding for S3 clients).

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between b7f013f and bfa670a.

📒 Files selected for processing (2)
  • plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1 hunks)
  • plugins/repository-s3/src/test/java/org/opensearch/repositories/s3/S3RepositoryPluginTests.java (1 hunks)

@github-actions
Copy link
Contributor

github-actions bot commented Dec 3, 2025

❌ Gradle check result for bfa670a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@Gautam-aman Gautam-aman force-pushed the fix-disable-chunked-encoding branch from bfa670a to 217b8b7 Compare December 3, 2025 19:03
Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1)

366-368: DISABLE_CHUNKED_ENCODING registration is correct; consider cleaning up the comment

The addition of S3ClientSettings.DISABLE_CHUNKED_ENCODING to getSettings() is exactly what’s needed for the node setting to be registered; behavior-wise this looks good.

The inline comment // Fixed the bug in this line is not very descriptive and will age quickly. I’d either drop it or replace it with a comment about what the setting does instead of the historical note.

For example, to just remove the historical comment:

-            S3ClientSettings.SIGNER_OVERRIDE,
-            // Fixed the bug in this line
-            S3ClientSettings.DISABLE_CHUNKED_ENCODING,
+            S3ClientSettings.SIGNER_OVERRIDE,
+            S3ClientSettings.DISABLE_CHUNKED_ENCODING,

Or, if you want a descriptive comment:

-            S3ClientSettings.SIGNER_OVERRIDE,
-            // Fixed the bug in this line
-            S3ClientSettings.DISABLE_CHUNKED_ENCODING,
+            S3ClientSettings.SIGNER_OVERRIDE,
+            // Allows disabling AWS SDK chunked encoding for S3-compatible storage
+            S3ClientSettings.DISABLE_CHUNKED_ENCODING,
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between bfa670a and 217b8b7.

📒 Files selected for processing (2)
  • plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1 hunks)
  • plugins/repository-s3/src/test/java/org/opensearch/repositories/s3/S3RepositoryPluginTests.java (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • plugins/repository-s3/src/test/java/org/opensearch/repositories/s3/S3RepositoryPluginTests.java
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: gradle-check

@github-actions
Copy link
Contributor

github-actions bot commented Dec 3, 2025

❌ Gradle check result for 217b8b7: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@Gautam-aman Gautam-aman force-pushed the fix-disable-chunked-encoding branch from 217b8b7 to 8d42f07 Compare December 4, 2025 04:39
@github-actions
Copy link
Contributor

github-actions bot commented Dec 4, 2025

❌ Gradle check result for 8d42f07: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@Gautam-aman Gautam-aman force-pushed the fix-disable-chunked-encoding branch from 8d42f07 to 068a2bd Compare December 4, 2025 05:17
@Gautam-aman
Copy link
Author

CI didn't start due to a Jenkins trigger 403 / jq parsing error.
Requesting to retrigger gradle-check manually. Thank you!

@github-actions
Copy link
Contributor

github-actions bot commented Dec 4, 2025

❌ Gradle check result for 068a2bd: null

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@Gautam-aman
Copy link
Author

I ran the suite locally via ./gradlew :plugins:repository-s3:internalClusterTest and it passes successfully.

The CI failure appears to be due to the Jenkins trigger issue (403 / jq parsing).
Requesting maintainers to manually retrigger gradle-check. Thank you!

@Gautam-aman
Copy link
Author

Thanks for the suggestion! 👍
I will extend this PR to expose all missing S3 client settings (not only disable_chunked_encoding) as recommended.

I’m working on adding:

  • REQUEST_TIMEOUT_SETTING
  • CONNECTION_TIMEOUT_SETTING
  • CONNECTION_TTL_SETTING
  • MAX_CONNECTIONS_SETTING
  • MAX_SYNC_CONNECTIONS_SETTING
  • CONNECTION_ACQUISITION_TIMEOUT
  • MAX_PENDING_CONNECTION_ACQUIRES
  • DISABLE_CHUNKED_ENCODING

Will push an update shortly.

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 0

🧹 Nitpick comments (1)
plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1)

367-367: Consider revising the comment for accuracy.

The comment "required new settings" is misleading—these settings already exist in S3ClientSettings but were not previously registered. Consider a more accurate comment such as "additional s3 client configuration settings" or "previously unregistered s3 client settings."

Apply this diff to improve the comment:

-            // required new settings
+            // additional s3 client configuration settings
📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 068a2bd and 7251e94.

📒 Files selected for processing (3)
  • plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3ClientSettings.java (2 hunks)
  • plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (2 hunks)
  • plugins/repository-s3/src/test/java/org/opensearch/repositories/s3/S3RepositoryPluginTests.java (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (2)
  • plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3ClientSettings.java
  • plugins/repository-s3/src/test/java/org/opensearch/repositories/s3/S3RepositoryPluginTests.java
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: gradle-check
🔇 Additional comments (1)
plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1)

368-375: Verify that all settings exist in S3ClientSettings.

The addition of these eight settings is essential for allowing users to configure them in opensearch.yml. Ensure that all these setting constants (REQUEST_TIMEOUT_SETTING, CONNECTION_TIMEOUT_SETTING, CONNECTION_TTL_SETTING, MAX_CONNECTIONS_SETTING, MAX_SYNC_CONNECTIONS_SETTING, CONNECTION_ACQUISITION_TIMEOUT, MAX_PENDING_CONNECTION_ACQUIRES, DISABLE_CHUNKED_ENCODING) are properly defined in S3ClientSettings to prevent compilation or runtime issues.

Signed-off-by: Aman Gautam <[email protected]>
@github-actions
Copy link
Contributor

github-actions bot commented Dec 4, 2025

❌ Gradle check result for aeceb9a: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

github-actions bot commented Dec 4, 2025

❌ Gradle check result for 12ac7d1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

@github-actions
Copy link
Contributor

github-actions bot commented Dec 4, 2025

❌ Gradle check result for 07f99f1: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 07f99f1 and 1d67079.

📒 Files selected for processing (1)
  • plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (2 hunks)
🧰 Additional context used
🧬 Code graph analysis (1)
plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1)
plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3ClientSettings.java (1)
  • S3ClientSettings (66-771)
⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (1)
  • GitHub Check: gradle-check
🔇 Additional comments (1)
plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java (1)

346-371: Unable to verify this review comment due to repository access constraints. The comment makes several specific claims that require codebase examination:

  1. Claims 14 settings were removed from getSettings() (based on an "AI summary" not provided)
  2. Claims 8 settings were promised in PR objectives but only 4 were added
  3. References specific setting names as removed

The provided code snippet shows only the final state of getSettings() without a diff showing what was removed. To properly verify, I would need to examine:

  • The full git diff showing removed vs. added settings
  • The actual PR description/objectives
  • The complete S3ClientSettings.java and S3Repository.java files
  • Whether the allegedly removed settings are actually defined elsewhere in the codebase

Without access to the actual codebase, I cannot confirm:

  • Whether these settings were actually removed
  • Whether they are still accessible through other means
  • The accuracy of the "8 promised settings" claim
  • Whether this truly constitutes a breaking change for production systems

The review comment contains actionable concerns, but the factual claims require codebase verification to assess validity.

Comment on lines +366 to +369
S3ClientSettings.MAX_SYNC_CONNECTIONS_SETTING,
S3ClientSettings.CONNECTION_ACQUISITION_TIMEOUT,
S3ClientSettings.MAX_PENDING_CONNECTION_ACQUIRES,
S3ClientSettings.DISABLE_CHUNKED_ENCODING
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

Incomplete implementation: Missing 4 promised settings.

In the PR comments, you stated you would add 8 settings: REQUEST_TIMEOUT_SETTING, CONNECTION_TIMEOUT_SETTING, CONNECTION_TTL_SETTING, MAX_CONNECTIONS_SETTING, MAX_SYNC_CONNECTIONS_SETTING, CONNECTION_ACQUISITION_TIMEOUT, MAX_PENDING_CONNECTION_ACQUIRES, and DISABLE_CHUNKED_ENCODING.

However, only 4 settings were added here. The missing settings are:

  • S3ClientSettings.REQUEST_TIMEOUT_SETTING
  • S3ClientSettings.CONNECTION_TIMEOUT_SETTING
  • S3ClientSettings.CONNECTION_TTL_SETTING
  • S3ClientSettings.MAX_CONNECTIONS_SETTING

These are important S3 client configuration settings that should be registered for users to configure connection behavior.

Apply this diff to add the missing settings:

 // additional s3 client configuration settings added in this PR
+            S3ClientSettings.REQUEST_TIMEOUT_SETTING,
+            S3ClientSettings.CONNECTION_TIMEOUT_SETTING,
+            S3ClientSettings.CONNECTION_TTL_SETTING,
+            S3ClientSettings.MAX_CONNECTIONS_SETTING,
             S3ClientSettings.MAX_SYNC_CONNECTIONS_SETTING,
             S3ClientSettings.CONNECTION_ACQUISITION_TIMEOUT,
             S3ClientSettings.MAX_PENDING_CONNECTION_ACQUIRES,
             S3ClientSettings.DISABLE_CHUNKED_ENCODING
             );
📝 Committable suggestion

‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.

Suggested change
S3ClientSettings.MAX_SYNC_CONNECTIONS_SETTING,
S3ClientSettings.CONNECTION_ACQUISITION_TIMEOUT,
S3ClientSettings.MAX_PENDING_CONNECTION_ACQUIRES,
S3ClientSettings.DISABLE_CHUNKED_ENCODING
// additional s3 client configuration settings added in this PR
S3ClientSettings.REQUEST_TIMEOUT_SETTING,
S3ClientSettings.CONNECTION_TIMEOUT_SETTING,
S3ClientSettings.CONNECTION_TTL_SETTING,
S3ClientSettings.MAX_CONNECTIONS_SETTING,
S3ClientSettings.MAX_SYNC_CONNECTIONS_SETTING,
S3ClientSettings.CONNECTION_ACQUISITION_TIMEOUT,
S3ClientSettings.MAX_PENDING_CONNECTION_ACQUIRES,
S3ClientSettings.DISABLE_CHUNKED_ENCODING
);
🤖 Prompt for AI Agents
In
plugins/repository-s3/src/main/java/org/opensearch/repositories/s3/S3RepositoryPlugin.java
around lines 366 to 369, the settings list only includes
MAX_SYNC_CONNECTIONS_SETTING, CONNECTION_ACQUISITION_TIMEOUT,
MAX_PENDING_CONNECTION_ACQUIRES and DISABLE_CHUNKED_ENCODING but omitted the
four promised settings; update the settings registration to also include
S3ClientSettings.REQUEST_TIMEOUT_SETTING,
S3ClientSettings.CONNECTION_TIMEOUT_SETTING,
S3ClientSettings.CONNECTION_TTL_SETTING and
S3ClientSettings.MAX_CONNECTIONS_SETTING so all eight configured S3 client
settings are registered and available for user configuration.

@github-actions
Copy link
Contributor

github-actions bot commented Dec 4, 2025

❌ Gradle check result for 1d67079: FAILURE

Please examine the workflow log, locate, and copy-paste the failure(s) below, then iterate to green. Is the failure a flaky test unrelated to your change?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant