Skip to content

Support configurable checksum algorithms for s3 uploads#367

Open
prathamalwayscomeslast wants to merge 7 commits intojenkinsci:masterfrom
prathamalwayscomeslast:configurable-checksum-algo
Open

Support configurable checksum algorithms for s3 uploads#367
prathamalwayscomeslast wants to merge 7 commits intojenkinsci:masterfrom
prathamalwayscomeslast:configurable-checksum-algo

Conversation

@prathamalwayscomeslast
Copy link
Copy Markdown

fixes JENKINS-71836
Checksum algorithms are now configurable with this change, with CRC32 as the default.

Testing done

Manually tested checksumAlgorithm: 'CRC32C', 'SHA256', 'SHA1' and 'CRC32' s3 uploads and all show correct, corresponding checksum functions in S3.

Note

CRC64NVME algorithm led to unstable build and failed uploads as it requires aws crt dependency, so it is currently marked as unsupported.

Submitter checklist

  • Make sure you are opening from a topic/feature/bugfix branch (right side) and not your main branch!
  • Ensure that the pull request title represents the desired changelog entry
  • Please describe what you did
  • Link to relevant issues in GitHub or Jira
  • Link to relevant pull requests, esp. upstream and downstream changes
  • Ensure you have provided tests that demonstrate the feature works or the issue is fixed

Signed-off-by: PRATHAM ~ <prathamcomeslast@gmail.com>
Signed-off-by: PRATHAM ~ <prathamcomeslast@gmail.com>
Signed-off-by: PRATHAM ~ <prathamcomeslast@gmail.com>
Signed-off-by: PRATHAM ~ <prathamcomeslast@gmail.com>
@prathamalwayscomeslast prathamalwayscomeslast marked this pull request as draft March 25, 2026 18:20
…ling test

Signed-off-by: PRATHAM ~ <prathamcomeslast@gmail.com>
@prathamalwayscomeslast prathamalwayscomeslast marked this pull request as ready for review March 25, 2026 19:06
@prathamalwayscomeslast
Copy link
Copy Markdown
Author

@rsandell for this change, I have added the related unit test here but I've been running into too many pitfalls with that, NoSuchKey errors and such. And I am unsure how to move forward with that, I'd really appreciate any guidance on this :)

[ERROR] Errors: 
[ERROR]   MinIOTest.testChecksumAlgorithms:257 » NoSuchKey (Service: S3, Status Code: 404, Request ID: 18A022557ADDD619, Extended Request ID: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8) (SDK Attempt Count: 1) (Service: S3, Status Code: 404, Request ID: 18A022557ADDD619, Extended Request ID: dd9025bab4ad464b049177c95eb6ebf374d3b3fd1af9251148b658df7ac2e3e8)                                                                                               
[INFO] 
[ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0

However, I have manually tested this on S3 bucket with different algorithms, most work as expected. So will this be safe to merge?

Signed-off-by: PRATHAM ~ <prathamcomeslast@gmail.com>
Signed-off-by: PRATHAM ~ <prathamcomeslast@gmail.com>
@rsandell
Copy link
Copy Markdown
Member

Well I don't know enough to give any definite answer, but the first thing I see is that the line where the test fails is in code that seems to do some jenkins calls under the hood but outside of a running jenkins instance, i.e. not inside rr.then...

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants