Skip to content

Implement failIfAlreadyExists in S3 repositories #128565

@DaveCTurner

Description

@DaveCTurner

Several write methods on BlobContainer accept a failIfAlreadyExists parameter to prevent Elasticsearch from overwriting an existing blob. In the past, the S3 API had no way to express this constraint so S3BlobContainer simply ignores it and hopes for the best. However since late 2024 AWS S3 now supports conditional writes, as does MinIO, so we should make use of this new feature in Elasticsearch for extra repo-corruption protection.

In terms of testing, we should verify that we are really doing conditional writes by adding a test to S3BlobStoreRepositoryTests, adjusting S3HttpHandler as required, and therefore adding to S3HttpHandlerTests too. Since all the other repository types support conditional writes, with this change we can safely add a test for conditional writes in org.elasticsearch.repositories.AbstractThirdPartyRepositoryTestCase. Ideally we would also extend RepositoryAnalyzeAction to verify conditional writes work correctly in real clusters, but we could leave that for a follow-up PR if desired.

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions