[state sync] migrate to new bucket format

[phoenix-o]

Description

migrates state sync to the new bucket format

---

Release notes

Check each box that your changes affect. If none of the boxes relate to your changes, release notes aren't required.

For each box you select, include information after the relevant heading that describes the impact of your changes that a user might notice and any actions they must take to implement updates.

• Protocol:
• Nodes (Validators and Full nodes):
• gRPC:
• JSON-RPC:
• GraphQL:
• CLI:
• Rust SDK:
• Indexing Framework:

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
sui-docs	Ready	Preview, Comment	Mar 23, 2026 7:06pm

2 Skipped Deployments

Project	Deployment	Actions	Updated (UTC)
multisig-toolkit	Ignored	Preview	Mar 23, 2026 7:06pm
sui-kiosk	Ignored	Preview	Mar 23, 2026 7:06pm

[amnn]

@phoenix-o, could you motivate your use of the indexing framework here, as well as some of the changes you need to make to it?

• Why does state sync need those changes, and are those needs general to all users of the indexing framework or specific to state sync?
• How much of the indexing framework are you using?

It's possible/likely that the best option is to not use the indexing framework, but implement a small/custom interface over object_store instead that caters exactly to the node software's needs.

[phoenix-o]

@amnn: State sync currently uses the old ingestion framework. This PR migrates it(last callsite) and utilizes the new checkpoint format.
Only IngestionClient is used to download the checkpoints.
There are not many changes to the framework itself.
A new optional parameter is added remote_store_extra_settings. It's empty by default, so it does not change the behavior of any existing clients.
This allows dynamically passing extra settings to the underlying object store library for more precise configuration.
For instance here StateSync needs to pass down credentials for requester-pays behavior.
I only need this for AWS at the moment, but I updated GCP and Azure as well (can revert if needed)

[amnn]

Only IngestionClient is used to download the checkpoints.

If only IngestionClient is used and even within that you only need to be able to fetch checkpoints from an object store, then I'm still leaning towards not using the indexing framework for this, but just implementing a thin wrapper around object_store, and there's a lot hanging off that dependency that you do not need/use. I am also thinking of the fact that we want to move the indexing framework out of the mono-repo at some point and publish it to crates.io.

For instance here StateSync needs to pass down credentials for requester-pays behavior.

Separately, I think this issue has come up before, so maybe it is worth solving in the framework, even if state sync doesn't use it:

• @nickvikeras I recall you needed to solve this problem for the analytics indexer, and you were able to do this without updating the framework -- could you explain what you did (and whether you have a path towards generalising it in the framework?)
• @rushrs what does the set-up look like for our indexers -- do they use requester pays buckets and if so how do they get configured? I believe we expose configuration via environment variables for this kind of thing, but I'm not sure we use it.

[phoenix-o]

@amnn: sounds good, I’ll revert the changes to IngestionClientArgs. Just thought it might be helpful for more precise future configuration of underlying object store clients

[amnn]

Thanks @phoenix-o, this is now mostly outside of my domain, so will leave the rest of the review to @bmwill or @nickvikeras

[amnn]

Should be able to get rid of this now, right?