Deal with flaky storage providers #116
Description
This is primarily an M1/M2 question, and we need a more sophisticated selection mechanism at some point, but for now select your storage provider by:
- If you have an existing proofset that matches your parameters (
withCDNis the only parameter we care about atm), then we select that provider and proofset - If you don't have one then we randomly assign you one and that's all you get
But, in either case, if we can't connect to your storage provider then we have problems. We currently have one of our providers in the Pandora contract having connectivity issues.
I think maybe in here we should do a ping to the provider after we select them before we proceed after a createStorage. If we can't connect to the provider, then we should try again and exclude that provider from selection. Which might mean that it's the provider we were dealing with previously but now we have to set up with a new provider.
We also need to think of the UX/DX beyond M1/M2, how should you go about selecting a provider? What tools should we give you? I think at least being able to ping them would be good. So, I think a good outcome here is a /ping interface in pdp/server.ts that can be used when selecting a provider. So we should at least build that. The handler is here, we're just looking for a 200.
- Create a method that does a /ping for a given provider in our PDP server tool
- Use the /ping method after selecting a provider to ensure we can talk to them, if not, we should select again but explicitly exclude them, and continue in a loop until we exhaust the list and have no more providers to try