Tracker: developing the right software for HTTP-IPFS Gateway usecases

[BigLep]

Purpose

This issue is a placeholder to link docs, issues, PRs, etc. concerning software that is targeted for IPFS Gateways.

Background

Many (most?) Gateways Providers use go-ipfs or some forked version. go-ipfs is in a challenging position of being a single binary supporting desktop application and web-service needs via configuration. go-ipfs maintainers are concerned that we're either not serving either party very well and/or more slowly than we ought.

What are we going to do?

We are not committed to any changes yet, but we believe it's appropriate to at the minimum:

1. Check in and review current Gateway operator usecases and understand Gateway operator needs/wants.
2. Determine if a separate/different development effort targeted towards gateways would be appropriate. Should the answer be yes, the next step would be to create a plan as a community and also execute it as a collective.

Feel free to link/post ideas/thoughts, and go-ipfs maintainers will as well.

When are we going to do anything?

go-ipfs maintainers are focused on the 0.11 and 0.12 releases for 2021Q4, but will be having the conversations above in parallel.

Additional Notes

1. This likely should be a GitHub Discussion rather than issue, but consciously not taking that on now to avoid slowing down getting the issue started. We can certainly move to a Discussion after.
2. There is chatter happening on this in Filecoin Slack #ipfs-operators as well. I'll pull out relevant pieces here as well.

[BigLep]

This is a brainstorm of topics that would like to have operators share about their gateway setup for a being-planned meetup (likely 2021-10-28):

The meeting is currently being scheduled and before the meeting we'll ask attendees to fill out answers in advance.

Software

1. Do you operate go-ipfs or a forked version?
2. If a forked version, what do you include in the fork?
3. What datastore do you use and why?
4. What are your biggest pain points with go-ipfs as an operator as it exists today?

Configuration

1. What go-ipfs configuration is most critical beyond the server profile?
2. What configuration knobs in go-ipfs do you wish you could turn?
3. Do you have any special peering arrangements?

Process

1. How do you handle "bad bits" (e.g., DMCA takedown requests, illegal content)? How would you want to handle them?

Infrastructure

1. If your installation is deployed to multiple "regions" (e.g., NA Pacific, EU West), how does routing happen (e.g., BGP, DNS with geolocation routing policy, rely on users to use a different DNS for each region)
2. How do you load balance requests to the various go-ipfs instances in a given region?
3. Do you use a proxy in front of go-ipfs (e.g., nginx)? If so, what do you use it for (e.g., TLS termination, caching)?
4. Do you use a CDN?
5. What is your caching story?
6. What metrics do you wish go-ipfs emitted?
7. Where do your hosts live (e.g., public cloud, on prem, other bare metal provider)?
8. How do you (or would like to) package & deploy Gateway software? micro services, Kubernetes container orchestration?
9. What metrics (operational, performance, etc.) do you use to make decisions (availability, performance, caching, scale up/down, node health, routing, etc.)?
10. How do you scale Gateway infrastructure, and what signals do you look for?
11. How do you monitor and troubleshoot Gateway infra?
12. What are some of the key initiatives/projects you have implemented as part of continuous improvement?
13. How do you plan and manage capacity?

State

1. How homogenous are your gateways or do you want them to be?
2. Do you pin any data on your gateway nodes? If so, why?
3. How important are repo migrations to your gateway deployments? Said another way, what data stored on gateway nodes would you be sad if it disappeared between versions of gateway deployments?

[proto-sid]

Continuing on

Infrastructure:

8. How do you (or would like to) package & deploy Gateway software - micro services, Kubernetes container orchestration?
9. What metrics (operational, performance, etc.) do you use to take decisions (availability, performance, caching, scale up/down, node health, routing, etc.)?
10. How do you scale Gateway infrastructure, and what signals do you look for?
11. How do you monitor and troubleshoot Gateway infra?
12. What are some of the key initiatives/projects you have implemented as part of continuous improvement?
13. How do you plan and manage capacity?

[BigLep]

Form for collecting input to the questions above is here: https://docs.google.com/forms/d/e/1FAIpQLSfmIB5FrS39E8QQpkd_lfa_RIMF5jzZmeycOPRbJq4IGtpF2A/viewform?usp=sf_link

Responses are viewable here: https://docs.google.com/spreadsheets/d/1GK9Xd5QmFUKYI8S_YDsuM_7B7qoVgTs1XRhYoRNyAdE/edit?usp=sharing

[BigLep]

There is a meetup amongst operators happening on 2021-10-28 per https://lu.ma/kgk9dszt

Agenda / notes during the meeting will be in https://docs.google.com/document/d/1dS7-w-YIIFeyP0nTp9suaBmEAbU_H9Iw1hCROZbvW1Q/edit# and then copied back here.