-
Notifications
You must be signed in to change notification settings - Fork 80
Description
This is the outcome of #6928, but focused on OVH instead of Terraform due to https://github.com/2i2c-org/meta/issues/2781 and me doing some of the necessary work in jupyterhub/mybinder.org-deploy#3476.
Outcome
The outcome is to have a single JupyterHub running on OVH from our infrastructure, with the following features
- Running in its own cloud project, so billing can be distinct and separate
- A cluster is a single node VM, automatically set up via k3s w/ terraform (pinned to a certain version).
- All state (hub db, prometheus, grafana, home directories) are set up via size limited but persistent disk, separate from the VM's root disk
- Floating IP used for ingress into the VM (so this can be shifted without DNS hijinks in the future)
- CI/CD setup to run from the infrastructure repository
- An error message when capacity is reached
We'll also need a vague plan for how we'll handle k3s upgrades.
Non-features
To start with, we won't have:
- Dynamic image building (we can add this on later if needed)
- Support for cost monitoring via grafana (we can build this in later too if needed)
- No dask-gateway
- No object storage (we can add this later if we need)
What kind of community will this serve?
I believe this kind of hub can serve educational institutions with basic needs. More specifically as an MVP, if the following is true Total number of students who can use the service at a given time * RAM per student < 200 they are a good fit. I believe there's a large number of groups for whom this is true.
We can offer this to them at a reasonably fixed cost. OVH is more expensive than hetzner, but a safer place to start from right now. You can see OVH pricing in https://www.ovhcloud.com/en/public-cloud/prices/.
Using the formula of InstanceSizePricePerHour + (PersistentDiskSize * $0.000132) + (InstanceBaseDiskSize * $0.000017) * 730 for total price, and assuming a 250G disk for everything
Roughly speaking, we can offer:
| Max active students * RAM | Price | Notes |
|---|---|---|
| 50 | $225-$250 / month | r3-64 instance |
| 100 | $400-$425 / month | r3-128 instance |
| 200 | $800 - $825 / month | r3-256 instance |
We can size these up and down via support tickets, because otherwise they will sit idle - there's no autoscaling here. An upshot is that server startups will always be instantaneous.
Please see additional comments in https://github.com/2i2c-org/meta/issues/2738#issuecomment-3408666718 about potential product fit.
Comparison to Hetzner
Note that this is more expensive than Hetzner! Hetzner is cheaper by almost 40%. If that is a dealbreaker, we can skip OVH and go straight to hetzner. However, I believe that OVH is a safer place to start for this kind of product, and we can expand to hetzner in the future - primarily because hetzner's support has been a bit iffy, and I don't want us to end up in a situation where a community has an outage due to hetzner kicking us out.
Technical questions to answer
- Is this possible at all within our timebox without contorting our infrastructure?
- How long will it take an engineer to set up a new hub under this model?
- Do we have a clear k3s upgrade pathway here?
The good news is that even if we decide to not offer this as a product, it will drastically improve our ability to do https://github.com/2i2c-org/meta/issues/2781