[DISCUSSION] Consider moving to a wasi monorepo

[ricochet]

Key friction points:

• CI/CD drift
• Releasing wasip2 and wasip3 monolithically but waiting for each individual repo to release takes time and frequently encounters drift in errors and transient network failures.
• Inter-dependencies between proposals like clocks with filesystem. Frequently we want to make and test those changes in one go.

The minimal "monorepo" change would Include all phase 3 proposals. It could potentially also include:

• All proposals including very early proposals (wasi-nn, wasi-keyvalue, etc)
• Include wasi-testsuite

I am looking for reasons not to do this, and solutions to concerns raised.

[sunfishcode]

This would entail a unified issue tracker. A solution could be to establish a system of issue labels to identify which proposal an issue belongs to.

This would entail unified repo permissions. A solution could be to set up a CODEOWNERS file to designate that champions are reviewers for changes that affect their proposals.

And, this would entail unified notifications. A solution to allow people to subscribe to individual proposals without receiving notifications for all proposals could be to set up a subscribe-to-label system.

[lukewagner]

A monorepo makes sense to me too; it sounds like it'd save a bunch of churn and hassle and make WASI more approachable. 1-2 years ago I was vaguely worried that a monorepo for WIT sources would lead to WASI being treated like a single monolithic interface, as opposed to the modular set of interfaces we set out to define. But now that worlds and packages have been built out and folks are actively using them to do modular interface design in production, this doesn't seem like a concern any more.

[tschneidereit]

In addition to what Dan an Luke are saying, a key reason we decided not to use a monorepo was that we wanted to set WASI up for maximal flexibility to cover a wide range of APIs. I still am somewhat concerned giving this up would bite us in the long term.

Would a potential alternative be to treat the main WASI repo as an aggregator that is the source of truth for all APIs as far as releases go, but let development continue in separate repositories, similar to how Wasm proposals work? That way, discussions would stay focused, scalability would be retained, but the release process would be simpler?

I do appreciate that this is more work per-release, but I'm pretty concerned in particular about unified notifications: subscribe-to-label doesn't scale for this kind of work, because we want to have a wide audience be able to follow along with interfaces of their choosing, instead of a small group of maintainers setting up their individualized subscriptions by sending in PRs.

[pchickey]

I'm in favor of moving to a monorepo. Prior attempts to use the main WASI repo as an aggregator have only increased complexity.

As far as scalability is concerned, I think we over-rotated on scalability early in the post-p1 process and haven't seen the benefits. No proposals outside of the p2 launch set have reached phase 3 in the whole lifetime of p2, and none of the phase 2 proposals look to me like they will advance to phase 3 the next year or two. If we had seen a different sort of growth in the interfaces ecosystem, I'd still believe the scalability arguments I supported a few years ago, but it hasn't come to be, and I don't think moving to a monorepo now is going to hurt that.

If we are lucky enough that a monorepo is unsustainable in half a decade, we can figure out a new system that works for the community then.