Combine projects into monorepo

[Topher-the-Geek]

Would you consider collecting all these separate repos into one?

My primary reason for asking: it would be a lot easier for me to search wikis, issues and code.

Other great reasons to do this:

• Bazel's remote execution doc list a few options. The one for buildbarn leads to this. I don't know where to start, so I move on to one of the other options in Bazel's list.
• I want to star something. Which repo? Shrug. So I star nothing.
• I (and likely many others) are looking at contributors, frequency of contributions, stars, forks, etc to gauge project health. With everything in one repo, there will be more activity.

Eventually, I have found my way around. I found the sample Kubernetes deployment and took that as my quickstart. I've found helpful docs. I love the write-ups of your decisions here. Overall, I'm happy I've take some time to dig look into this(these) project(s). It would have been nice if it had known where to start.

If you use a monorepo, people will know to start with README.md at the root—and next they'll look for "Quickstart" or such. They'll have an easier time searching the wiki and issues when things fail. They'll know what to star, and they'll see lots of activity. They'll be more likely to adopt buildbarn, and that in turn will create a virtuous cycle.

[EdSchouten]

This is all great if all of Buildbarn was developed at the same pace/cadence, which is clearly not the case:

• bb-asset-hub is part of the Buildbarn project, but it's not maintained by the same people as bb-storage, bb-remote-execution, etc.. The former does depend on the latter in terms of library code. If all these projects were part of the same repository, then it'll be harder to make API changes.
• The example deployments in bb-deployments/ are only updated once in a while. By merging all repositories together, there would most likely be this requirement that deployments need to be updated every time we do a code change. I'm not sure that's realistic. The project lacks infrastructure to test these deployments automatically. By keeping bb-deployments separate, it is easier to let it lag behind.

Furthermore, having separate repositories also makes it easier to deorbit things. bb-clientd is still in an experimental phase. What if it turns out it isn't the right approach after all? If it's all stored in a separate repository, we can easily archive it, just like we did for bb-event-service.

[EdSchouten]

That said, there are certain things we can do to address some of the things you were running into. We could, for example, disable the creation of issues in any repository other than bb-storage. Suggestions are welcome.

[Topher-the-Geek]

I see. It does seem perfectly reasonable to keep bb-clientd and bb-remote-asset and in their own repos. They are experimental, and they are not essential to a useful Buildbarn deployment. If they ever become solid and more broadly useful, they can be rolled into (graduated into) the main repo.

As an prospective adopter, I need bb-storage, bb-remote-execution and bb-browser working together, so it does make sense to nudge them into the same cadence. Furthermore, I need to be able to deploy them, so I as a prospective adopter I need at least one deployment that's in sync too.

I can see a case for moving the key services and one sample deployment (I'd nominate K8S or bare) into the same repo, and making an effort to keep it in sync. This would be helpful to potential adopters. In my attempt to deploy on K8S, this lack of synchronization was one of the things that prompted me to open this issue in the first place.

The other sample deployments could reasonably be kept in a bb-contrib-deployments repo; or even in bb-contrib-aws, bb-contrib-gcp, etc repos. This latter arrangement also provides a natural home for bb-autoscaler, since that looks AWS centric. Tho it looks to me like that can be archived, because...this is a separate conversation, but long story short: that's not how I would do it (I use AWS ASG a lot right now).

Overall, I like Buildbarn. I have found your docs really easy to understand, once I found my way around them. I like that Buildbarn has sample deployments at all, even tho I stumbled to use Bazelbarn on this point. This is a great contrast to a certain competitor who's docs are in comprehensible, and who's quick start is completely out of touch with devops (their quick start reads: "run bazel run server... and in a separate terminal run bazel run worker...". yeah, that's not how we run things for realz).