service: add Trigger(name) to trigger bundle rebuild

[srenatus]

Adds the machinery to pool and service to trigger bundle rebuilds by name. If the bundle worker is queued, it'll be pulled into the front of the line, to be picked up next; if it's currently not in the queue, it must be executing: if so, we're overriding the next deadline to be now, causing an immediate re-run of the build process.

Use a config snippet like this

service:
  bundle_rebuild_interval: 1h
  reconfiguration_interval: 1h

to put the automatic rebuilds and config checks almost on hold.

It's now exposed with a a new role, "automation", and via HTTP, POST /v1/bundles/{name}. These are things we should probably discuss.

Next steps:

1. Figure out of this is the right thing, and
2. Find answers to the remaining questions related to exposing this trigger via an HTTP endpoint

[mjungsbluth]

@srenatus this looks like a great start but I think we can simplify this (and also solve the horizontal scalability issues):

• I would add a configuration flag like bundle_rebuild_periodically: true/false to allow disabling any kind of intrinsic periodicity. This would allow to just horizontally scale and have more than once OCP instance running
• It would actually be nice if the trigger would block until the build is complete which would make it a lot simpler for clients in CI/CD pipelines to wait until the build is done (and pushed to S3 for example). This otherwise becomes a problem of all clients to implement some waiting mechanism and error propagation (build errors, ...) becomes a bit harder to report.

[mefarazath]

@srenatus is this feature/improvement pending due to unanswered questions highlighted in "Next steps:" in the PR description?

[srenatus]

Yeah this needs some love. @mefarazath Do you have a similar use case? Would it help as implemented there? If so, I can put in some time this week to land this.

[srenatus]

@mjungsbluth Sorry for the radio silence! (Must have overlooked that notification... 😓)

I would add a configuration flag like bundle_rebuild_periodically: true/false to allow disabling any kind of intrinsic periodicity. This would allow to just horizontally scale and have more than once OCP instance running

Horizontal scaling is something we're definitely keen on with OCP. That said, if you configure your rebuild period to, say, 10 years, you'll have the same effect now. But obviously without an external trigger, nothing would ever happen.

Which is why I'll refresh this PR. 🙃

It would actually be nice if the trigger would block until the build is complete which would make it a lot simpler for clients in CI/CD pipelines to wait until the build is done (and pushed to S3 for example).

Yeah I can see that. Let's see what we can do.