[Brigade] New Event Gateways

[nate-double-u]

Brigade event gateways are peripheral components that are installed alongside Brigade to receive events from external systems, transform those events into Brigade events, and enqueue them on Brigade's event bus. Most gateways are small, simple programs. On the more sophisticated end of the spectrum, gateways may go as far as utilizing the Brigade API to monitor the status of events they have created so they can report those statuses "upstream" to the original source of the event. This project invites candidates to propose and implement one or more new gateways.

Mentor(s): Kent Rancourt (@krancour)

https://github.com/cncf/mentoring/blob/main/summerofcode/2022.md#new-event-gateways

[krancour]

In brigadecore/brigade#1817, @AnuragThePathak asked:

Hello, I was thinking of working on this issue for GSOC 2022. I have gone through the gateway page of docs.
To learn more about developing gateways will I need to directly checkout codebase of any of the existing gateways or there are other resources available?

There are several existing gateways at various stages of maturity:

• Azure Container Registry - https://github.com/brigadecore/brigade-acr-gateway
• Bitbucket - https://github.com/brigadecore/brigade-bitbucket-gateway/tree/v2
• CloudEvents 1.0 - https://github.com/brigadecore/brigade-cloudevents-gateway
• Docker Hub - https://github.com/brigadecore/brigade-dockerhub-gateway
• GitHub - https://github.com/brigadecore/brigade-github-gateway
• Slack - https://github.com/brigadecore/brigade-slack-gateway

I know of someone, unrelated to GSoC, who has expressed interest in working on a Teams and Discord gateways, so consider the possibility that those might already be works-in-progress somewhere outside the @brigadecore organization.

There are also a couple other things that create events that aren't strictly considered gateways because they're not brokering events from some external source:

• Cron Event Source - https://github.com/brigadecore/brigade-cron-event-source
• Noisy Neighbor - https://github.com/brigadecore/brigade-noisy-neighbor

Looking at any one of these would give you the basic ideas behind how to send events to Brigade. Some of them are simple (to a fault, even). Some, like the GitHub and Slack gateways are more complex, and as long as you don't get bogged down in the GitHub-specific or Slack-specific aspects of them, you would find a nice pattern in those two that you can replicate. That pattern involves two components -- a receiver and a monitor. The receiver, as its name might imply, receives non-Brigade events from some external source and converts those to Brigade events and sends them to Brigade. The monitor follows up on events the receiver has created and reports status "upstream" to the original source of the event (if that's an applicable thing).

If I were to offer some non-technical advice to anyone interested in creating a gateway, it would be to shy away from anything CI/CD-related. The project is trying to overcome the mis-perception that it is a CI/CD platform. Brigade is a general-purpose automation platform. CI/CD is only one specialized case of automation, but because this is the use case that's been exercised the most (even by the maintainers), to date, it's created the misconception that that's what Brigade is. We'd really love to show off how Brigade is useful for other things.

Brigade can connect any arbitrary event to any arbitrary workflow, as long as someone is willing to write a gateway for that type of event.

Things that are imaginative and useful would be awesome!

[sudhanshu456]

Hi, I'm interested in teaming with @AnuragThePathak for writing more new gateway. I'm also checking docs to understand furthermore, I will go into examples that are illustrated in docs.

[krancour]

@sudhanshu456, have you discussed teaming up with @AnuragThePathak? Also, I'll be transparent that I don't know if GSoC permits teams. Maybe @nate-double-u knows.

[nate-double-u]

Hi @sudhanshu456, only an individual may work on a given project.

[krancour]

Thanks for clarifying that @nate-double-u!

[sudhanshu456]

thanks for the clarification, I wasn't sure of it. @nate-double-u

[AnuragThePathak]

Hi, is there any particular template and word limit for Brigade or CNCF proposals?

[krancour]

Nothing specific on the Brigade end.

[AnuragThePathak]

@krancour will the Twitter gateway be a useful one?
And what about GitLab? I thought that as Github is being mostly used nowadays so GitLab gateway won't be that useful. What's your opinion on that?
Also please feel free to suggest any other gateways if that's not a problem.

[krancour]

@AnuragThePathak GitLab would be cool. You're not the first to ask if that would be a good one.

Twitter also would be very interesting. In fact, I'll be open that I think that's more interesting than GitLab because it would showcase how Brigade can be used for things that aren't CI. (Brigade is too often mistaken for a CI platform. It can do CI, but it's a general purpose automation platform, so gateways that help enable other uses are especially welcome!)

[AnuragThePathak]

@krancour the Twitter webhooks (for enterprise and paid premium) require confirmation about receipt of the event. I think as it does not require the status of the event after receipt, the event is also not required to be followed up by a monitor in this case. Am I correct?
And even though we don't need a monitor for that purpose, should we create one for running health checks only?

[krancour]

The receiver/monitor pattern is only helpful for gateways where there's status to report upstream. GitHub is a great example because there are many webhooks where we receive them, schedule some async work, and then update GitHub about the success/failure of that work. This may not apply to Twitter, in which case there's no reason to have the monitor component.

One bit of caution though is that if all a Twitter gateway does is receive webhooks and turn them into Brigade events, this might be only a few hours of work -- not the 175 that I forecast for this project. (All GSoC projects are sized as 175 or 350.) With this being the case, we'd probably have to say no to a Twitter gateway as a GSoC project unless the proposal includes some justification for being a 175 hour project or if it was, perhaps, part of a bundle of several new gateways.

[AnuragThePathak]

Ok I've almost completed the proposal for Twitter gateway, and you'll be able to review it and judge if that's suitable for 175 hours of work.
Should I dm that to you in Slack or directly submit on GSoC site?

[krancour]

@AnuragThePathak sorry for the delay. I slacked you about it.

[AnuragThePathak]

no issues, i too have opened slack a few moments ago.