Install connectors at build-time

[mfernest]

Chuck (@chuck-confluent) pointed out the connectors here are installed at runtime, which will impose an avoidable expense on the user. We could greatly improve the experience by letting the build handle this work.

[zdhrc]

Hey Michael, which lab/connectors are you referring to. I'm happy to change whatever

[chuck-confluent]

Hey zacharydhamilton, this is more of a general comment on using kafka connect in docker. Our recommended approach in the docs is to install connectors as a part of a custom Dockerfile based on the connect image. I recently did this for the SIEM demo. See:

• custom dockerfile
• simpler docker compose declaration, including healthcheck

There are a couple of benefits of doing this:

• More comprehensible to someone poking around trying to make sense of how things work
• If you run the demo multiple times, you aren't installing the same thing over and over every time

The benefits are enhanced when the demo moves off of the laptop onto a repeatable remote dev environment like https://gitpod.io where you can do things like configure prebuilds (e.g. docker containers are pre-built and pre-pulled).

[zdhrc]

Chuck (@chuck-confluent) I'm in agreement with you. The custom images used in these labs (unless I'm missing one) were made by installing the connectors during the image build. To save users of the labs a step in the process I've put the images on my docker hub so they can just pull them.

Is there a specific place I can take a look at?

[chuck-confluent]

zacharydhamilton for example:

• https://github.com/confluentinc/stream-me-up-scotty/blob/main/demos/messaging-modernization/docker-compose.yml#L88-L106

Compare that to:

• https://github.com/confluentinc/demo-siem-optimization/blob/main/docker-compose.yml#L52-L67

[zdhrc]

Chuck (@chuck-confluent) Fair enough! I will look into changing that. I didn't create that lab 😃 Thank you for bringing it to my attention