Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 41 additions & 0 deletions doc/technical_configuration.md
Original file line number Diff line number Diff line change
Expand Up @@ -94,6 +94,39 @@ KAFNUS_TESTS_MONGO_PORT: "27017"

> ✅ These environment variables are available to all sink connectors via `${env:...}` references thanks to the `config.providers=env` setting in the Kafnus Connect distributed configuration.

### 🔐 Security Configuration

Kafnus Connect supports Kafka authentication via SASL. You can configure security by setting the following environment variables in your Docker Compose file:

```yaml
# Security for Connect worker
CONNECT_SECURITY_PROTOCOL: SASL_PLAINTEXT
CONNECT_SASL_MECHANISM: PLAIN
CONNECT_SASL_JAAS_CONFIG: >
org.apache.kafka.common.security.plain.PlainLoginModule required
username="connect-user"
password="connect-pass";

# Security for producers and consumers
CONNECT_PRODUCER_SECURITY_PROTOCOL: SASL_PLAINTEXT
CONNECT_PRODUCER_SASL_MECHANISM: PLAIN
CONNECT_PRODUCER_SASL_JAAS_CONFIG: >
org.apache.kafka.common.security.plain.PlainLoginModule required
username="connect-user"
password="connect-pass";

CONNECT_CONSUMER_SECURITY_PROTOCOL: SASL_PLAINTEXT
CONNECT_CONSUMER_SASL_MECHANISM: PLAIN
CONNECT_CONSUMER_SASL_JAAS_CONFIG: >
org.apache.kafka.common.security.plain.PlainLoginModule required
username="connect-user"
password="connect-pass";
```

These variables are automatically applied by the `docker-entrypoint.sh` script when starting Kafka Connect in distributed mode. If `CONNECT_SECURITY_PROTOCOL` is defined, the script appends the corresponding security and SASL configuration to `connect-distributed.properties` for the worker, producer, and consumer.

This ensures that all Kafka connections (incoming and outgoing) respect the authentication settings without modifying connector definitions directly.

---

## 🧩 Kafnus Connect Plugins
Expand Down Expand Up @@ -709,3 +742,11 @@ docker exec -it kafka /opt/kafka/bin/kafka-console-consumer.sh
```

Check tables in PostGIS or MongoDB after running the corresponding test input.

---

## 📚 Operational & Advanced Topics

For complete operational guidance, multi-tenant management, and security best practices, please refer to the **Kafnus main repository**:

- [Advanced Topics](https://github.com/telefonicaid/kafnus/blob/main/doc/03_advanced_topics.md) – security and operational guide.
23 changes: 22 additions & 1 deletion docker-entrypoint.sh
Original file line number Diff line number Diff line change
Expand Up @@ -17,11 +17,14 @@
# You should have received a copy of the GNU Affero General Public License
# along with kafnus. If not, see http://www.gnu.org/licenses/.

#!/bin/bash
set -e

CONFIG_FILE="/home/appuser/config/connect-distributed.properties"

cat > "${CONFIG_FILE}" <<EOF
> "${CONFIG_FILE}"

cat >> "${CONFIG_FILE}" <<EOF
bootstrap.servers=${CONNECT_BOOTSTRAP_SERVERS:-kafka:9092}
group.id=${CONNECT_GROUP_ID:-connect-cluster}

Expand All @@ -46,6 +49,24 @@ config.providers=env
config.providers.env.class=org.apache.kafka.common.config.provider.EnvVarConfigProvider
EOF

# Security (optional)
if [ -n "${CONNECT_SECURITY_PROTOCOL}" ]; then
cat >> "${CONFIG_FILE}" <<EOF

security.protocol=${CONNECT_SECURITY_PROTOCOL}
sasl.mechanism=${CONNECT_SASL_MECHANISM}
sasl.jaas.config=${CONNECT_SASL_JAAS_CONFIG}

producer.security.protocol=${CONNECT_PRODUCER_SECURITY_PROTOCOL:-${CONNECT_SECURITY_PROTOCOL}}
producer.sasl.mechanism=${CONNECT_PRODUCER_SASL_MECHANISM:-${CONNECT_SASL_MECHANISM}}
producer.sasl.jaas.config=${CONNECT_PRODUCER_SASL_JAAS_CONFIG:-${CONNECT_SASL_JAAS_CONFIG}}

consumer.security.protocol=${CONNECT_CONSUMER_SECURITY_PROTOCOL:-${CONNECT_SECURITY_PROTOCOL}}
consumer.sasl.mechanism=${CONNECT_CONSUMER_SASL_MECHANISM:-${CONNECT_SASL_MECHANISM}}
consumer.sasl.jaas.config=${CONNECT_CONSUMER_SASL_JAAS_CONFIG:-${CONNECT_SASL_JAAS_CONFIG}}
EOF
fi

echo ">> Starting Kafka Connect with config:"
cat "${CONFIG_FILE}"

Expand Down