destination/mongodb: adjust based on new auth approach (#609)

* adjust based on new auth approach

* add comments for temporary code

* add working defaults for the mongodb deployment

* add some documentation

Author: tim-quix

docker/mongodb/README.md

@@ -2,6 +2,21 @@
 
 This sample demonstrates how to deploy and use MongoDB in your pipeline. Please note: this image is provided by MongoDB and is offered as-is, with no specific support from Quix.
 
+## Using with a Quix Cloud MongoDB Connector
+
+This deployment will work seamlessly with the [Quix Cloud MongoDB sink connector](https://github.com/quixio/quix-samples/tree/main/python/destinations/mongodb).
+
+Simply provide the following arguments to the connector, 
+where `username` and `password` are the credentials used when 
+creating this service: 
+
+```shell
+MONGODB_USERNAME="<YOUR USERNAME>"  # (default: "admin")
+MONGODB_PASSWORD="<YOUR PASSWORD>"
+MONGODB_HOST="mongodb"
+MONGODB_PORT="27017"
+```
+
 ## How to Run
 
 1. Log in or sign up at [Quix](https://portal.platform.quix.io/signup?xlink=github) and navigate to the Code Samples section.

python/destinations/mongodb/README.md

@@ -4,6 +4,20 @@
 demonstrates how to consume data from a Kafka topic in Quix and write the data to a 
 MongoDB database using the [Quix Streams MongoDB sink](https://quix.io/docs/quix-streams/connectors/sinks/mongodb-sink.html).
 
+## Using with a Quix Cloud MongoDB Service
+
+This deployment will work seamlessly with a [Quix Cloud MongoDB service](https://github.com/quixio/quix-samples/tree/main/docker/mongodb).
+
+Simply provide the following arguments to this connector, 
+where `username` and `password` are are the credentials used when 
+creating the **Quix Cloud MongoDB service**: 
+
+```shell
+MONGODB_USERNAME="<YOUR USERNAME>"  # (default: "admin")
+MONGODB_PASSWORD="<YOUR PASSWORD>"
+MONGODB_HOST="mongodb"
+MONGODB_PORT="27017"
+```
 ## How to run
 
 Create a [Quix](https://portal.platform.quix.io/signup?xlink=github) account or log-in and visit the `Connectors` tab to use this connector.

python/destinations/mongodb/dockerfile

@@ -1,5 +1,12 @@
 FROM python:3.12.5-slim-bookworm
-			
+
+# TODO: remove this RUN block when done doing "@ git+" install in requirements.txt
+# This should be done BEFORE merging PR
+RUN apt-get update && \
+  apt-get install -y git && \
+  apt-get clean && \
+  rm -rf /var/lib/apt/lists/*
+
 # Set environment variables for non-interactive setup and unbuffered output
 ENV DEBIAN_FRONTEND=noninteractive \
     PYTHONUNBUFFERED=1 \
@@ -12,11 +19,6 @@ ARG MAINAPPPATH=.
 # Set working directory inside the container
 WORKDIR /app
 
-RUN apt-get update && \
-  apt-get install -y git && \
-  apt-get clean && \
-  rm -rf /var/lib/apt/lists/*
-
 # Copy requirements to leverage Docker cache
 COPY "${MAINAPPPATH}/requirements.txt" "${MAINAPPPATH}/requirements.txt"
 

python/destinations/mongodb/library.json

@@ -22,10 +22,26 @@
       "Required": true
     },
     {
-      "Name": "MONGODB_URL",
+      "Name": "MONGODB_HOST",
+      "Type": "EnvironmentVariable",
+      "InputType": "FreeText",
+      "Description": "MongoDB host name",
+      "DefaultValue": "mongodb",
+      "Required": true
+    },
+    {
+      "Name": "MONGODB_USERNAME",
+      "Type": "EnvironmentVariable",
+      "InputType": "FreeText",
+      "Description": "MongoDB username",
+      "DefaultValue": "admin",
+      "Required": true
+    },
+    {
+      "Name": "MONGODB_PASSWORD",
       "Type": "EnvironmentVariable",
       "InputType": "Secret",
-      "Description": "MongoDB url; most commonly mongodb://username:password@host:port",
+      "Description": "MongoDB password",
       "Required": true
     },
     {
@@ -42,6 +58,14 @@
       "Description": "MongoDB collection name",
       "Required": true
     },
+    {
+      "Name": "MONGODB_PORT",
+      "Type": "EnvironmentVariable",
+      "InputType": "FreeText",
+      "Description": "MongoDB host port",
+      "DefaultValue": "27017",
+      "Required": false
+    },
     {
       "Name": "MONGODB_DOCUMENT_MATCHER",
       "Type": "EnvironmentVariable",

python/destinations/mongodb/main.py

@@ -100,10 +100,13 @@ def document_matcher_env_parser() -> Optional[Callable[[SinkItem], MongoQueryFil
 kwargs_defaults = get_kwargs_defaults()
 mongodb_sink = MongoDBSink(
     # required settings
-    url=os.environ["MONGODB_URL"],
+    host=os.environ["MONGODB_HOST"],
+    username=os.environ["MONGODB_USERNAME"],
+    password=os.environ["MONGODB_PASSWORD"],
     db=os.environ["MONGODB_DB"],
     collection=os.environ["MONGODB_COLLECTION"],
     # optional settings (have defaults)
+    port=int(port) if (port := os.getenv("MONGODB_PORT")) else kwargs_defaults["port"],
     update_method=os.getenv("MONGODB_UPDATE_METHOD", kwargs_defaults["update_method"]),
     upsert=_as_bool(os.getenv("MONGODB_UPSERT", kwargs_defaults["upsert"])),
     document_matcher=document_matcher_env_parser() or kwargs_defaults["document_matcher"],

python/destinations/mongodb/requirements.txt

@@ -1,3 +1,3 @@
-quixstreams[mongodb] @ git+https://github.com/quixio/quix-streams.git@sink/mongodb-add-metadata-fix
-python-dotenv
-
+# TODO: remove "@ git+" version of install before merging PR
+quixstreams[mongodb] @ git+https://github.com/quixio/quix-streams.git@sink/mongodb-auth-adjust
+python-dotenv