Redis destination connector: add support for custom element_id prefixes (#603)

Author: Paul-Cornell

snippets/destination_connectors/redis.sh.mdx

@@ -12,7 +12,10 @@ unstructured-ingest \
     --partition-endpoint $UNSTRUCTURED_API_URL \
     --additional-partition-args="{\"split_pdf_page\":\"true\", \"split_pdf_allow_failed\":\"true\", \"split_pdf_concurrency_level\": 15}" \
   redis \
-    --uri $REDIS_URI
+    --uri $REDIS_URI,
+    --no-ssl \
+    --key-prefix $REDIS_KEY_PREFIX \
+    --batch-size 100
 
 # Use Redis connection properties:
 unstructured-ingest \
@@ -30,4 +33,6 @@ unstructured-ingest \
     --database 0 \
     --username $REDIS_USERNAME \
     --password $REDIS_PASSWORD \
-    --no-ssl
+    --no-ssl \
+    --key-prefix $REDIS_KEY_PREFIX \
+    --batch-size 100

snippets/destination_connectors/redis.v2.py.mdx

@@ -57,5 +57,8 @@ if __name__ == "__main__":
             username=os.getenv("REDIS_USERNAME"),
             ssl=False
         ),
-        uploader_config=RedisUploaderConfig(batch_size=100)
+        uploader_config=RedisUploaderConfig(
+            batch_size=100
+            key_prefix=os.getenv("REDIS_KEY_PREFIX")
+        )
     ).run()

snippets/destination_connectors/redis_rest_create.mdx

@@ -12,6 +12,7 @@ curl --request 'POST' --location \
         "database": <database>,
         "ssl": <true|false>,
         "batch_size": <batch-size>,
+        "key_prefix": "<key-prefix>",
 
         # For URI authentication:
         "uri": "<uri>"

snippets/destination_connectors/redis_sdk.mdx

@@ -19,6 +19,7 @@ with UnstructuredClient(api_key_auth=os.getenv("UNSTRUCTURED_API_KEY")) as clien
                     database="<database>",
                     ssl=<True|False>,
                     batch_size=<batch-size>,
+                    key_prefix="<key-prefix>",
 
                     # For URI authentication:
                     uri="<uri>"

snippets/general-shared-text/redis-api-placeholders.mdx

@@ -6,4 +6,5 @@
 - `<password>` (_required_ for password authentication): The database user's password.
 - `<database>`: The number (index) for the target database. The default is `0` unless otherwise specified.
 - For `ssl`, `true` to enable SSL encryption for the connection. Set to `true` only if SSL is already set up and enabled for the target database. The default is `true` unless otherwise specified.
+- `<key-prefix>`: A string to prepend to each element's `element_id`. This is useful for distinguishing between different data sets in the same Redis instance. The default is an empty string.
 - `<batch-size>`: The maximum number of records to upload in a single batch. The default is `100` unless otherwise specified.

snippets/general-shared-text/redis-cli-api.mdx

@@ -22,4 +22,5 @@ The following environment variables:
   - `REDIS_PORT` - The database's port number, represented by `--port` (CLI) or `port` (Python).
   - `REDIS_DATABASE` - The database number of the target database, represented by `--database` (CLI) or `database` (Python).
   - `REDIS_USERNAME` - The username for the database, represented by `--username` (CLI) or `username` (Python).
-  - `REDIS_PASSWORD` - The user's password, represented by `--password` (CLI) or `password` (Python).
+  - `REDIS_PASSWORD` - The user's password, represented by `--password` (CLI) or `password` (Python).
+  - `REDIS_KEY_PREFIX` - A string to prepend to each element's `element_id`, represented by `--key-prefix` (CLI) or `key_prefix` (Python).

snippets/general-shared-text/redis-platform.mdx

@@ -8,4 +8,5 @@ Fill in the following fields:
 - **Password** (_required_ if **Password** is selected for **Authentication Method**): The database user's password.
 - **Database Index**: The number (index) for the target database. The default is `0` unless otherwise specified.
 - **SSL**: Check this box to enable SSL encryption for the connection. Check this box only if SSL is already set up and enabled for the target database. This box is checked by default.
+- **Key Prefix**: A string to prepend to each element's `element_id`. This is useful for distinguishing between different data sets in the same Redis instance. The default is an empty string.
 - **Batch Size**: The maximum number of records to upload in a single batch. The default is `100` unless otherwise specified.