Confluence source connector: now accepts username/password, username/API token, or PAT for authentication (#455)

Author: Paul-Cornell

snippets/general-shared-text/confluence-api-placeholders.mdx

@@ -1,7 +1,22 @@
 - `<name>` (_required_) - A unique name for this connector.
 - `<url>` (_required_) - The URL to the target Confluence Cloud instance.
-- `<user-email>` (_required_) - The email address of the user who has access to the instance.
-- `<api-token>` (_required_) - The Confluence API token that provides access to the instance.
 - `<max-num-of-spaces>` - The maximum number of Confluence spaces to access within the Confluence Cloud instance. The default is `500` unless otherwise specified.
 - `<max-num-of-docs-from-each-space>` - The maximum number of documents to access within each space. The default is `150` unless otherwise specified.
-- `spaces` is an array of strings, with each `<space-name>` specifying the name of a space to access, for example: `["luke","paul"]`. By default, if no space names are specified, and the `<max-num-of-spaces>` is exceeded for the instance, be aware that you might get unexpected results.
+- `spaces` is an array of strings, with each `<space-name>` specifying the name of a space to access, for example: `["luke","paul"]`. By default, if no space names are specified, and the `<max-num-of-spaces>` is exceeded for the instance, be aware that you might get unexpected results.
+
+For API token authentication:
+
+- `<username>` - The name or email address of the target user.
+- `<api-token>` - The user's API token value.
+- For `cloud`, `true` if you are using Confluence Cloud. The default is `false` if not otherwise specified.
+
+For personal access token (PAT) authentication:
+
+- `<personal-access-token>` - The target user's PAT value.
+- `cloud` should always be `false`.
+
+For password authentication:
+
+- `<username>` - The name or email address of the target user.
+- `<password>` - The user's password.
+- For `cloud`, `true` if you are using Confluence Cloud. The default is `false` if not otherwise specified.

snippets/general-shared-text/confluence-cli-api.mdx

@@ -10,16 +10,16 @@ import AdditionalIngestDependencies from '/snippets/general-shared-text/ingest-d
 
 The following environment variables:
 
-- `CONFLUENCE_URL` - The URL of the Confluence instance, represented by `--url` (CLI) or `url` (Python).
+- `CONFLUENCE_URL` - The target Confluence site's URL, represented by `--url` (CLI) or `url` (Python).
 - One of the following:
 
-  - `CONFLUENCE_API_TOKEN` - The value of the Confluence API token for authenticating with the Confluence instance, represented by `--api-token` (CLI) or `api_token` (Python).
-  - `CONFLUENCE_ACCESS_TOKEN` - The value of the Confluence personal access token for authenticating with the Confluence instance, represented by `--access-token` (CLI) or `access_token` (Python).
-
-- `CONFLUENCE_USER_EMAIL` - The user's email address for authenticating with the Confluence instance, represented by `--user-email` (CLI) or `user_email` (Python).
+  - For API token authentication: `CONFLUENCE_USERNAME` and `CONFLUENCE_TOKEN` - The name or email address,and API token of the target Confluence user, represented by `--username` (CLI) or `username` (Python) and `--token` (CLI) or `token` (Python), respectively.
+  - For personal access token (PAT) authentication: `CONFLUENCE_TOKEN` - The PAT for the target Confluence user, represented by `--token` (CLI) or `token` (Python).
+  - For password authentication: `CONFLUENCE_USERNAME` and `CONFLUENCE_PASSWORD` - The name or email address, and password of the target Confluence user, represented by `--username` (CLI) or `username` (Python) and `--password` (CLI) or `password` (Python), respectively.
 
 Additional settings include:
 
 - `--spaces` (CLI) or `spaces` (Python): Optionally, the list of the names of the specific spaces to access, expressed as a comma-separated list of strings (CLI) or an array of strings (Python), with each string representing a space's name. The default is no specific spaces, if not otherwise specified.
 - `--max-num-of-spaces` (CLI) or `max_num_of_spaces` (Python): Optionally, the maximum number of spaces to access, expressed as an integer. The default value is `500` if not otherwise specified.
 - `--max-num-of-docs-from-each-space` (CLI) or `max_num_of_docs_from_each_space` (Python): Optionally, the maximum number of documents to access from each space, expressed as an integer. The default value is `100` if not otherwise specified.
+- `--cloud` or `--no-cloud` (CLI) or `cloud` (Python): Optionally, whether to use Confluence Cloud (`--cloud` for CLI or `cloud=True` for Python). The default is `--no-cloud` (CLI) or `cloud=False` (Python) if not otherwise specified.

snippets/general-shared-text/confluence-platform.mdx

@@ -1,13 +1,13 @@
 Fill in the following fields:
 
 - **Name** (_required_): A unique name for this connector.
-- **URL** (_required_): The URL to the target Confluence Cloud instance.
-- **User Email** (_required_): The email address of the user who has access to the instance.
-- **API Token** (_required_): The Confluence API token that provides access to the instance.
-- **Max Number of Spaces**: The maximum number of Confluence spaces to access within the Confluence Cloud instance. 
+- **URL** (_required_): The target Confluence site's URL.
+- For personal access token (PAT) authentication: for **Authentication Method**, select **Personal Access Token**. Then enter the PAT into the **Personal Access Token** field.
+- For API token or password authentication: for **Authentication Method**, select **Password or API token**. Then enter the user's name or email address into the **Username** field and the API token or password into the **Password** field. Also, if you are using Confluence Cloud, check the **Cloud** box.
+- **Max number of spaces**: The maximum number of Confluence spaces to access within the Confluence Cloud instance. 
   The default is 500 unless otherwise specified.
-- **Max Number of Docs Per Space**: The maximum number of documents to access within each space. 
+- **Max number of docs per space**: The maximum number of documents to access within each space. 
   The default is 150 unless otherwise specified.
-- **List of Spaces**: A comma-separated string that lists the names of all of the spaces to access, for example: `luke,paul`. 
+- **List of spaces**: A comma-separated string that lists the names of all of the spaces to access, for example: `luke,paul`. 
   By default, if no space names are specified, and the **Max Number of Spaces** is reached for the instance, be aware that you might get 
   unexpected results.

snippets/general-shared-text/confluence.mdx

@@ -1,3 +1,27 @@
+- A [Confluence Cloud account](https://www.atlassian.com/software/confluence/pricing) or 
+  [Confluence Data Center installation](https://confluence.atlassian.com/doc/installing-confluence-data-center-203603.html).
+- The site URL for your [Confluence Cloud account](https://community.atlassian.com/t5/Confluence-questions/confluence-cloud-url/qaq-p/1157148) or 
+  [Confluence Data Center installation](https://confluence.atlassian.com/confkb/how-to-find-your-site-url-to-set-up-the-confluence-data-center-and-server-mobile-app-938025792.html).
+- A user in your [Confluence Cloud account](https://confluence.atlassian.com/cloud/invite-edit-and-remove-users-744721624.html) or 
+  [Confluence Data Center installation](https://confluence.atlassian.com/doc/add-and-invite-users-138313.html).
+- The user must have the correct permissions in your 
+  [Conflunce Cloud account](https://support.atlassian.com/confluence-cloud/docs/what-are-confluence-cloud-permissions-and-restrictions/) or 
+  [Confluence Data Center installation](https://confluence.atlassian.com/doc/permissions-and-restrictions-139557.html) to 
+  access the target spaces and pages.
+- One of the following:
+
+  - For Confluence Cloud or Confluence Data Center, the target user's name or email address, and password. 
+    [Change a Confluence Cloud user's password](https://support.atlassian.com/confluence-cloud/docs/change-your-confluence-password/). 
+    [Change a Confluence Data Center user's password](https://confluence.atlassian.com/doc/change-your-password-139416.html).
+  - For Confluence Cloud only, the target user's name or email address, and API token. 
+    [Create an API token](https://support.atlassian.com/atlassian-account/docs/manage-api-tokens-for-your-atlassian-account/).
+  - For Confluence Data Center only, the target user's personal access token (PAT). 
+    [Create a PAT](https://confluence.atlassian.com/enterprise/using-personal-access-tokens-1026032365.html).
+
+- Optionally, the names of the specific [spaces](https://support.atlassian.com/confluence-cloud/docs/navigate-spaces/) in the Confluence instance to access.
+
+The following video provides related setup information for Confluence Cloud:
+
 <iframe
 width="560"
 height="315"
@@ -6,16 +30,4 @@ title="YouTube video player"
 frameborder="0"
 allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture"
 allowfullscreen
-></iframe>
-
-- A [Confluence account](https://www.atlassian.com/software/confluence).
-- The URL for the target Confluence instance.
-- A [Confluence user](https://confluence.atlassian.com/doc/add-and-invite-users-138313.html) with 
-  the correct [permissions](https://support.atlassian.com/confluence-cloud/docs/what-are-confluence-cloud-permissions-and-restrictions/) to 
-  access the target spaces and pages in the Confluence instance.
-- One of the following:
-
-  - A [Confluence Cloud API token](https://support.atlassian.com/atlassian-account/docs/manage-api-tokens-for-your-atlassian-account/) for the target Confluence user.
-  - For Unstructured Ingest only, a [Confluence personal access token](https://confluence.atlassian.com/enterprise/using-personal-access-tokens-1026032365.html) for the target Confluence user.
-
-- Optionally, the names of the specific [spaces](https://support.atlassian.com/confluence-cloud/docs/navigate-spaces/) in the Confluence instance to access.
+></iframe>

snippets/source_connectors/confluence.sh.mdx

@@ -1,18 +1,52 @@
 ```bash CLI
 #!/usr/bin/env bash
 
+# For API token authentication:
 unstructured-ingest \
   confluence \
+    --token $CONFLUENCE_TOKEN \
     --url $CONFLUENCE_URL \
+    --username $CONFLUENCE_USERNAME \
+    --cloud \
     --spaces luke,paul \
-    --user-email $CONFLUENCE_USER_EMAIL \
-    --api-token $CONFLUENCE_API_TOKEN \
-    # Or:
-    # --access-token $CONFLUENCE_ACCESS_TOKEN \
+    --max-num-of-spaces 500 \
+    --max-num-of-docs-from-each-space 150 \
     --output-dir $LOCAL_FILE_OUTPUT_DIR \
     --partition-by-api \
     --api-key $UNSTRUCTURED_API_KEY \
     --partition-endpoint $UNSTRUCTURED_API_URL \
     --chunking-strategy by_title \
     --embedding-provider huggingface
-```
+
+# For personal access token (PAT) authentication:
+unstructured-ingest \
+  confluence \
+    --token $CONFLUENCE_TOKEN \
+    --url $CONFLUENCE_URL \
+    --spaces luke,paul \
+    --max-num-of-spaces 500 \
+    --max-num-of-docs-from-each-space 150 \
+    --output-dir $LOCAL_FILE_OUTPUT_DIR \
+    --partition-by-api \
+    --api-key $UNSTRUCTURED_API_KEY \
+    --partition-endpoint $UNSTRUCTURED_API_URL \
+    --chunking-strategy by_title \
+    --embedding-provider huggingface
+
+# For password authentication:
+unstructured-ingest \
+  confluence \
+    --password $CONFLUENCE_PASSWORD \
+    --url $CONFLUENCE_URL \
+    --username $CONFLUENCE_USERNAME \
+    --no-cloud \
+    --spaces luke,paul \
+    --max-num-of-spaces 500 \
+    --max-num-of-docs-from-each-space 150 \
+    --output-dir $LOCAL_FILE_OUTPUT_DIR \
+    --partition-by-api \
+    --api-key $UNSTRUCTURED_API_KEY \
+    --partition-endpoint $UNSTRUCTURED_API_URL \
+    --chunking-strategy by_title \
+    --embedding-provider huggingface
+```

snippets/source_connectors/confluence.v2.py.mdx

@@ -22,18 +22,40 @@ if __name__ == "__main__":
     Pipeline.from_configs(
         context=ProcessorConfig(),
         indexer_config=ConfluenceIndexerConfig(
-            spaces=["luke", "paul"]
+            spaces=["luke", "paul"],
+            max_num_of_spaces=500,
+            max_num_of_docs_from_each_space=150
         ),
         downloader_config=ConfluenceDownloaderConfig(download_dir=os.getenv("LOCAL_FILE_DOWNLOAD_DIR")),
+        
+        # For API token authentication:
         source_connection_config=ConfluenceConnectionConfig(
             access_config=ConfluenceAccessConfig(
-                api_token=os.getenv("CONFLUENCE_API_TOKEN")
-                # Or:
-                # access_token=os.getenv("CONFLUENCE_ACCESS_TOKEN")
+                token=os.getenv("CONFLUENCE_TOKEN")
             ),
             url=os.getenv("CONFLUENCE_URL"),
-            user_email=os.getenv("CONFLUENCE_USER_EMAIL")
+            username=os.getenv("CONFLUENCE_USERNAME"), 
+            cloud=True
         ),
+
+        # For personal access token (PAT) authentication:
+        # source_connection_config=ConfluenceConnectionConfig(
+        #     access_config=ConfluenceAccessConfig(
+        #         token=os.getenv("CONFLUENCE_TOKEN")
+        #     ),
+        #     url=os.getenv("CONFLUENCE_URL")
+        # ),
+
+        # For password authentication:
+        # source_connection_config=ConfluenceConnectionConfig(
+        #     access_config=ConfluenceAccessConfig(
+        #         password=os.getenv("CONFLUENCE_PASSWORD")                
+        #     ),
+        #     url=os.getenv("CONFLUENCE_URL"),
+        #     username=os.getenv("CONFLUENCE_USERNAME"), 
+        #     cloud=False
+        # ),
+
         partitioner_config=PartitionerConfig(
             partition_by_api=True,
             api_key=os.getenv("UNSTRUCTURED_API_KEY"),

snippets/source_connectors/confluence_rest_change.mdx

@@ -8,11 +8,26 @@ curl --request 'PUT' --location \
 '{
     "config": {
         "url": "<url>",
-        "user_email": "<user-email>",
-        "api_token": "<api-token>",
         "max_num_of_spaces": <max-num-of-spaces>,
         "max_num_of_docs_from_each_space": <max-num-of-docs-from-each-space>,
-        "spaces": ["<space-name>", "<space-name>"]
+        "spaces": ["<space-name>", "<space-name>"],
+        
+        # For API token authentication:
+
+        "username": "<username>",
+        "token": "<api-token>",
+        "cloud": "<true|false>"
+
+        # For personal access token (PAT) authentication:
+
+        "token": "<personal-access-token>",
+        "cloud": "false"
+
+        # For password authentication:
+
+        "username": "<username>",
+        "password": "<password>",
+        "cloud": "<true|false>"
     }
 }'
 ```

snippets/source_connectors/confluence_rest_create.mdx

@@ -10,11 +10,26 @@ curl --request 'POST' --location \
     "type": "confluence",
     "config": {
         "url": "<url>",
-        "user_email": "<user-email>",
-        "api_token": "<api-token>",
         "max_num_of_spaces": <max-num-of-spaces>,
         "max_num_of_docs_from_each_space": <max-num-of-docs-from-each-space>,
-        "spaces": ["<space-name>", "<space-name>"]
+        "spaces": ["<space-name>", "<space-name>"],
+
+        # For API token authentication:
+
+        "username": "<username>",
+        "token": "<api-token>",
+        "cloud": "<true|false>"
+
+        # For personal access token (PAT) authentication:
+
+        "token": "<personal-access-token>",
+        "cloud": "false"
+
+        # For password authentication:
+
+        "username": "<username>",
+        "password": "<password>",
+        "cloud": "<true|false>"        
     }
 }'
 ```