Skip to content

Commit 8c05e88

Browse files
authored
Platform API: Workflow Endpoint - Python SDK - strongly typed connector classes (#503)
1 parent 3253e50 commit 8c05e88

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

51 files changed

+621
-432
lines changed

platform-api/api/destinations/motherduck.mdx

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -12,10 +12,12 @@ import MotherDuckPrerequisites from '/snippets/general-shared-text/motherduck.md
1212

1313
To create or change a MotherDuck destination connector, see the following examples.
1414

15+
import MotherDuckSDK from '/snippets/destination_connectors/motherduck_sdk.mdx';
1516
import MotherDuckAPIRESTCreate from '/snippets/destination_connectors/motherduck_rest_create.mdx';
1617
import MotherDuckAPIRESTChange from '/snippets/destination_connectors/motherduck_rest_change.mdx';
1718

1819
<CodeGroup>
20+
<MotherDuckSDK />
1921
<MotherDuckAPIRESTCreate />
2022
<MotherDuckAPIRESTChange />
2123
</CodeGroup>

platform-api/api/destinations/overview.mdx

Lines changed: 21 additions & 21 deletions
Original file line numberDiff line numberDiff line change
@@ -18,25 +18,25 @@ To use the [Unstructured Platform Workflow Endpoint](/platform-api/api/overview)
1818
To create or update a destination connector, you must also provide settings that are specific to that connector.
1919
For the list of specific settings, see:
2020

21-
- [Astra DB](/platform-api/api/destinations/astradb) (`astradb`)
22-
- [Azure AI Search](/platform-api/api/destinations/azure-ai-search) (`azure_ai_search`)
23-
- [Couchbase](/platform-api/api/destinations/couchbase) (`couchbase`)
24-
- [Databricks Volumes](/platform-api/api/destinations/databricks-volumes) (`databricks_volumes`)
25-
- [Delta Tables in Amazon S3](/platform-api/api/destinations/delta-table) (`delta_table`)
26-
- [Delta Tables in Databricks](/platform-api/api/destinations/databricks-delta-table) (`databricks_volume_delta_tables`)
27-
- [Elasticsearch](/platform-api/api/destinations/elasticsearch) (`elasticsearch`)
28-
- [Google Cloud Storage](/platform-api/api/destinations/google-cloud) (`gcs`)
29-
- [Kafka](/platform-api/api/destinations/kafka) (`kafka-cloud`)
30-
- [Milvus](/platform-api/api/destinations/milvus) (`milvus`)
31-
- [MongoDB](/platform-api/api/destinations/mongodb) (`mongodb`)
32-
- [MotherDuck](/platform-api/api/destinations/motherduck) (`motherduck`)
33-
- [Neo4j](/platform-api/api/destinations/neo4j) (`neo4j`)
34-
- [OneDrive](/platform-api/api/destinations/onedrive) (`onedrive`)
35-
- [Pinecone](/platform-api/api/destinations/pinecone) (`pinecone`)
36-
- [PostgreSQL](/platform-api/api/destinations/postgresql) (`postgres`)
37-
- [Qdrant](/platform-api/api/destinations/qdrant) (`qdrant-cloud`)
38-
- [Redis](/platform-api/api/destinations/redis) (`redis`)
39-
- [Snowflake](/platform-api/api/destinations/snowflake) (`snowflake`)
40-
- [S3](/platform-api/api/destinations/s3) (`s3`)
41-
- [Weaviate](/platform-api/api/destinations/weaviate) (`weaviate`)
21+
- [Astra DB](/platform-api/api/destinations/astradb) (`ASTRADB` for the Python SDK or `astradb` for `curl` or Postman)
22+
- [Azure AI Search](/platform-api/api/destinations/azure-ai-search) (`AZURE_AI_SEARCH` for the Python SDK or `azure_ai_search` for `curl` or Postman)
23+
- [Couchbase](/platform-api/api/destinations/couchbase) (`COUCHBASE` for the Python SDK or `couchbase` for `curl` or Postman)
24+
- [Databricks Volumes](/platform-api/api/destinations/databricks-volumes) (`DATABRICKS_VOLUMES` for the Python SDK or `databricks_volumes` for `curl` or Postman)
25+
- [Delta Tables in Amazon S3](/platform-api/api/destinations/delta-table) (`DELTA_TABLE` for the Python SDK or `delta_table` for `curl` or Postman)
26+
- [Delta Tables in Databricks](/platform-api/api/destinations/databricks-delta-table) (`DATABRICKS_VOLUME_DELTA_TABLES` for the Python SDK or `databricks_volume_delta_tables` for `curl` or Postman)
27+
- [Elasticsearch](/platform-api/api/destinations/elasticsearch) (`ELASTICSEARCH` for the Python SDK or `elasticsearch` for `curl` or Postman)
28+
- [Google Cloud Storage](/platform-api/api/destinations/google-cloud) (`GCS` for the Python SDK or `gcs` for `curl` or Postman)
29+
- [Kafka](/platform-api/api/destinations/kafka) (`KAFKA_CLOUD` for the Python SDK or `kafka-cloud` for `curl` or Postman)
30+
- [Milvus](/platform-api/api/destinations/milvus) (`MILVUS` for the Python SDK or `milvus` for `curl` or Postman)
31+
- [MongoDB](/platform-api/api/destinations/mongodb) (`MONGODB` for the Python SDK or `mongodb` for `curl` or Postman)
32+
- [MotherDuck](/platform-api/api/destinations/motherduck) (`MOTHERDUCK` for the Python SDK or `motherduck` for `curl` or Postman)
33+
- [Neo4j](/platform-api/api/destinations/neo4j) (`NEO4J` for the Python SDK or `neo4j` for `curl` or Postman)
34+
- [OneDrive](/platform-api/api/destinations/onedrive) (`ONEDRIVE` for the Python SDK or `onedrive` for `curl` or Postman)
35+
- [Pinecone](/platform-api/api/destinations/pinecone) (`PINECONE` for the Python SDK or `pinecone` for `curl` or Postman)
36+
- [PostgreSQL](/platform-api/api/destinations/postgresql) (`POSTGRES` for the Python SDK or `postgres` for `curl` or Postman)
37+
- [Qdrant](/platform-api/api/destinations/qdrant) (`QDRANT_CLOUD` for the Python SDK or `qdrant-cloud` for `curl` or Postman)
38+
- [Redis](/platform-api/api/destinations/redis) (`REDIS` for the Python SDK or `redis` for `curl` or Postman)
39+
- [Snowflake](/platform-api/api/destinations/snowflake) (`SNOWFLAKE` for the Python SDK or `snowflake` for `curl` or Postman)
40+
- [S3](/platform-api/api/destinations/s3) (`S3` for the Python SDK or `s3` for `curl` or Postman)
41+
- [Weaviate](/platform-api/api/destinations/weaviate) (`WEAVIATE` for the Python SDK or `weaviate` for `curl` or Postman)
4242

platform-api/api/sources/overview.mdx

Lines changed: 18 additions & 18 deletions
Original file line numberDiff line numberDiff line change
@@ -18,23 +18,23 @@ To use the [Unstructured Platform Workflow Endpoint](/platform-api/api/overview)
1818
To create or update a source connector, you must also provide settings that are specific to that connector.
1919
For the list of specific settings, see:
2020

21-
- [Azure](/platform-api/api/sources/azure-blob-storage) (`azure`)
22-
- [Box](/platform-api/api/sources/box) (`box`)
23-
- [Confluence](/platform-api/api/sources/confluence) (`confluence`)
24-
- [Couchbase](/platform-api/api/sources/couchbase) (`couchbase`)
25-
- [Databricks Volumes](/platform-api/api/sources/databricks-volumes) (`databricks_volumes`)
26-
- [Dropbox](/platform-api/api/sources/dropbox) (`dropbox`)
27-
- [Elasticsearch](/platform-api/api/sources/elasticsearch) (`elasticsearch`)
28-
- [Google Cloud Storage](/platform-api/api/sources/google-cloud) (`gcs`)
29-
- [Google Drive](/platform-api/api/sources/google-drive) (`google_drive`)
30-
- [Kafka](/platform-api/api/sources/kafka) (`kafka-cloud`)
31-
- [MongoDB](/platform-api/api/sources/mongodb) (`mongodb`)
32-
- [OneDrive](/platform-api/api/sources/onedrive) (`onedrive`)
33-
- [Outlook](/platform-api/api/sources/outlook) (`outlook`)
34-
- [PostgreSQL](/platform-api/api/sources/postgresql) (`postgres`)
35-
- [S3](/platform-api/api/sources/s3) (`s3`)
36-
- [Salesforce](/platform-api/api/sources/salesforce) (`salesforce`)
37-
- [SharePoint](/platform-api/api/sources/sharepoint) (`sharepoint`)
38-
- [Snowflake](/platform-api/api/sources/snowflake) (`snowflake`)
21+
- [Azure](/platform-api/api/sources/azure-blob-storage) (`AZURE` for the Python SDK or `azure` for `curl` and Postman)
22+
- [Box](/platform-api/api/sources/box) (`BOX` for the Python SDK or `box` for `curl` and Postman)
23+
- [Confluence](/platform-api/api/sources/confluence) (`CONFLUENCE` for the Python SDK or `confluence` for `curl` and Postman)
24+
- [Couchbase](/platform-api/api/sources/couchbase) (`COUCHBASE` for the Python SDK or `couchbase` for `curl` and Postman)
25+
- [Databricks Volumes](/platform-api/api/sources/databricks-volumes) (`DATABRICKS_VOLUMES` for the Python SDK or `databricks_volumes` for `curl` and Postman)
26+
- [Dropbox](/platform-api/api/sources/dropbox) (`DROPBOX` for the Python SDK or `dropbox` for `curl` and Postman)
27+
- [Elasticsearch](/platform-api/api/sources/elasticsearch) (`ELASTICSEARCH` for the Python SDK or `elasticsearch` for `curl` and Postman)
28+
- [Google Cloud Storage](/platform-api/api/sources/google-cloud) (`GCS` for the Python SDK or `gcs` for `curl` and Postman)
29+
- [Google Drive](/platform-api/api/sources/google-drive) (`GOOGLE_DRIVE` for the Python SDK or `google_drive` for `curl` and Postman)
30+
- [Kafka](/platform-api/api/sources/kafka) (`KAFKA_CLOUD` for the Python SDK or `kafka-cloud` for `curl` and Postman)
31+
- [MongoDB](/platform-api/api/sources/mongodb) (`MONGODB` for the Python SDK or `mongodb` for `curl` and Postman)
32+
- [OneDrive](/platform-api/api/sources/onedrive) (`ONEDRIVE` for the Python SDK or `onedrive` for `curl` and Postman)
33+
- [Outlook](/platform-api/api/sources/outlook) (`OUTLOOK` for the Python SDK or `outlook` for `curl` and Postman)
34+
- [PostgreSQL](/platform-api/api/sources/postgresql) (`POSTGRES` for the Python SDK or `postgres` for `curl` and Postman)
35+
- [S3](/platform-api/api/sources/s3) (`S3` for the Python SDK or `s3` for `curl` and Postman)
36+
- [Salesforce](/platform-api/api/sources/salesforce) (`SALESFORCE` for the Python SDK or `salesforce` for `curl` and Postman)
37+
- [SharePoint](/platform-api/api/sources/sharepoint) (`SHAREPOINT` for the Python SDK or `sharepoint` for `curl` and Postman)
38+
- [Snowflake](/platform-api/api/sources/snowflake) (`SNOWFLAKE` for the Python SDK or `snowflake` for `curl` and Postman)
3939

4040

Lines changed: 13 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,17 +1,21 @@
11
```python Python SDK
22
# ...
3-
from unstructured_client.models.shared import <Create|Update>DestinationConnector
3+
from unstructured_client.models.shared import (
4+
<Create|Update>DestinationConnector,
5+
DestinationConnectorType,
6+
AstraDBConnectorConfigInput
7+
)
48
# ...
59
destination_connector = <Create|Update>DestinationConnector(
610
name="<name>", # Create only.
7-
type="astradb", # Create only.
8-
config={
9-
"token": "<token>",
10-
"api_endpoint": "<api-endpoint>",
11-
"collection_name": "<collection-name>",
12-
"keyspace": "<keyspace>",
13-
"batch_size": <batch-size>
14-
}
11+
type=DestinationConnectorType.ASTRADB, # Create only.
12+
config=AstraDBConnectorConfigInput(
13+
token="<token>",
14+
api_endpoint="<api-endpoint>",
15+
collection_name="<collection-name>",
16+
keyspace="<keyspace>",
17+
batch_size=<batch-size>
18+
)
1519
)
1620
# ...
1721
```
Lines changed: 11 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,15 +1,19 @@
11
```python Python SDK
22
# ...
3-
from unstructured_client.models.shared import <Create|Update>DestinationConnector
3+
from unstructured_client.models.shared import (
4+
<Create|Update>DestinationConnector,
5+
DestinationConnectorType,
6+
AzureAISearchConnectorConfigInput
7+
)
48
# ...
59
destination_connector = <Create|Update>DestinationConnector(
610
name="<name>", # Create only.
7-
type="azure_ai_search", # Create only.
8-
config={
9-
"endpoint": "<endpoint>",
10-
"index": "<index>",
11-
"azure_ai_search_key": "<azure-ai-search-key>"
12-
}
11+
type=DestinationConnectorType.AZURE_AI_SEARCH, # Create only.
12+
config=AzureAISearchConnectorConfigInput(
13+
endpoint="<endpoint>",
14+
index="<index>",
15+
key="<azure-ai-search-key>"
16+
)
1317
)
1418
# ...
1519
```

snippets/destination_connectors/couchbase_rest_change.mdx

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -13,8 +13,7 @@ curl --request 'PUT' --location \
1313
"scope": "<scope>",
1414
"collection": "<collection>",
1515
"password": "<password>",
16-
"batch_size": <batch-size>,
17-
"collection_id": "<collection-id>"
16+
"batch_size": <batch-size>
1817
}
1918
}'
2019
```

snippets/destination_connectors/couchbase_rest_create.mdx

Lines changed: 1 addition & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -15,8 +15,7 @@ curl --request 'POST' --location \
1515
"scope": "<scope>",
1616
"collection": "<collection>",
1717
"password": "<password>",
18-
"batch_size": <batch-size>,
19-
"collection_id": "<collection-id>"
18+
"batch_size": <batch-size>
2019
}
2120
}'
2221
```
Lines changed: 15 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -1,20 +1,23 @@
11
```python Python SDK
22
# ...
3-
from unstructured_client.models.shared import <Create|Update>DestinationConnector
3+
from unstructured_client.models.shared import (
4+
<Create|Update>DestinationConnector,
5+
DestinationConnectorType,
6+
CouchbaseDestinationConnectorConfigInput
7+
)
48
# ...
59
destination_connector = <Create|Update>DestinationConnector(
610
name="<name>", # Create only.
7-
type="couchbase", # Create only.
8-
config={
9-
"username": "<username>",
10-
"bucket": "<bucket>",
11-
"connection_string": "<connection-string>",
12-
"scope": "<scope>",
13-
"collection": "<collection>",
14-
"password": "<password>",
15-
"batch_size": <batch-size>,
16-
"collection_id": "<collection-id>"
17-
}
11+
type=estinationConnectorType.COUCHBASE, # Create only.
12+
config=CouchbaseDestinationConnectorConfigInput(
13+
username="<username>",
14+
bucket="<bucket>",
15+
connection_string="<connection-string>",
16+
scope="<scope>",
17+
collection="<collection>",
18+
password="<password>",
19+
batch_size=<batch-size>
20+
)
1821
)
1922
# ...
2023
```
Lines changed: 19 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -1,23 +1,27 @@
11
```python Python SDK
22
# ...
3-
from unstructured_client.models.shared import <Create|Update>DestinationConnector
3+
from unstructured_client.models.shared import (
4+
<Create|Update>DestinationConnector,
5+
DestinationConnectorType,
6+
DatabricksVDTDestinationConnectorConfigInput
7+
)
48
# ...
59
destination_connector = <Create|Update>DestinationConnector(
610
name="<name>", # Create only.
7-
type="databricks_volume_delta_tables", # Create only.
8-
config={
9-
"server_hostname": "<server-hostname>",
10-
"http_path": "<http-path>",
11-
"token": "<token>",
12-
"client_id": "<client-id>",
13-
"client_secret": "<client-secret>",
14-
"volume": "<volume>",
15-
"catalog": "<catalog>",
16-
"volume_path": "<volume-path>",
17-
"schema": "<schema>",
18-
"database": "<database>",
19-
"table_name": "<table-name>"
20-
}
11+
type=DestinationConnectorType.DATABRICKS_VOLUME_DELTA_TABLES, # Create only.
12+
config=DatabricksVDTDestinationConnectorConfigInput(
13+
server_hostname="<server-hostname>",
14+
http_path="<http-path>",
15+
token="<token>",
16+
client_id="<client-id>",
17+
client_secret="<client-secret>",
18+
volume="<volume>",
19+
catalog="<catalog>",
20+
volume_path="<volume-path>",
21+
schema="<schema>",
22+
database="<database>",
23+
table_name="<table-name>"
24+
)
2125
)
2226
# ...
2327
```
Lines changed: 15 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,23 @@
11
```python Python SDK
22
# ...
3-
from unstructured_client.models.shared import <Create|Update>DestinationConnector
3+
from unstructured_client.models.shared import (
4+
<Create|Update>DestinationConnector,
5+
DestinationConnectorType,
6+
DatabricksVolumesConnectorConfigInput
7+
)
48
# ...
59
destination_connector = <Create|Update>DestinationConnector(
610
name="<name>", # Create only.
7-
type="databricks_volumes", # Create only.
8-
config={
9-
"host": "<host>",
10-
"catalog": "<catalog>",
11-
"schema": "<schema>",
12-
"volume": "<volume>",
13-
"volume_path": "<volume-path>",
14-
"client_secret": "<client-secret>",
15-
"client_id": "<client-id>"
16-
}
11+
type=DestinationConnectorType.DATABRICKS_VOLUMES, # Create only.
12+
config=DatabricksVolumesConnectorConfigInput(
13+
host="<host>",
14+
catalog="<catalog>",
15+
schema="<schema>",
16+
volume="<volume>",
17+
volume_path="<volume-path>",
18+
client_secret="<client-secret>",
19+
client_id="<client-id>"
20+
)
1721
)
1822
# ...
1923
```

0 commit comments

Comments
 (0)