You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/reference/ingestion-tools/search-connectors/index.md
+8-11Lines changed: 8 additions & 11 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,18 +7,18 @@ mapped_pages:
7
7
8
8
# Search connectors
9
9
10
-
% Internal links rely on the following IDs being on this page (e.g. as a heading ID, paragraph ID, etc):
11
-
12
10
$$$es-connectors-native$$$
13
11
14
12
15
-
:::{tip}
16
-
This page is about Search connectors that synchronize third-party data into Elasticsearch. If you’re looking for Kibana connectors to integrate with services like generative AI model providers, refer to [Kibana Connectors](docs-content://deploy-manage/manage-connectors.md).
13
+
:::{note}
14
+
This page is about Search connectors that synchronize third-party data into {{es}}. If you’re looking for Kibana connectors to integrate with services like generative AI model providers, refer to [Kibana Connectors](docs-content://deploy-manage/manage-connectors.md).
17
15
:::
18
16
19
-
A _connector_ is a type of [Elastic integration](https://www.elastic.co/integrations/data-integrations) that syncs data from an original data source to Elasticsearch. Each connector extracts the original files, records, or objects; and transforms them into documents within Elasticsearch.
17
+
A _connector_ is an Elastic integration that syncs data from an original data source to {{es}}. Use connectors to create searchable, read-only replicas of your data in {{es}}.
18
+
19
+
Each connector extracts the original files, records, or objects; and transforms them into documents within {{es}}.
20
20
21
-
_Connector clients_ are **self-managed** connectors that you run on your own infrastructure. These connectors are written in Python and the source code is available in the [`elastic/connectors`](https://github.com/elastic/connectors/tree/main/connectors/sources) repo.
21
+
These connectors are written in Python and the source code is available in the [`elastic/connectors`](https://github.com/elastic/connectors/tree/main/connectors/sources) repo.
22
22
23
23
## Available connectors
24
24
@@ -28,9 +28,7 @@ As of Elastic 9.0, managed connectors on Elastic Cloud Hosted are no longer avai
28
28
::::
29
29
30
30
31
-
Connector clients are available for the following third-party data sources:
32
-
33
-
The following connectors are available as self-managed connectors:
31
+
Connectors are available for the following third-party data sources:
@@ -86,7 +84,6 @@ In order to set up, configure, and run a connector you’ll be moving between yo
86
84
87
85
### Data source prerequisites
88
86
89
-
90
-
The first decision you need to make before deploying a connector is which third party service (data source) you want to sync to Elasticsearch. See the list of [available connectors](#available-connectors).
87
+
The first decision you need to make before deploying a connector is which third party service (data source) you want to sync to {{es}}. See the list of [available connectors](#available-connectors).
91
88
92
89
Note that each data source will have specific prerequisites you’ll need to meet to authorize the connector to access its data. For example, certain data sources may require you to create an OAuth application, or create a service account. You’ll need to check the [individual connector documentation](connector-reference.md) for these details.
Copy file name to clipboardExpand all lines: docs/reference/ingestion-tools/search-connectors/self-managed-connectors.md
+10-20Lines changed: 10 additions & 20 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -10,22 +10,15 @@ Self-managed connectors were initially known as "connector clients". You might f
10
10
11
11
::::
12
12
13
-
14
13
Self-managed [Elastic connectors](/reference/ingestion-tools/search-connectors/index.md) are run on your own infrastructure. This means they run outside of your Elastic deployment.
15
14
16
-
You can run the [connectors service](#es-connectors-deploy-connector-service) from source or from a Docker container.
17
-
18
-
We also have a quickstart option using **Docker Compose**, to spin up all the required services at once: Elasticsearch, Kibana, and the connectors service. Refer to [Docker Compose quickstart](/reference/ingestion-tools/search-connectors/es-connectors-docker-compose-quickstart.md) for more information.
19
-
20
-
21
15
## Availability and Elastic prerequisites [es-build-connector-prerequisites]
22
16
23
17
::::{note}
24
18
Self-managed connectors currently don’t support Windows. Use this [compatibility matrix](https://www.elastic.co/support/matrix#matrix_os) to check which operating systems are supported by self-managed connectors. Find this information under **self-managed connectors** on that page.
25
19
26
20
::::
27
21
28
-
29
22
:::::{dropdown} Expand for Elastic prerequisites information
30
23
Your Elastic deployment must include the following Elastic services:
31
24
@@ -63,17 +56,16 @@ Note the following information regarding support for self-managed connectors:
The first decision you need to make before deploying a connector is which third party service (data source) you want to sync to Elasticsearch. Note that each data source will have specific prerequisites you’ll need to meet to authorize the connector to access its data. For example, certain data sources may require you to create an OAuth application, or create a service account.
71
-
72
-
You’ll need to check the individual connector documentation for these details.
73
-
74
-
::::
75
-
61
+
In order to set up, configure, and run a connector you’ll be moving between your third-party service, the Elastic UI, and your terminal. At a high-level, the workflow looks like this:
76
62
63
+
1. Satisfy any data source prerequisites (e.g., create an OAuth application).
64
+
2. Create a connector in the UI (or via the API).
65
+
3. Deploy the connector service:
66
+
-[Option 1: Run with Docker](es-connectors-run-from-docker.md) (recommended)
67
+
-[Option 2: Run from source](es-connectors-run-from-source.md)
68
+
4. Enter data source configuration details in the UI.
77
69
78
70
## Deploy the connector service [es-connectors-deploy-connector-service]
79
71
@@ -87,16 +79,14 @@ You can run the connector service from source or use Docker:
87
79
* Refer to our [Docker Compose quickstart](/reference/ingestion-tools/search-connectors/es-connectors-docker-compose-quickstart.md) for a quick way to spin up all the required services at once.
88
80
89
81
90
-
91
82
## Tutorials [es-build-connector-example]
92
83
93
84
* Follow our [UI-based tutorial](/reference/ingestion-tools/search-connectors/es-postgresql-connector-client-tutorial.md) to learn how run the self-managed connector service and a set up a self-managed connector, **using the UI**.
94
85
* Follow our [API-based tutorial](/reference/ingestion-tools/search-connectors/api-tutorial.md) to learn how to set up a self-managed connector **using the**[**connector APIs**](https://www.elastic.co/docs/api/doc/elasticsearch/group/endpoint-connector).
95
86
96
87
These examples use the PostgreSQL connector but the basic process is the same for all self-managed connectors.
97
88
98
-
99
-
## Connector testing [es-build-connector-testing]
89
+
## E2E testing [es-build-connector-testing]
100
90
101
91
The connector framework enables you to run end-to-end (E2E) tests on your self-managed connectors, against a real data source.
102
92
@@ -105,7 +95,7 @@ To avoid tampering with a real Elasticsearch instance, E2E tests run an isolated
105
95
E2E tests use **default** configuration values for the connector. Find instructions about testing in each connector’s documentation.
0 commit comments