You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
title: 'Integrating Amazon Glue with ClickHouse and Spark'
8
8
---
9
-
9
+
import Image from '@theme/IdealImage';
10
10
import Tabs from '@theme/Tabs';
11
11
import TabItem from '@theme/TabItem';
12
+
import notebook_connections_config from '@site/static/images/integrations/data-ingestion/aws-glue/notebook-connections-config.png';
13
+
import dependent_jars_path_option from '@site/static/images/integrations/data-ingestion/aws-glue/dependent_jars_path_option.png';
12
14
13
-
# Integrating Amazon Glue with ClickHouse
15
+
# Integrating Amazon Glue with ClickHouse and Spark
14
16
15
17
[Amazon Glue](https://aws.amazon.com/glue/) is a fully managed, serverless data integration service provided by Amazon Web Services (AWS). It simplifies the process of discovering, preparing, and transforming data for analytics, machine learning, and application development.
16
18
19
+
# Installation
20
+
21
+
To integrate your Glue code with ClickHouse, you can use our official Spark connector in Glue via one of the following:
22
+
- Installing the ClickHouse Glue connector from the AWS Marketplace (recommended).
23
+
- Manually adding the Spark Connector's jars to your Glue job.
To access the connector in your account, subscribe to the ClickHouse AWS Glue Connector from AWS Marketplace.
30
+
31
+
2.### Grant Required Permissions
32
+
Ensure your Glue job’s IAM role has the necessary permissions, as described in the minimum privileges [guide](https://docs.aws.amazon.com/glue/latest/dg/getting-started-min-privs-job.html#getting-started-min-privs-connectors).
33
+
34
+
3.### Activate the Connector & Create a Connection
35
+
You can activate the connector and create a connection directly by clicking [this link](https://console.aws.amazon.com/gluestudio/home#/connector/add-connection?connectorName="ClickHouse%20AWS%20Glue%20Connector"&connectorType="Spark"&connectorUrl=https://709825985650.dkr.ecr.us-east-1.amazonaws.com/clickhouse/clickhouse-glue:0.1&connectorClassName="com.clickhouse.spark.ClickHouseCatalog"), which opens the Glue connector creation page with key fields pre-filled.
36
+
37
+
4.### Set Up a Connection
38
+
Create a new Glue connection using the connector, providing your ClickHouse JDBC URL and credentials.
39
+
40
+
5.### Use in Glue Job
41
+
In your Glue job, select the `Job details` tab, and expend the `Advanced properties` window. Under the `Connections` section, select the connection you just created. The connector automatically injects the required JARs into the job runtime.
0 commit comments