You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/en/integrations/data-ingestion/clickpipes/postgres/faq.md
+1-1Lines changed: 1 addition & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -43,7 +43,7 @@ Yes! ClickPipes for Postgres offers two ways to connect to databases in private
43
43
- us-east-1
44
44
- us-east-2
45
45
- eu-central-1
46
-
- For detailed setup instructions, see our [PrivateLink documentation](https://clickhouse.com/docs/knowledgebase/aws-privatelink-setup-for-clickpipes#requirements)
46
+
- For detailed setup instructions, see our [PrivateLink documentation](/docs/knowledgebase/aws-privatelink-setup-for-clickpipes#requirements)
47
47
- For regions where PrivateLink is not available, please use SSH tunneling
The VPC must be located in one of our ClickPipes regions: us-east-1, us-east-2 or eu-central-1
19
+
(/docs/en/integrations/clickpipes#list-of-static-ips), additionally the ClickHouse Instance must
20
+
be in the same region.
20
21
21
22
## Private link creation
22
23
23
-
Follow these steps to create a **VPC endpoint service** for your RDS instance. Repeat these steps if you have multiple instances that require endpoint services:
24
+
Follow these steps to create a **VPC endpoint service** for your RDS instance. Repeat these steps if you have multiple
25
+
RDS instances that require endpoint services (OR you may have different listener ports for different instances):
24
26
25
27
1. Locate Your VPC and [Create an NLB](https://docs.aws.amazon.com/elasticloadbalancing/latest/network/create-network-load-balancer.html)
26
-
- Navigate to your target VPC and create a Network Load Balancer (NLB).
28
+
- Navigate to your target VPC and create a Network Load Balancer (NLB). Note that the NLB should be internal (private) and not internet-facing (public).
27
29
28
30
2. Configure the Target Group
29
31
- The target group should point to the RDS instance's endpoint IP and Port (typically 5432 for PostgreSQL or 3306 for MySQL).
30
-
- Ensure that the TCP protocol is used to avoid TLS termination by the NLB.
32
+
33
+
:::note
34
+
35
+
If you would like to automate the process of updating the target group with the new RDS endpoint IP, you can use AWS Lambda functions or other automation tools.
36
+
One of the terraform modules that can be used for this purpose is [this](https://github.com/MaterializeInc/terraform-aws-rds-privatelink).
37
+
38
+
:::
31
39
-**IMPORTANT**: Make sure the RDS instance endpoint used in case of DB Cluster/Aurora is ONLY the WRITER Endpoint and NOT the common endpoint.
40
+
- Ensure that the TCP protocol is used to avoid TLS termination by the NLB.
32
41
33
42
3. Set the Listener Port
34
43
- The listener port of the load balancer must match the port used by the target group (typically 5432 for PostgreSQL or 3306 for MySQL).
35
44
36
-
4. Ensure the Load Balancer is Private
37
-
- Configure the NLB to be private, ensuring it is only accessible within the VPC.
38
-
39
-
5. Create the VPC Endpoint Service
45
+
4. Create the VPC Endpoint Service
40
46
- In the VPC, create an endpoint service that points to the NLB.
41
47
- Enable acceptance of connection requests from specific accounts.
42
48
43
-
6. Authorize ClickPipes to Use the Endpoint Service
49
+
5. Authorize ClickPipes to Use the Endpoint Service
44
50
- Grant permission to the ClickPipes account to request this endpoint service.
45
51
- Configure allowed principals by adding the following principal ID:
46
52
```
47
53
arn:aws:iam::072088201116:root
48
54
```
49
55
56
+
6. Disable "Enforce Security Group Inbound Rules on Private Link Traffic" on the NLB (if a security group is attached to the NLB)
57
+
- Navigate to the NLB's settings and disable the "Enforce Security Group Inbound Rules on Private Link Traffic" setting if a security group is attached to the NLB.
58
+
- If using Terraform, set the `enforce_security_group_inbound_rules_on_private_link_traffic` attribute to `false` for the NLB
59
+
- This setting is **required** to allow traffic from the ClickPipes VPC to the NLB.
60
+
50
61
## Initiating connection
51
62
52
-
When it's done, share details such as private DNS name, VPC service name and availability zone.
63
+
When it's done, share details such as AWS region, VPC service name and availability zone.
53
64
ClickPipes team will initiate VPC endpoints creation in ClickPipes VPC. This will require [connection request acceptance](https://docs.aws.amazon.com/vpc/latest/privatelink/configure-endpoint-service.html#accept-reject-connection-requests) on your side.
54
65
55
66
## Creating ClickPipes
56
67
57
-
Use your RDS's private DNS endpoints to create your ClickPipes.
68
+
ClickPipes team will provide the DNS name to be used in the ClickPipe creation process. You can now [create your ClickPipe](/docs/en/integrations/clickpipes/postgres).
69
+
70
+
## Dynamically updating the RDS endpoint IP
71
+
72
+
When the RDS endpoint IP changes (in case of restarts/failovers/updates), you need to update the NLB target group with
73
+
the new IP. You can automate this process using AWS Lambda functions or other automation tools.
0 commit comments