You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: docs/integrations/data-ingestion/clickpipes/postgres/faq.md
+30-1Lines changed: 30 additions & 1 deletion
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -146,8 +146,37 @@ These adjustments should significantly enhance the performance of the initial lo
146
146
147
147
### How should I scope my publications when setting up replication? {#how-should-i-scope-my-publications-when-setting-up-replication}
148
148
149
-
You can let ClickPipes manage your publications (requires write access) or create them yourself. With ClickPipe-managed publications, we automatically handle table additions and removals as you edit the pipe. If self-managing, carefully scope your publications to only include tables you need to replicate - including unnecessary tables will slow down Postgres WAL decoding. Importantly, exclude tables without primary keys if you're not replicating them to avoid potential replication slowness.
149
+
You can let ClickPipes manage your publications (requires write access) or create them yourself. With ClickPipes-managed publications, we automatically handle table additions and removals as you edit the pipe. If self-managing, carefully scope your publications to only include tables you need to replicate - including unnecessary tables will slow down Postgres WAL decoding.
150
150
151
+
If you include any table in your publication, make sure it has either a primary key or `REPLICA IDENTITY FULL`. If you have tables without a primary key, creating a publication for all tables will cause DELETE and UPDATE operations to fail on those tables.
152
+
153
+
To identify tables without primary keys in your database, you can use this query:
154
+
```sql
155
+
SELECT table_schema, table_name
156
+
FROM information_schema.tables
157
+
WHERE
158
+
(table_catalog, table_schema, table_name) NOT IN (
159
+
SELECT table_catalog, table_schema, table_name
160
+
FROM information_schema.table_constraints
161
+
WHERE constraint_type = 'PRIMARY KEY') AND
162
+
table_schema NOT IN ('information_schema', 'pg_catalog', 'pgq', 'londiste');
163
+
```
164
+
165
+
You have two options when dealing with tables without primary keys:
166
+
167
+
1. **Exclude tables without primary keys from ClickPipes**:
168
+
Create the publication with only the tables that have a primary key:
169
+
```sql
170
+
CREATE PUBLICATION my_publication FOR TABLE table_with_primary_key1, table_with_primary_key2, ...;
171
+
```
172
+
173
+
2. **Include tables without primary keys in ClickPipes**:
174
+
If you want to include tables without a primary key, you need to alter their replica identity to `FULL`. This ensures that UPDATE and DELETE operations work correctly:
175
+
```sql
176
+
ALTER TABLE table_without_primary_key1 REPLICA IDENTITY FULL;
177
+
ALTER TABLE table_without_primary_key2 REPLICA IDENTITY FULL;
178
+
CREATE PUBLICATION clickpipes_publication FOR ALL TABLES;
0 commit comments