Skip to content

Commit 6cd1d50

Browse files
authored
Merge pull request #3248 from ClickHouse/publication-faqs
docs: enhance Postgres publication setup guidance
2 parents eee422c + ba4abfc commit 6cd1d50

File tree

1 file changed

+30
-1
lines changed
  • docs/integrations/data-ingestion/clickpipes/postgres

1 file changed

+30
-1
lines changed

docs/integrations/data-ingestion/clickpipes/postgres/faq.md

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,8 +146,37 @@ These adjustments should significantly enhance the performance of the initial lo
146146
147147
### How should I scope my publications when setting up replication? {#how-should-i-scope-my-publications-when-setting-up-replication}
148148
149-
You can let ClickPipes manage your publications (requires write access) or create them yourself. With ClickPipe-managed publications, we automatically handle table additions and removals as you edit the pipe. If self-managing, carefully scope your publications to only include tables you need to replicate - including unnecessary tables will slow down Postgres WAL decoding. Importantly, exclude tables without primary keys if you're not replicating them to avoid potential replication slowness.
149+
You can let ClickPipes manage your publications (requires write access) or create them yourself. With ClickPipes-managed publications, we automatically handle table additions and removals as you edit the pipe. If self-managing, carefully scope your publications to only include tables you need to replicate - including unnecessary tables will slow down Postgres WAL decoding.
150150
151+
If you include any table in your publication, make sure it has either a primary key or `REPLICA IDENTITY FULL`. If you have tables without a primary key, creating a publication for all tables will cause DELETE and UPDATE operations to fail on those tables.
152+
153+
To identify tables without primary keys in your database, you can use this query:
154+
```sql
155+
SELECT table_schema, table_name
156+
FROM information_schema.tables
157+
WHERE
158+
(table_catalog, table_schema, table_name) NOT IN (
159+
SELECT table_catalog, table_schema, table_name
160+
FROM information_schema.table_constraints
161+
WHERE constraint_type = 'PRIMARY KEY') AND
162+
table_schema NOT IN ('information_schema', 'pg_catalog', 'pgq', 'londiste');
163+
```
164+
165+
You have two options when dealing with tables without primary keys:
166+
167+
1. **Exclude tables without primary keys from ClickPipes**:
168+
Create the publication with only the tables that have a primary key:
169+
```sql
170+
CREATE PUBLICATION my_publication FOR TABLE table_with_primary_key1, table_with_primary_key2, ...;
171+
```
172+
173+
2. **Include tables without primary keys in ClickPipes**:
174+
If you want to include tables without a primary key, you need to alter their replica identity to `FULL`. This ensures that UPDATE and DELETE operations work correctly:
175+
```sql
176+
ALTER TABLE table_without_primary_key1 REPLICA IDENTITY FULL;
177+
ALTER TABLE table_without_primary_key2 REPLICA IDENTITY FULL;
178+
CREATE PUBLICATION clickpipes_publication FOR ALL TABLES;
179+
```
151180
152181
## Recommended `max_slot_wal_keep_size` Settings {#recommended-max_slot_wal_keep_size-settings}
153182

0 commit comments

Comments
 (0)