Skip to content

chore: Warn on generated columns #448

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: main
Choose a base branch
from
Open

chore: Warn on generated columns #448

Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ea4a976
Warn on generated columns
caike Nov 10, 2025
a63e1f3
Simplify checking for generated columns.
caike Nov 20, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
37 changes: 36 additions & 1 deletion etl/src/replication/client.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -803,7 +803,8 @@ impl PgReplicationClient {
/// Retrieves schema information for all columns in a table.
///
/// If a publication is specified, only columns included in that publication
/// will be returned.
/// will be returned. Generated columns are always excluded since they are not
/// supported in PostgreSQL logical replication.
async fn get_column_schemas(
&self,
table_id: TableId,
Expand Down Expand Up @@ -864,6 +865,40 @@ impl PgReplicationClient {
publication_predicate = publication_filter.predicate,
);

// Check for generated columns so we can warn if there are any.
let generated_columns_check_query = format!(
r#"select exists (
select 1
from pg_attribute
where attrelid = {table_id}
and attnum > 0
and not attisdropped
and attgenerated != ''
) as has_generated;"#
);

for message in self
.client
.simple_query(&generated_columns_check_query)
.await?
{
if let SimpleQueryMessage::Row(row) = message {
let has_generated_columns =
Self::get_row_value::<String>(&row, "has_generated", "pg_attribute").await?
== "t";
if has_generated_columns {
warn!(
"Table {} contains generated columns that will NOT be replicated. \
Generated columns are not supported in PostgreSQL logical replication and will \
be excluded from the ETL schema. These columns will NOT appear in the destination.",
table_id
Copy link
Author

@caike caike Nov 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Figured that keeping at least the table_id in the warn message would be helpful and at no performance cost

);
}
// Explicity break for clarity; this query returns a single SimpleQueryMessage::Row.
break;
}
}

let mut column_schemas = vec![];
for message in self.client.simple_query(&column_info_query).await? {
if let SimpleQueryMessage::Row(row) = message {
Expand Down