Skip to content

Remove unused TIMELINE_HISTORY command from logical replication connection #1039

Description

@kushazsehgal

Summary

pgsql_identify_system() issues a TIMELINE_HISTORY <N> replication command when the source server's timeline is greater than 1. The result — a timeline history file written to disk and catalog entries stored in SQLite — is never read back or used anywhere in pgcopydb. This makes the command dead code.

In addition to being unused, the command is problematic on managed PostgreSQL services (e.g. Azure Database for PostgreSQL – Flexible Server) that disallow TIMELINE_HISTORY on logical replication connections. Since pgcopydb establishes a logical replication connection for CDC streaming, the command fails outright on these platforms — blocking pgcopydb users even though the data it fetches serves no purpose.


Evidence: The data pipeline is write-only

1. The command is issued in pgsql_timeline.c

/* src/bin/pgcopydb/pgsql_timeline.c, inside pgsql_identify_system() */
if (system->timeline > 1)
{
    char sql[BUFSIZE] = { 0 };
    sformat(sql, sizeof(sql), "TIMELINE_HISTORY %d", system->timeline);

    result = PQexec(connection, sql);
    if (!is_response_ok(result))
    {
        log_error("Failed to request TIMELINE_HISTORY: %s",
                  PQerrorMessage(connection));
        /* returns false, failing the whole identify_system call */
        return false;
    }

    (void) parseTimelineHistoryResult((void *) &hContext, result, cdcPathDir);
    /* stores filename in system->timelineHistoryFilename */
}

2. The result is written to disk and stored in the SQLite catalog

/* parse_timeline_history_file() -> register_timeline_hook() -> catalog_add_timeline_history() */
if (!catalog_add_timeline_history(context->catalog, &entry))
{
    log_error("Failed to add timeline history entry to catalog");
    return false;
}

3. The read-back function is defined but never called outside its own file

catalog_lookup_timeline_history() is declared in catalog.h and defined in catalog.c, but appears in no other file:

$ gh api "search/code?q=catalog_lookup_timeline_history+repo:dimitri/pgcopydb" \
    --jq '.items[].path'
src/bin/pgcopydb/catalog.h
src/bin/pgcopydb/catalog.c

Likewise, ld_store_insert_timeline_history() is only referenced within ld_store.c itself.

4. The IdentifySystem struct carries the filename field, but nothing acts on it

/* src/bin/pgcopydb/pgsql.h */
typedef struct IdentifySystem
{
    uint64_t identifier;
    uint32_t timeline;
    char xlogpos[PG_LSN_MAXLENGTH];
    char dbname[NAMEDATALEN];
    TimelineHistoryEntry currentTimeline;
    char timelineHistoryFilename[MAXPGPATH];  /* populated, never used */
} IdentifySystem;

Impact on managed PostgreSQL services

Azure Database for PostgreSQL – Flexible Server (and potentially other managed providers) restricts the TIMELINE_HISTORY command on logical replication connections. Because pgsql_identify_system treats a TIMELINE_HISTORY error as fatal:

if (!is_response_ok(result))
{
    log_error("Failed to request TIMELINE_HISTORY: %s",
              PQerrorMessage(connection));
    PQfinish(connection);
    return false;   // aborts identify_system entirely
}

pgcopydb CDC fails completely on these platforms, even though the data the command would have fetched is never used.


Proposed fix

Remove the TIMELINE_HISTORY block from pgsql_identify_system() in pgsql_timeline.c. Follow-on cleanup:

  • Remove parseTimelineHistoryResult() and writeTimelineHistoryFile() (only called from that block)
  • Remove parse_timeline_history_file() and register_timeline_hook() from pgsql_timeline.c / pgsql_timeline.h
  • Remove catalog_add_timeline_history() and catalog_lookup_timeline_history() from catalog.c / catalog.h
  • Remove ld_store_insert_timeline_history() from ld_store.c / ld_store.h
  • Remove the timelineHistoryFilename field from IdentifySystem in pgsql.h
  • Drop the timeline_history table DDL from catalog.c

None of these are referenced by any non-dead path.

Files affected:

  • src/bin/pgcopydb/pgsql_timeline.c
  • src/bin/pgcopydb/pgsql_timeline.h
  • src/bin/pgcopydb/pgsql.h
  • src/bin/pgcopydb/catalog.c
  • src/bin/pgcopydb/catalog.h
  • src/bin/pgcopydb/ld_store.c
  • src/bin/pgcopydb/ld_store.h

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions