Skip to content

[BUG] TRUNCATE keyword breaks replication #216

@Lotuashvili

Description

@Lotuashvili

Bug Description

TRUNCATE keyword breaks replication, even if that table is not being replicated from config.yaml file.

Steps to Reproduce

  1. Replicate any table via config.yaml (Let's say "users")
  2. Truncate another table
  3. Replication will fail

Expected Behavior

Expected to truncate given table, or ignore truncation of the table which is not being replicated

Actual Behavior

telescope_entries table is not being replicated by config, but it still breaks.

binlogrepl 2025-11-11 18:10:42,960     INFO] last transaction id: ('mysql-bin.000425', 5812497), processed events: 54977
[runner 2025-11-11 18:10:53,214     INFO] mysql databases: ['forge', 'information_schema', 'mysql', 'performance_schema', 'scandiweb', 'sys']
[runner 2025-11-11 18:10:53,214     INFO] mysql databases filtered: ['forge']
[dbrepl forge 2025-11-11 18:10:55,662     INFO] stats: {"last_transaction": ["mysql-bin.000425", 5851510], "events_count": 130, "insert_events_count": 0, "insert_records_count": 0, "erase_events_count": 0, "erase_records_count": 0, "no_events_count": 200, "cpu_load": 0.001}
[dbrepl forge 2025-11-11 18:10:55,662     INFO] ch_stats: {"total": {"inserts": {"duration": 0.0, "events": 0, "records": 0}, "erases": {"duration": 0.0, "events": 0, "records": 0}}}
[dbrepl forge 2025-11-11 18:11:32,610    ERROR] unhandled exception
Traceback (most recent call last):
  File "/home/forge/.local/share/pipx/venvs/mysql-ch-replicator/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator.py", line 253, in run
    self.run_realtime_replication()
  File "/home/forge/.local/share/pipx/venvs/mysql-ch-replicator/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator.py", line 260, in run_realtime_replication
    self.realtime_replicator.run_realtime_replication()
  File "/home/forge/.local/share/pipx/venvs/mysql-ch-replicator/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator_realtime.py", line 73, in run_realtime_replication
    self.handle_event(event)
  File "/home/forge/.local/share/pipx/venvs/mysql-ch-replicator/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator_realtime.py", line 94, in handle_event
    event_handlers[event.event_type](event)
  File "/home/forge/.local/share/pipx/venvs/mysql-ch-replicator/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator_realtime.py", line 196, in handle_query_event
    self.handle_truncate_query(query, event.db_name)
  File "/home/forge/.local/share/pipx/venvs/mysql-ch-replicator/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator_realtime.py", line 270, in handle_truncate_query
    raise Exception('Invalid TRUNCATE query format', query)
Exception: ('Invalid TRUNCATE query format', 'TRUNCATE `telescope_entries`')
Traceback (most recent call last):
  File "/home/forge/.local/bin/mysql_ch_replicator", line 7, in <module>
    sys.exit(main())
             ^^^^^^
  File "/home/forge/.local/share/pipx/venvs/mysql-ch-replicator/lib/python3.12/site-packages/mysql_ch_replicator/main.py", line 202, in main
    run_db_replicator(args, config)
  File "/home/forge/.local/share/pipx/venvs/mysql-ch-replicator/lib/python3.12/site-packages/mysql_ch_replicator/main.py", line 116, in run_db_replicator
    db_replicator.run()
  File "/home/forge/.local/share/pipx/venvs/mysql-ch-replicator/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator.py", line 253, in run
    self.run_realtime_replication()
  File "/home/forge/.local/share/pipx/venvs/mysql-ch-replicator/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator.py", line 260, in run_realtime_replication
    self.realtime_replicator.run_realtime_replication()
  File "/home/forge/.local/share/pipx/venvs/mysql-ch-replicator/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator_realtime.py", line 73, in run_realtime_replication
    self.handle_event(event)
  File "/home/forge/.local/share/pipx/venvs/mysql-ch-replicator/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator_realtime.py", line 94, in handle_event
    event_handlers[event.event_type](event)
  File "/home/forge/.local/share/pipx/venvs/mysql-ch-replicator/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator_realtime.py", line 196, in handle_query_event
    self.handle_truncate_query(query, event.db_name)
  File "/home/forge/.local/share/pipx/venvs/mysql-ch-replicator/lib/python3.12/site-packages/mysql_ch_replicator/db_replicator_realtime.py", line 270, in handle_truncate_query
    raise Exception('Invalid TRUNCATE query format', query)
Exception: ('Invalid TRUNCATE query format', 'TRUNCATE `telescope_entries`')
[runner 2025-11-11 18:11:33,219  WARNING] Process dead (exit code: 1), restarting

Environment

  • mysql_ch_replicator version: Latest
  • Operating System: Ubuntu
  • Python version: 3.12

MySQL Configuration

[mysqld]
server-id = 1
log_bin = /var/log/mysql/mysql-bin.log
binlog_format = ROW
binlog_row_metadata = FULL
expire_logs_days = 2
gtid_mode = ON
enforce_gtid_consistency = ON
log_slave_updates = ON
sync_binlog = 1

Replicator Configuration

mysql:
  host: 'localhost'
  port: 3306
  user: 'clickhouse'
  password: 'xxxxxxxxx'

clickhouse:
  host: 'localhost'
  port: 8123
  user: 'default'
  password: 'xxxxxxxxxxxxxxxx'

binlog_replicator:
  data_dir: '/home/user/mysql-clickhouse-replicator/binlog/'
  records_per_file: 50000

databases: 'myapp'
tables:
  - activities
  - admins
  - balances

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions