Feature Request: Support for MySQL option binlog_rows_query_log_events

[ellayarmogray]

Feature Description

Vitess does not currently support the MySQL option binlog_rows_query_log_events as shown in this config stanza. This setting enables MySQL to include the original SQL query for each binlog event. For example, MySQL binlogs include rows_query log events that look like this:

#260209 18:20:26 server id 123  end_log_pos 1431 ...         Rows_query
# insert into table(id, create_date, update_date, varchar) values (10, 1770661226, 1770661226, 'abc')

Ideally, Vitess would provide the same functionality above to support the MySQL option – including query logs in VStream when this setting is enabled.

Use Case(s)

Etsy currently uses the Debezium connector for MySQL to implement a change data capture system. We enable binlog_rows_query_log_events to include query logs in the binlog and make SQL statements available to Debezium as mentioned in the Debezium docs. We have written a custom SMT (single message transform) which extracts query comments from these query logs to pass information to downstream consumers. We currently only use the query comments from the SQL string, but future functionality may require the entire SQL query. The query comments we add are of the form "[key1=value1] [key2=value2]".

We are interested in migrating to Debezium's Vitess connector which provides support for resharding operations. However, Vitess does not support including these query logs in VStream so we are unable to replicate this functionality with the Vitess connector.

[mattlord]

Hi @ellayarmogray,

This appears to be the other key part: https://debezium.io/documentation/reference/stable/connectors/mysql.html#mysql-property-include-query

Can you share an example of what this looks like in the Debezium connector for MySQL event stream?

Does it matter how closely that is mimicked in the Debezium connector for Vitess?

AFAICT, this feature would require work both in Vitess and in the Debezium connector for Vitess.

[ellayarmogray]

Hi @mattlord, thanks for your reply. I am aware that work would be required in the Debezium connector as well to support the include.query setting, but I was attempting to limit the scope of this ticket to the Vitess's support for that MySQL option since it is a pre-requisite for that. I was under the impression that any work past that would fall outside of PlanetScale's domain.

However, to answer your question, the query shows up in the source metadata for a changelog which you can see here in the Debezium docs.