Skip to content

[MySQL] BinLog Processing Queue backpressure handling #269

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 16 commits into from
Jun 2, 2025
Merged

[MySQL] BinLog Processing Queue backpressure handling #269

Show file tree
Hide file tree
Changes from 12 commits
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
e5db119
Removed zongji type mappings which are now provided by the Zongji pac…
Rentacookie May 21, 2025
d03360e
Moved most of the binlog event handling logic to a separate BinlogLis…
Rentacookie May 21, 2025
924ecd8
Updated the BinLogStream to use the new BinLogListener
Rentacookie May 21, 2025
404dcde
Renamed BinlogListener to BinLogListener
Rentacookie May 21, 2025
b1b8c30
Added changeset
Rentacookie May 21, 2025
16c1235
Merge branch 'main' into mysql-binlog-backpressure-handling
Rentacookie May 21, 2025
126f9b3
Simplified BinLogListener stopping mechanism
Rentacookie May 22, 2025
a03260f
Corrected BinLogListener name.
Rentacookie May 26, 2025
e147318
Supply port for binlog listener connections.
Rentacookie May 27, 2025
999a8dc
Only set up binlog heartbeat once the listener is fully started up.
Rentacookie May 27, 2025
782b43c
Merge branch 'main' into mysql-binlog-backpressure-handling
Rentacookie May 27, 2025
07201e8
Updated changeset
Rentacookie May 27, 2025
079a2f5
Changed binlog backpressure mechanism to be based on processing queue…
Rentacookie May 29, 2025
59afb33
Merge branch 'main' into mysql-binlog-backpressure-handling
Rentacookie May 29, 2025
286ba16
Changed binlog backpressure mechanism to be based on processing queue…
Rentacookie May 29, 2025
cce00b9
Fix: Use port for mysql pool creation
Rentacookie Jun 2, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions .changeset/honest-ties-crash.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
---
'@powersync/service-module-mysql': minor
---

Added a configurable limit for the MySQL binlog processing queue to limit memory usage.
Removed MySQL Zongji type definitions, they are now instead imported from the `@powersync/mysql-zongji` package.
Now passing in port for the Zongji connection, so that it can be used with MySQL servers that are not running on the default port 3306.
4 changes: 2 additions & 2 deletions modules/module-mysql/package.json
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,11 +29,11 @@
},
"dependencies": {
"@powersync/lib-services-framework": "workspace:*",
"@powersync/mysql-zongji": "^0.1.0",
"@powersync/service-core": "workspace:*",
"@powersync/service-jsonbig": "workspace:*",
"@powersync/service-sync-rules": "workspace:*",
"@powersync/service-types": "workspace:*",
"@powersync/service-jsonbig": "workspace:*",
"@powersync/mysql-zongji": "0.0.0-dev-20250527085137",
"async": "^3.2.4",
"mysql2": "^3.11.0",
"semver": "^7.5.4",
Expand Down
Loading