Before the successful renaming, a session accessed the ghost table, w… #1536
Conversation
…hich had already unlocked the original table. There is a very small probability that other sessions dml operations on the original table will occur, and this dml operation will appear in the original table after renaming, resulting in data loss.
abaowhy
requested review from
meiji163,
rashiq and
timvaillancourt
as code owners
|
@abaowhy Hey, thanks for the bug report and the code changes! 🙇♂️ ❤️ Is there a straightforward way to reproduce this issue? It'd be good to have a test cases that could show that a) this issue exists and b) the proposed code changes fix that issue. 🤔 |
Cutover single testing is quite difficult, so I first provided a picture to illustrate |
|
It looks similar to this PR #1269 . |
@arthurschreiber I have provided test cases that issue can be stably reproduced on master and fixed in PR. |
|
@meiji163 I have provided test cases， please take some time to review, and if there are any issues, I will promptly correct them. |
|
@meiji163 I have fixed three issues during the inspection. Could you please take some time to check again |
|
@meiji163 @timvaillancourt @rashiq I have provided test cases that issue can be stably reproduced on master and fixed in PR，please take some time to review, and if there are any issues, I will promptly correct them. |
|
@abaowhy I was able to reproduce the bug with your test case. 谢谢！ |
|
This change is now requiring "performance_schema" to be enabled or the script fails with this error because of the query at the beginning of StateMetadataLockInstrument(): Any way to bypass this check? |
|
I’m planning to use gh-ost to run migrations on my system, but I’m hitting an error because Performance Schema is not enabled. Enabling Performance Schema requires a reboot, and since I’m running on Amazon Aurora (RDS), I’m trying to avoid a restart on the writer instance. Are there any alternative solutions? |
Since #1536, performance_schema.metadata_locks is required to check the rename session holds the metadata lock on the migrated table during cut-over. On some setups such as Aurora RDS performance_schema is not enabled by default and it may be infeasible to enable. This PR adds --skip-metadata-lock-check flag to skip the check in this case.
|
@smartinec @thenam153 See |
6 participants
Description
Before the successful renaming, a session accessed the ghost table, which had already unlocked the original table.
There is a very small probability that other sessions dml operations on the original table will occur,
and this dml operation will appear in the original table after renaming, resulting in data loss.
https://cloud.tencent.com/developer/article/2303777
This PR avoid the dataloss in this situation by checking that the rename session is waiting for the lock of the original table in
performance_schema.metadata_locks. Only then will the lock of the original table be released, and finally rename can be executed. If rename does not wait for the lock, it will exit gh-ost.