feat: auto purge stale prepared statements from cache and retry query

[Invader444]

Even with proper tuning of maxPreparedStatements, it is possible that the automatic LRU cache contains stale prepared statements that have been released on the server from time to time.

Normally, with cached prepared statements, a user could catch the ER_UNKNOWN_STMT_HANDLER error, purge from their cache, and try again with a newly prepared statement. However, to do that with the internal LRU cache, a user would need to touch internals that are not meant to be public APIs... which could break at any time.

This handles that invalidation and retry logic internally so the user doesn't have to touch private/internal implementation details.

This may help issue #805

[codecov]

Codecov Report

❌ Patch coverage is 92.85714% with 6 lines in your changes missing coverage. Please review.
✅ Project coverage is 90.87%. Comparing base (2498c20) to head (debdc76).

Files with missing lines	Patch %	Lines
lib/base/connection.js	92.85%	6 Missing ⚠️

Additional details and impacted files

@@           Coverage Diff           @@
##           master    #4199   +/-   ##
=======================================
  Coverage   90.86%   90.87%           
=======================================
  Files          89       89           
  Lines       14490    14568   +78     
  Branches     1864     1882   +18     
=======================================
+ Hits        13167    13239   +72     
- Misses       1323     1329    +6     

Flag	Coverage Δ
compression-0	90.14% <92.85%> (+0.01%)	⬆️
compression-1	90.85% <92.85%> (+<0.01%)	⬆️
static-parser-0	88.61% <92.85%> (+0.01%)	⬆️
static-parser-1	89.35% <92.85%> (+0.01%)	⬆️
tls-0	90.32% <92.85%> (+0.01%)	⬆️
tls-1	90.65% <92.85%> (+<0.01%)	⬆️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Copilot]

Pull request overview

Adds internal invalidation/retry behavior for cached prepared statements when the server reports a stale statement handler, so users don’t need to manually purge internal caches.

Changes:

• Detect ER_UNKNOWN_STMT_HANDLER during execute() and retry after purging the cached prepared statement.
• Add an integration test that simulates a stale cached statement and asserts automatic retry + cache refresh.

Reviewed changes

Copilot reviewed 2 out of 2 changed files in this pull request and generated no comments.

File	Description
lib/base/connection.js	Implements stale prepared-statement detection, cache purge, and automatic retry for execute().
test/integration/connection/test-execute-cached.test.mts	Adds coverage ensuring stale cached prepared statements are discarded and queries succeed after automatic retry.

[Invader444]

Added additional coverage for the EventEmitter path.

[Invader444]

Ok, this time I think it's ready; apologies for all the noise 😅

[Invader444]

Should be at 100% coverage now, and also cleaned up the "emit 'end' twice" that was happening when the internal prepare statement failed.

[AlekseyLkh]

I think this PR does not cover all possible cases where a prepared statement gets unexpectedly invalidated.

There are essentially two scenarios:

• Unexpected caching when using prepare() manually, and consequently unexpected invalidation in the case of eviction (I ran into this while studying the library's behavior manually).
• Invalidation during ZDP (Zero Downtime Patching), for example on AWS RDS. (I encountered this problem recently when AWS was patching RDS servers and it is the reason why I found this PR. Below is what I came up with after Investigate.)

The first case, in my view, should be fixed by removing the cache from the manual mechanism. If someone is manually using prepare(), stmt.execute(), stmt.close(), it should not interact with the cache at all. The caching mechanism should remain exclusively on the execute method side.

Regarding the second case: During ZDP, the session is essentially destroyed and ER_UNKNOWN_STMT_HANDLER errors start flooding in. It would be logical to remove statements from the cache one by one in this situation, but that approach is problematic due to MySQL's internal behavior. The server generates prepared statement IDs starting from zero, and during ZDP the counter is apparently reset — so newly created statements may receive the same ID as one already present in the cache. This leads to ER_WRONG_ARGUMENTS errors rather than ER_UNKNOWN_STMT_HANDLER errors. Therefore, I believe it is more practical to completely clear the cache whenever the caching mechanism encounters an ER_UNKNOWN_STMT_HANDLER error.

[sidorares]

The first case, in my view, should be fixed by removing the cache from the manual mechanism. If someone is manually using prepare(), stmt.execute(), stmt.close(), it should not interact with the cache at all. The caching mechanism should remain exclusively on the execute method side.

agree.

Regarding the second case: During ZDP, the session is essentially destroyed and ER_UNKNOWN_STMT_HANDLER errors start flooding in. It would be logical to remove statements from the cache one by one in this situation, but that approach is problematic due to MySQL's internal behavior. The server generates prepared statement IDs starting from zero, and during ZDP the counter is apparently reset — so newly created statements may receive the same ID as one already present in the cache. This leads to ER_WRONG_ARGUMENTS errors rather than ER_UNKNOWN_STMT_HANDLER errors. Therefore, I believe it is more practical to completely clear the cache whenever the caching mechanism encounters an ER_UNKNOWN_STMT_HANDLER error.

interesting. I wonder if there is any way to get notified about statement counter reset / ZDP. If that's not possible, clearing cache on ER_UNKNOWN_STMT_HANDLER seems like a logical thing to do ( still, potentially unexpected behaviour when the query that is actiually executed might not match to what was in execute() callsite

[Invader444]

#3726 mentions this...

[Invader444]

I think for the ZDP scenario where statement IDs might point to the wrong statement... a Map of statement ID to statement key would allow us, during prepare response, to detect that this has happened and remove the old statement that had the same ID... thoughts?

This would need to include all prepared statements, even if they weren't automatically created via execute... though we should also stop caching if not coming from execute in the first place.

[sidorares]

I was thinking if changing prepare() / unprepare() caching behavior needs a major version release, but reading #805 (comment) I'm more inclined to release it as a bugfix as its currently already quite broken and likely almost never used. I think we can merge this change and have a follow up PR which removes caching from manual prepare()

[Invader444]

Awesome, but the ZDP thing for me (and I think that's what bit me too which prompted me to write this), highlights a bigger problem with mismatched query IDs to queries... admittedly unlikely on a given connection to SELECT * FROM widgets WHERE id = ?, and the server thinks that prepared statement is DELETE FROM widgets WHERE id = ?, but possible?

[sidorares]

I initially thought it's possible, but maybe not. After ZDP happens, the very next .execute() command results in either ER_UNKNOWN_STMT_HANDLER if it was previously cached or in id conflict which we can also detect. I haven't read you changes carefully yet @Invader444 but I assumed your PR handles both scenarios?

ER_UNKNOWN_STMT_HANDLER -> reset cache, retry prepare + execute again
server returns id that we already have -> reset cache, save new statement, continue

[Invader444]

I thought it would be execute -> prepare/cache store -> execute it, execute again -> cache fetch succeed -> ZDP -> another out of band prepare makes the same query id for a delete with same number and type of arguments -> second execute performs the wrong query.

[Invader444]

But if you are using the pool correctly that's not possible... so maybe not a real issue 🤔

[Invader444]

Unclear on how id conflict would work, I don't think the id is part of the cache key so how would we know without a full cache scan (which sounds like a bad thing to do...)

[Invader444]

That leads to my map of query id to cache key.
prepare will always store query id to cache key map. Releasing a prepared statement would remove from that map.
Cache lookup would fetch the key from the cache, then check if the id to key map contains the query id. If it doesn't, or it does and points to the same key, use the cache. Otherwise, delete from cache and execute a new prepare.

[sidorares]

@Invader444 yes, unless we add statement id -> cache key map and reset connection on the first conflict the following is possible now:

-> client calls execute('select 1 where ? = ?', [1, 1])
-> client does not have it cached yet
-> sends COM_STMT_PREPARE "select 1 where ? = ?"
<- server sends statement id 0
-> client caches: "select 1 where ? = ?" -> statement id 0
-> sends COM_STMT_EXECUTE id 0 with params [1, 1]
<- server returns result for "select 1 where ? = ?"
--- server ZDP / session reset happens ---
   server-side prepared statements are gone
   client cache still says:
   "select 1 where ? = ?" -> statement id 0
--- some new prepares happen on that same logical connection before client retries this query ---
-> another prepare happens for "delete from t where a = ? and b = ?"
<- server sends statement id 0
   because statement ids restarted from 0 after reset
-> client later calls execute('select 1 where ? = ?', [1, 1])
-> client looks in cache and finds statement id 0
-> sends COM_STMT_EXECUTE id 0 with params [1, 1]
<- server executes the statement currently bound to id 0 which is now "delete from t where a = ? and b = ?"