fix: resolve stale lock and same-instance write contention (#622, #623)

[jlin53882]

Summary

Fixes two related lock issues in memory-lancedb-pro:

• [BUG] Windows: stale .memory-write.lock not released, blocks smart-extractor writes #622: Stale .memory-write.lock\ not released, blocking writes on Windows
• [BUG] Same-instance concurrent writes竞争 lock，导致 Lock file is already being held #623: Same-instance concurrent writes competing for lock

Changes

src/store.ts

• Added \updateQueue\ for intra-instance write serialization
• Added \lockfilePath\ option with explicit lock path
• Added proactive stale lock cleanup (5-minute threshold)
• Fixed nested deadlock where \update()\ used both
  unWithFileLock()\ AND
  unSerializedUpdate()\

test/lock-recovery.test.mjs

• Tests for stale lock recovery

test/store-write-queue.test.mjs

• Tests for queue serialization

Testing

• 7 tests pass, 1 skip

Closes #622
Closes #623

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f691cff8bd

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[chatgpt-codex-connector]

Clear legacy lock artifacts immediately

This cleanup gate can leave a legacy .memory-write.lock regular file in place for up to 5 minutes, but the new lockfile.lock(this.config.dbPath, { lockfilePath }) flow expects that path to be created as a lock directory. In upgrade/restart scenarios where the old file was created recently, lock acquisition can fail (ELOCKED/ENOTDIR) and block all writes until the age threshold is crossed. Remove incompatible pre-existing lock artifacts regardless of age (or use the same short stale window as the lock) before acquiring the new lock.

Useful? React with 👍 / 👎.

[jlin53882]

CI Failure Analysis

The cli-smoke and storage-and-schema job failures are NOT related to this PR's changes.

Root Cause

These failures are tracked in:

• Issue cli-smoke test failing: missing store.count() mock (regression from PR #582) #590: cli-smoke test failing: missing store.count() mock (regression from PR fix: skip 75ms retry when store is empty (Bug 2) #582)
• Issue cli-smoke test: store mock missing count() method #596: cli-smoke test: store mock missing count() method

The cli-smoke.mjs mock for store is missing a count() method. PR #582 added a countStore parameter to
etrieveWithRetry(), but the test mock was never updated.

Verification

I tested on base commit

[rwmjhb]

Please resolve conflict at first

[jlin53882]

@rwmjhb 幫我看一下～

現況更新

✅ 衝突已解決 — PR 已 rebase 到 upstream/master 最新，零衝突。

⚠️ packaging-and-workflow CI 失敗

CI 失敗在 verify-ci-test-manifest.mjs：

Error: unexpected manifest entry: test/hook-dedup-phase1.test.mjs

初步確認這是 upstream #617 merge 時產生的既有问题，不是本 PR 引進的：

• test/hook-dedup-phase1.test.mjs 在 scripts/ci-test-manifest.mjs 中（來自 feat(phase1): hook event deduplication (continuation of PR #430) #617）
• 但不在 scripts/verify-ci-test-manifest.mjs 的 EXPECTED_BASELINE 中
• 導致 verifyExactOnceCoverage() 失敗
• upstream master 自己的 CI 也同樣 fail 在這裡

正在確認 upstream #617 的 CI 狀態，確認後會再更新。

[jlin53882]

關聯 PR #617 Hook Event Deduplication 的潛在問題

嗨 @CortexReach/memory-lancedb-pro 維護者，

PR #617 實作了 hook event deduplication（_hookEventDedup Set，max 200 entries，prune to newest 100），並將新測試檔註冊進了 scripts/ci-test-manifest.mjs。

注意到 PR #626 這次新增了兩個測試檔：

• test/lock-recovery.test.mjs
• test/store-write-queue.test.mjs

建議：確認這兩個測試檔是否也已註冊進 scripts/ci-test-manifest.mjs（類似的 packaging-and-workflow CI 失敗在 PR #617 時曾短暫出現，後來在 merge 前修復）。

另外，PR #617 的 hook deduplication 邏輯（index.ts）和 PR #626 的 lock recovery（src/store.ts）涉及不同的模組級狀態，如果有任何互動（例如 hook 層的 dedup skip 是否影響 store 層的 lock 事件時序），建議確認測試覆蓋。

謝謝！