Fixes #4875: Fix codebase re-indexing race condition

[roomote]

• Fixed race condition in CodeIndexOrchestrator where clearIndexData and startIndexing could conflict
• Separated _isProcessing check from state validation in startIndexing for better error reporting
• Made webview startIndexing handler properly await the operation
• Removed _isProcessing reset from stopWatcher to prevent interference with calling methods
• Added comprehensive tests for clearIndexData and startIndexing sequence
• Ensures users can successfully re-index after clearing index data

---

Important

Fixes race condition in CodeIndexOrchestrator by improving startIndexing logic and adding tests for re-indexing sequence.

• Behavior:
  • Fixes race condition in CodeIndexOrchestrator by separating _isProcessing check from state validation in startIndexing().
  • Ensures webviewMessageHandler.ts properly awaits startIndexing().
  • Removes _isProcessing reset from stopWatcher() to prevent interference.
• Tests:
  • Adds tests in manager.spec.ts for clearIndexData and startIndexing sequence.
  • Tests handle rapid calls and ensure no errors in sequence execution.
• Misc:
  • Ensures users can re-index after clearing index data without errors.

^{This description was created by for 65e1821. You can customize this summary. It will automatically update as commits are pushed.}

[ellipsis-dev]

In startIndexing(), the early return when _isProcessing is true may drop an indexing request rather than waiting for clearIndexData to complete. The rapid call test expects the call to wait until the current operation finishes. Consider implementing a waiting/queuing mechanism (e.g. using a simple loop or a p-wait-for) so that if _isProcessing is true, startIndexing waits until it’s false before proceeding.

[daniel-lxs]

Closing, the actual issue got solved in #4857.

Also this: await manager.startIndexing() should never be done, awaiting for the indexing process will basically halt execution for any function/method that awaits it until the indexing process finishes which might takes minutes to complete.