Codebase indexing ignores .gitignore and inconsistently applies .rooignore, leading to incorrect and unpredictable file indexing

[andreilungeanu]

App Version

v3.23.7

API Provider

Ollama

Model Used

nomic-embed-text

Roo Code Task Links (Optional)

The codebase indexing feature is not correctly respecting file exclusion rules, leading to inconsistent and incorrect indexing of the workspace. Specifically:

• The .gitignore file seems to have no effect on the indexing process. This results in the inclusion of large and irrelevant directories like node_modules, which dramatically inflates the number of indexed blocks and pollutes the search results.
• The .rooignore file is respected, but its application is inconsistent and buggy. After creating a .rooignore file (e.g., by copying the contents of .gitignore), the index correctly filters the files, but only after a full restart of VSCode.
• The most critical issue is that if the index is cleared using the "Clear Index Data" button, the .rooignore file is ignored on the subsequent re-indexing. The index reverts to including all files, and the only way to force it to re-apply the .rooignore rules is to restart VSCode again.

This behavior makes the codebase indexing feature unreliable and difficult to manage, as it requires manual intervention (restarting the editor) and leads to unpredictable search results.

🔁 Steps to Reproduce

1. Open a project with a standard .gitignore file that excludes node_modules.
2. Enable and run the codebase indexing.
3. Observe that the number of indexed blocks is very high because node_modules and other ignored files have been included. In my case, this was 2248 blocks.
4. Create a .rooignore file in the root of the project and copy the contents of .gitignore into it.
5. Restart VSCode.
6. The indexer will now run again and correctly respect the .rooignore file, resulting in a much smaller and more accurate index. In my case, this dropped to 157 blocks.
7. Click the status icon and use the "Clear Index Data" button to reset the index.
8. The indexer will start again automatically.
9. Observe that the bug reappears: the .rooignore file is now ignored, and the block count returns to the high value (e.g., 2248 blocks), including all previously ignored files.
10. The only way to get the correct indexing back is to restart VSCode again.

💥 Outcome Summary

Expected Behavior

• The codebase indexer should respect the .gitignore file by default, as is standard behavior for many development tools.
• If a .rooignore file is present, its rules should be applied consistently without requiring an editor restart.
• Clearing the index data should not cause the indexer to forget or ignore existing .rooignore rules on the next run. The re-indexing process should immediately re-apply all existing ignore rules.

Actual Behavior

• .gitignore is completely ignored.
• .rooignore is only applied after a VSCode restart.
• Clearing the index causes .rooignore to be ignored until the next VSCode restart, leading to a cycle of incorrect indexing.

with .rooignore

random values after restart

without .rooignore

📄 Relevant Logs or Errors (Optional)

[dosubot]

Based on the documentation, Roo Code's indexer is supposed to respect both .gitignore and .rooignore patterns for file exclusion, and .rooignore changes should be picked up automatically without needing to restart VSCode. There’s no mention of any known issues or limitations about .gitignore being ignored, .rooignore requiring a restart, or .rooignore being ignored after clearing index data—what you’re seeing isn’t expected or documented behavior for v3.23.7 docs docs.

If you’re consistently seeing .gitignore ignored and .rooignore only applied after a restart (or ignored after clearing the index), this is likely a bug or an undocumented limitation. The docs recommend using both files for best results, but don’t mention any workarounds for the behavior you described.

If you need consistent indexing right now, restarting VSCode after updating .rooignore seems to be the only reliable workaround. If you have any more details (logs, config, etc.), sharing them could help pinpoint the root cause.

To reply, just mention @dosu.

---

^{How did I do? Good | Irrelevant | Incorrect | Verbose | Hallucination | Report 🐛 | Other}   

[omagoduck]

@hannesrudolph Indexing, even Roo should never respect .gitignore automatically by default. Cause .gitignore is for git, and .rooignore should be for Roo. Maybe you can add a settings option to respect .gitignore by Roo, by preference.

[andreilungeanu]

Based on the documentation, Roo Code’s indexer is expected to respect both .gitignore and .rooignore patterns for file exclusion.

In my view, .gitignore should act as a fallback when .rooignore is either missing or empty.

Indexing folders like node_modules or vendor is generally unnecessary and inefficient. However, if someone does need to include them, that behavior should be configurable through .rooignore.

[roo-code-preview]

I'm exploring a potential fix for the codebase indexing ignore file issues and will open a PR to share progress. This involves fixing the .gitignore/.rooignore handling inconsistencies during indexing and after clearing index data.