IndexData folder size grows very large over time

[InCerryGit]

Describe the bug
First of all, thank you for Fluent Search — it’s very useful.
I noticed that Fluent Search appears to be using a large amount of disk space over time. On my machine, the IndexData folder under ProgramData\Fluent Search has grown to about 92.4 GB.

I’m not sure whether this is expected behavior in some cases, but it seems higher than I anticipated for index data.

To Reproduce
Steps to reproduce the behavior:

Install and use Fluent Search for a while with normal daily usage.
Allow indexing to run in the background.
Check disk usage with a tool like WizTree/TreeSize/WinDirStat.
Go to C:\ProgramData\Fluent Search\IndexData.
Notice that the folder size may become very large.
Expected behavior
I would expect Fluent Search to keep index storage at a manageable size, or provide an easy way to control/clean old index data when it grows too much.

Screenshots
Attached screenshot for reference:

ProgramData: 116.7 GB
Fluent Search: 92.4 GB
IndexData: 92.4 GB
Desktop (please complete the following information):

Window 10/11 Version: 26200.7462
Fluent Search Version: 1.1.1.3
Installation source: Microsoft Store
Additional context
If this is expected in some scenarios, could you please share recommended cleanup steps or configuration suggestions?
If not, I’m happy to help test or provide more logs/details.

Thanks again for your work on this project.

[adirh3]

Hi, thanks for reporting this!

The indexer has been completely revamped in the latest nightly build — it should be much better in terms of disk usage and overall performance.

I'd suggest giving the nightly build a try, though keep in mind it may still be a bit buggy since it's a nightly release. You can download it from the main GitHub page: https://github.com/adirh3/Fluent-Search

Let me know how it goes!

[InCerryGit]

Hi, thanks for reporting this!你好，感谢你举报！

The indexer has been completely revamped in the latest nightly build — it should be much better in terms of disk usage and overall performance.索引器在最新的夜间版本中进行了彻底改造——在磁盘使用和整体性能方面应该会有更好的提升。

I'd suggest giving the nightly build a try, though keep in mind it may still be a bit buggy since it's a nightly release. You can download it from the main GitHub page: https://github.com/adirh3/Fluent-Search我建议你试试夜间版本，不过要注意它可能还是有点 bug，因为它是夜间版。你可以从主 GitHub 页面下载：https://github.com/adirh3/Fluent-Search

Let me know how it goes!告诉我结果如何！

Okay, I’ve just installed the latest version and am currently trying it out. If there are any issues, I’ll continue to provide feedback. Additionally, I’d like to ask whether the Dev Drive using the ReFS file system has been optimized. In previous versions, file indexing was very slow, and sometimes I had to rebuild the index.

[adirh3]

Thanks for trying the nightly. The IndexData size issue should be much better there.

ReFS / Dev Drive is not specifically optimized yet though. The fast indexer path is still NTFS-only, so Dev Drive currently falls back to the generic scan path. That can make initial indexing slower and can still require rescans.

So if the nightly fixes the size growth but Dev Drive is still slower than a normal NTFS drive, that would be expected with the current implementation. If you can, let me know whether the IndexData size now stays reasonable, and how Dev Drive compares to an NTFS drive.

[InCerryGit]

Thanks for trying the nightly. The IndexData size issue should be much better there.

ReFS / Dev Drive is not specifically optimized yet though. The fast indexer path is still NTFS-only, so Dev Drive currently falls back to the generic scan path. That can make initial indexing slower and can still require rescans.

So if the nightly fixes the size growth but Dev Drive is still slower than a normal NTFS drive, that would be expected with the current implementation. If you can, let me know whether the IndexData size now stays reasonable, and how Dev Drive compares to an NTFS drive.

I believe the issue of the indexdata size being too large has already been resolved. Based on observations over the past few days, everything looks normal. However, the update speed of the Dev Drive is very slow. For example, this *.sln file of mine still cannot be found through search even after several hour, and I have to manually reindex it.