Indexing seemingly causes windows protection to trigger

[decimad]

Hello,
with one cache file per source file, the cquery service can literally parse and write thousands of small files, looking suspicious to malware protection. Is the cache database architecture governed by libclang? I see other services using lightweight database libs f.e.

[decimad]

Another problem with the @A@b@c file name format would be, that it effectively doubles the path lengths, if you're indexing to a cquery subdirectory of your project directory. More like tripling it, because all files are further separated into two different directories, each containing portions of the path as well.

[jacobdufault]

The cache format is under our control; we could write to one big file, the thousands of small files is just simplest w.r.t. implementation. As the user edits the program the files need to be updated, so space management in the big file may be tricky.

Normalizing paths using a hash should fix the @A@b@c long-path issue (but it will make it hard to find the cache file for a specific file, but maybe a grep would be good enough for that).

[rdhafidh]

The cache format is under our control; we could write to one big file, the thousands of small files is just simplest w.r.t. implementation. As the user edits the program the files need to be updated, so space management in the big file may be tricky.

@jacobdufault , I hope that cquery can save this huge cache with relational dbms like sqlite or mysql instead of self implementations.. I'm interested now implementing this feature. Could you please tell me which source file is responsible doing this stuff behind the scene currently?
Thanks

Regards hafidh.

[jacobdufault]

@rdhafidh @decimad has made a lot of progress on this in #624. Feel free to hop on the gitter to chat.

[rdhafidh]

@jacobdufault , Yeah that nice one, but I want to have another wider rdbms backend by using QSqlDatabase. Will post back when it's ready ..

Thanks
Regards hafidh

[jacobdufault]

Sure, as a heads up, the cache is only used for offline storage. The querydb is entirely in-memory so changing caching implementations is only for restore/load and delta index updates.

I'm okay with moving querydb to be on-disk but it will need to be done carefully to avoid regressing performance. Definitely non-trivial and tricky work.

Also, FYI, I likely cannot merge something using QSqlDatabase due to the heavy dependency and/or licensing issues.

[ylemoigne]

Is it possible that this issue lead to :

2018-05-06 00:05:24.069 (   1.218s) [indexer2     ]          utils.cc:381    ERR| Cannot write to C:/....

I've disabled windows defender, and I can create the file from windows explorer, I don't understand why I get this issue.