clangd can exhaust all disk space in /tmp by writing preamble-*.pch files

[ehsan]

Bugzilla Link	52489
Version	unspecified
OS	All

Extended Description

I use the clangd extension in Visual Studio for a very large C++ project based on the Chromium code base.

I've noticed that clangd leaves behind preamble PCH files inside /tmp so frequently that I run out of disk space once every couple of weeks, with /tmp using 116GB, 130GB, etc worth in these files.

I noticed that others have experienced this issue too (e.g. https://youtrack.jetbrains.com/issue/CPP-19402) and some people recommend passing -pch-storage=memory to clangd. However I have 16GB of RAM on this server and I am unsure whether doing this is a good idea for a project of this size.

Could clangd be modified to manage these files better and prevent the out of disk space scenario?

[HighCommander4]

See also clangd/clangd#1007, though the issue there was more that the default storage location of /tmp was on a partition that didn't have enough space to store preambles during normal usage, and the solution was to use the TMPDIR environment variable to tell clangd to use a partition with more space.

Here, it sounds like the issue may be that clangd is not cleaning up the PCH files properly, perhaps due to a crash / unclean shutdown, and we should probably try harder to avoid that.

[memchr]

It may also be useful if clangd can provide a configuration for where to store the PCHs, as /tmp is commonly mounted as tmpfs on many Linux distributions (e.g. fedora), so in such cases --pch-storage=disk is effectively no different from --pch-storage=memory.

This is potentially a problem that can cause OOM kills.

[HighCommander4]

It may also be useful if clangd can provide a configuration for where to store the PCHs

As discussed in clangd/clangd#1007, clangd respects TMPDIR

[memchr]

It may also be useful if clangd can provide a configuration for where to store the PCHs

As discussed in clangd/clangd#1007, clangd respects TMPDIR

Thanks, I am aware of that, but TMPDIR is a canonical POSIX environment variable, which means that not only clangd will respect it. Changing TMPDIR can have unwanted effects. And in some clients such as vscode, there is no easy way to specifically make only clangd start with such an environment variable set to a different path, so an argument might be helpful here, since we can simply use clangd.arguments settings or the config.ymal

[HighCommander4]

Thanks, I am aware of that, but TMPDIR is a canonical POSIX environment variable, which means that not only clangd will respect it. Changing TMPDIR can have unwanted effects. And in some clients such as vscode, there is no easy way to specifically make only clangd start with such an environment variable set to a different path, so an argument might be helpful here, since we can simply use clangd.arguments settings or the config.ymal

Thanks for the added context. Would you mind filing a separate issue about adding a way to configure the PCH storage location, and copying over this explanation for why TMPDIR isn't sufficient?

(The current issue I'll leave open to track "clangd sometimes leaves behind PCH files on shutdown", which is an orthogonal problem.)

[Youda008]

So do i understand it correctly that clangd should, under normal circumstances, regularly clean up its temp files, and the fact that those files accumulate on disc is probably a result of crashes?

[HighCommander4]

So do i understand it correctly that clangd should, under normal circumstances, regularly clean up its temp files, and the fact that those files accumulate on disc is probably a result of crashes?

That's what I thought when I posted my previous comments in this issue, but since then additional investigation in clangd/clangd#1957 has suggested that, depending on the client (editor) used and how the client shuts down the server, leaking of PCH files could occur even in the case of an apparently normal shutdown.