Skip to content

Commit 2858c85

Browse files
[SYCL RTC] Introduce --persistent-auto-pch support
Built on top of `--auto-pch` (in-memory) introduced in #20226. The most significant technical decision was how to implement the filesystem cache. I've looked into the following options: * `sycl/source/detail/persistent_device_code_cache.hpp` Also, see `sycl/doc/design/KernelProgramCache.md` Seems to be tailored for the very specific usage scenarios, would be very resource consuming to split into a generic data structure that would then be used for two different use cases. This cache is disabled by default and I'm not sure how well-tested it is. Also, using plain ".lock" files for "advisory locking" instead of the native filesystem mechanisms (e.g., locking APIs in `fcntl`/`flock`/`CreateFile`/`LockFileEx`) made me question if it's worth generalizing and how much work would be necessary there. * `llvm/include/llvm/Support/Caching.hpp` Originally implemented as part of ThinLTO implementation, moved into `LLVMSupport` later with the following commit message: > We would like to move ThinLTO’s battle-tested file caching > mechanism to the LLVM Support library so that we can use it > elsewhere in LLVM. API is rather unexpected, so my research hasn't stopped here. * `lldb/include/lldb/Core/DataFileCache.h` Uses `LLVMSupport`'s caching from the previous bullet under the hood, but provides an easier to grasp API. If we were developing upstream I think uplifting that abstraction into `LLVMSupport` library and then using in both `lldb` and `libsycl` would probably be the choice I'd vote for. However, doing that downstream was too much efforts so I ultimately decided not to go with this approach. That cache also has a `std::mutex` on the "hot" `DataFileCache::GetCachedData` path, I presume to avoid creating the same entry from multiple threads. In the end, I've chosen to use `LLVMSupport`'s quirky (or maybe I just hasn't grown enough to appreciate it) caching API directly and that's what is done in this PR. Unlike `lldb`'s cache, I decided to trade possible duplicate work of building the preamble on a cache miss from concurrent threads in favor of no inter-thread synchronization (not profiled/measured though) on the cache hit path and implementation simplicity.
1 parent fc2ce23 commit 2858c85

File tree

12 files changed

+771
-88
lines changed

12 files changed

+771
-88
lines changed

clang/include/clang/Driver/Options.td

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -7557,6 +7557,10 @@ let Visibility = [SYCLRTCOnlyOption] in {
75577557
let Group = sycl_rtc_only_Group in {
75587558
def auto_pch : Flag<["--"], "auto-pch">,
75597559
HelpText<"Enable Auto-PCH for SYCL RTC Compilation">;
7560+
def persistent_auto_pch_EQ
7561+
: Joined<["--"], "persistent-auto-pch=">,
7562+
HelpText<"Use Persistent Auto-PCH cache located at <dir> for SYCL "
7563+
"RTC Compilation">;
75607564
} // let Group = sycl_rtc_only_Group
75617565
} // let Visibility = [SYCLRTCOnlyOption]
75627566

clang/include/clang/Frontend/PrecompiledPreamble.h

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -132,6 +132,8 @@ class PrecompiledPreamble {
132132
IntrusiveRefCntPtr<llvm::vfs::FileSystem> &VFS,
133133
llvm::MemoryBuffer *MainFileBuffer) const;
134134

135+
llvm::StringRef memoryContents() const;
136+
135137
private:
136138
PrecompiledPreamble(std::unique_ptr<PCHStorage> Storage,
137139
std::vector<char> PreambleBytes,

clang/lib/Frontend/PrecompiledPreamble.cpp

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -727,6 +727,10 @@ void PrecompiledPreamble::OverridePreamble(
727727
configurePreamble(Bounds, CI, VFS, MainFileBuffer);
728728
}
729729

730+
llvm::StringRef PrecompiledPreamble::memoryContents() const {
731+
return Storage->memoryContents();
732+
}
733+
730734
PrecompiledPreamble::PrecompiledPreamble(
731735
std::unique_ptr<PCHStorage> Storage, std::vector<char> PreambleBytes,
732736
bool PreambleEndsAtStartOfLine,

0 commit comments

Comments
 (0)