Optimize Module Dependency Handling for Efficient Memory Usage

[ayushpareek2003]

-Optimized addModuleFiles functions for both CompilerInvocation and CowCompilerInvocation to reduce redundant function calls and improve efficiency

-Introduced memory preallocation using reserve() when eager load is enabled to reduce reallocation overhead

-Used try_emplace() instead of insert() for PrebuiltModuleFiles to avoid unnecessary overwrites

[github-actions]

Thank you for submitting a Pull Request (PR) to the LLVM Project!

This PR will be automatically labeled and the relevant teams will be notified.

If you wish to, you can add reviewers by using the "Reviewers" section on this page.

If this is not working for you, it is probably because you do not have write permissions for the repository. In which case you can instead tag reviewers by name in a comment by using @ followed by their GitHub username.

If you have received no comments on your PR for a week, you can request a review by "ping"ing the PR by adding a comment “Ping”. The common courtesy "ping" rate is once a week. Please remember that you are asking for valuable time from other developers.

If you have further questions, they may be answered by the LLVM GitHub User Guide.

You can also ask questions in a comment on this PR, on the LLVM Discord or on the forums.

[llvmbot]

@llvm/pr-subscribers-clang

Author: Ayush Pareek (ayushpareek2003)

Changes

-Optimized addModuleFiles functions for both CompilerInvocation and CowCompilerInvocation to reduce redundant function calls and improve efficiency

-Introduced memory preallocation using reserve() when eager load is enabled to reduce reallocation overhead

-Used try_emplace() instead of insert() for PrebuiltModuleFiles to avoid unnecessary overwrites

---

Full diff: https://github.com/llvm/llvm-project/pull/132638.diff

1 Files Affected:

• (modified) clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp (+38-22)

diff --git a/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp b/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp
index d715ef874e002..b8b646ffb6d09 100644
--- a/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp
+++ b/clang/lib/Tooling/DependencyScanning/ModuleDepCollector.cpp
@@ -393,41 +393,57 @@ void ModuleDepCollector::addModuleMapFiles(
     CompilerInvocation &CI, ArrayRef<ModuleID> ClangModuleDeps) const {
   if (Service.shouldEagerLoadModules())
     return; // Only pcm is needed for eager load.
-
+   // Preallocate memory to avoid multiple allocations
+   CI.getFrontendOpts().ModuleMapFiles.reserve(
+       CI.getFrontendOpts().ModuleMapFiles.size() + ClangModuleDeps.size());
+ 
   for (const ModuleID &MID : ClangModuleDeps) {
-    ModuleDeps *MD = ModuleDepsByID.lookup(MID);
-    assert(MD && "Inconsistent dependency info");
-    CI.getFrontendOpts().ModuleMapFiles.push_back(MD->ClangModuleMapFile);
+    if (ModuleDeps *MD = ModuleDepsByID.lookup(MID)) { // Single lookup
+       assert(MD && "Inconsistent dependency info");
+       CI.getFrontendOpts().ModuleMapFiles.emplace_back(MD->ClangModuleMapFile);
+     }
   }
 }
 
 void ModuleDepCollector::addModuleFiles(
     CompilerInvocation &CI, ArrayRef<ModuleID> ClangModuleDeps) const {
+    // Preallocate memory if eager load is enabled
+   if (Service.shouldEagerLoadModules()) {
+     CI.getFrontendOpts().ModuleFiles.reserve(
+         CI.getFrontendOpts().ModuleFiles.size() + ClangModuleDeps.size());
+   }
   for (const ModuleID &MID : ClangModuleDeps) {
-    ModuleDeps *MD = ModuleDepsByID.lookup(MID);
-    std::string PCMPath =
-        Controller.lookupModuleOutput(*MD, ModuleOutputKind::ModuleFile);
-
-    if (Service.shouldEagerLoadModules())
-      CI.getFrontendOpts().ModuleFiles.push_back(std::move(PCMPath));
-    else
-      CI.getHeaderSearchOpts().PrebuiltModuleFiles.insert(
-          {MID.ModuleName, std::move(PCMPath)});
+    if (ModuleDeps *MD = ModuleDepsByID.lookup(MID)) {  
+       std::string PCMPath =
+           Controller.lookupModuleOutput(*MD, ModuleOutputKind::ModuleFile);
+       if (Service.shouldEagerLoadModules()) {
+         CI.getFrontendOpts().ModuleFiles.emplace_back(std::move(PCMPath));
+       } else {
+         CI.getHeaderSearchOpts().PrebuiltModuleFiles.try_emplace(
+             MID.ModuleName, std::move(PCMPath));
+       }
+    }
   }
 }
 
 void ModuleDepCollector::addModuleFiles(
     CowCompilerInvocation &CI, ArrayRef<ModuleID> ClangModuleDeps) const {
+      // Preallocation
+   if (Service.shouldEagerLoadModules()) {
+     CI.getMutFrontendOpts().ModuleFiles.reserve(
+         CI.getMutFrontendOpts().ModuleFiles.size() + ClangModuleDeps.size());
+   }
   for (const ModuleID &MID : ClangModuleDeps) {
-    ModuleDeps *MD = ModuleDepsByID.lookup(MID);
-    std::string PCMPath =
-        Controller.lookupModuleOutput(*MD, ModuleOutputKind::ModuleFile);
-
-    if (Service.shouldEagerLoadModules())
-      CI.getMutFrontendOpts().ModuleFiles.push_back(std::move(PCMPath));
-    else
-      CI.getMutHeaderSearchOpts().PrebuiltModuleFiles.insert(
-          {MID.ModuleName, std::move(PCMPath)});
+    if (ModuleDeps *MD = ModuleDepsByID.lookup(MID)) {  
+       std::string PCMPath =
+           Controller.lookupModuleOutput(*MD, ModuleOutputKind::ModuleFile);
+       if (Service.shouldEagerLoadModules()) {
+         CI.getMutFrontendOpts().ModuleFiles.emplace_back(std::move(PCMPath));
+       } else {
+         CI.getMutHeaderSearchOpts().PrebuiltModuleFiles.try_emplace(
+             MID.ModuleName, std::move(PCMPath));
+       }
+     }
   }
 }
 

[ayushpareek2003]

@Bigcheese sir please have a look