[NFC][SpecialCaseList] Convert preprocess into LazyInit
#167281
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Created using spr 1.3.7 [skip ci]
Created using spr 1.3.7
Member
|
@llvm/pr-subscribers-llvm-support Author: Vitaly Buka (vitalybuka) ChangesCurrently SpecialCaseList created at least twice, Also, deppending on enabled sanitizers, not all In both cases there is unnecessary RadixTree This patch changes And remove empty one from This saves saves 0.5% of clang time building large project. Full diff: https://github.com/llvm/llvm-project/pull/167281.diff 1 Files Affected:
diff --git a/llvm/lib/Support/SpecialCaseList.cpp b/llvm/lib/Support/SpecialCaseList.cpp
index 2ba53d63365d2..79032cbb07f3f 100644
--- a/llvm/lib/Support/SpecialCaseList.cpp
+++ b/llvm/lib/Support/SpecialCaseList.cpp
@@ -20,6 +20,7 @@
#include "llvm/ADT/StringMap.h"
#include "llvm/ADT/StringRef.h"
#include "llvm/ADT/iterator_range.h"
+#include "llvm/Support/Compiler.h"
#include "llvm/Support/GlobPattern.h"
#include "llvm/Support/LineIterator.h"
#include "llvm/Support/MemoryBuffer.h"
@@ -42,10 +43,9 @@ namespace {
class RegexMatcher {
public:
Error insert(StringRef Pattern, unsigned LineNumber);
- void preprocess();
-
unsigned match(StringRef Query) const;
+private:
struct Reg {
Reg(StringRef Name, unsigned LineNo, Regex &&Rg)
: Name(Name), LineNo(LineNo), Rg(std::move(Rg)) {}
@@ -60,10 +60,9 @@ class RegexMatcher {
class GlobMatcher {
public:
Error insert(StringRef Pattern, unsigned LineNumber);
- void preprocess();
-
unsigned match(StringRef Query) const;
+private:
struct Glob {
Glob(StringRef Name, unsigned LineNo, GlobPattern &&Pattern)
: Name(Name), LineNo(LineNo), Pattern(std::move(Pattern)) {}
@@ -72,15 +71,20 @@ class GlobMatcher {
GlobPattern Pattern;
};
+ void LazyInit() const;
+
std::vector<GlobMatcher::Glob> Globs;
- RadixTree<iterator_range<StringRef::const_iterator>,
- RadixTree<iterator_range<StringRef::const_reverse_iterator>,
- SmallVector<int, 1>>>
+ mutable RadixTree<iterator_range<StringRef::const_iterator>,
+ RadixTree<iterator_range<StringRef::const_reverse_iterator>,
+ SmallVector<int, 1>>>
PrefixSuffixToGlob;
- RadixTree<iterator_range<StringRef::const_iterator>, SmallVector<int, 1>>
+ mutable RadixTree<iterator_range<StringRef::const_iterator>,
+ SmallVector<int, 1>>
SubstrToGlob;
+
+ mutable bool Initialized = false;
};
/// Represents a set of patterns and their line numbers
@@ -89,7 +93,6 @@ class Matcher {
Matcher(bool UseGlobs, bool RemoveDotSlash);
Error insert(StringRef Pattern, unsigned LineNumber);
- void preprocess();
unsigned match(StringRef Query) const;
bool matchAny(StringRef Query) const { return match(Query); }
@@ -122,8 +125,6 @@ Error RegexMatcher::insert(StringRef Pattern, unsigned LineNumber) {
return Error::success();
}
-void RegexMatcher::preprocess() {}
-
unsigned RegexMatcher::match(StringRef Query) const {
for (const auto &R : reverse(RegExes))
if (R.Rg.match(Query))
@@ -142,7 +143,10 @@ Error GlobMatcher::insert(StringRef Pattern, unsigned LineNumber) {
return Error::success();
}
-void GlobMatcher::preprocess() {
+void GlobMatcher::LazyInit() const {
+ if (LLVM_LIKELY(Initialized))
+ return;
+ Initialized = true;
for (const auto &[Idx, G] : enumerate(Globs)) {
StringRef Prefix = G.Pattern.prefix();
StringRef Suffix = G.Pattern.suffix();
@@ -167,6 +171,8 @@ void GlobMatcher::preprocess() {
}
unsigned GlobMatcher::match(StringRef Query) const {
+ LazyInit();
+
int Best = -1;
if (!PrefixSuffixToGlob.empty()) {
for (const auto &[_, SToGlob] : PrefixSuffixToGlob.find_prefixes(Query)) {
@@ -224,10 +230,6 @@ Error Matcher::insert(StringRef Pattern, unsigned LineNumber) {
return std::visit([&](auto &V) { return V.insert(Pattern, LineNumber); }, M);
}
-void Matcher::preprocess() {
- return std::visit([&](auto &V) { return V.preprocess(); }, M);
-}
-
unsigned Matcher::match(StringRef Query) const {
if (RemoveDotSlash)
Query = llvm::sys::path::remove_leading_dotslash(Query);
@@ -237,7 +239,6 @@ unsigned Matcher::match(StringRef Query) const {
class SpecialCaseList::Section::SectionImpl {
friend class SpecialCaseList;
- void preprocess();
const Matcher *findMatcher(StringRef Prefix, StringRef Category) const;
public:
@@ -410,9 +411,6 @@ bool SpecialCaseList::parse(unsigned FileIdx, const MemoryBuffer *MB,
}
}
- for (Section &S : Sections)
- S.Impl->preprocess();
-
return true;
}
@@ -462,13 +460,6 @@ SpecialCaseList::Section::SectionImpl::findMatcher(StringRef Prefix,
return &II->second;
}
-void SpecialCaseList::Section::SectionImpl::preprocess() {
- SectionMatcher.preprocess();
- for (auto &[K1, E] : Entries)
- for (auto &[K2, M] : E)
- M.preprocess();
-}
-
unsigned SpecialCaseList::Section::getLastMatch(StringRef Prefix,
StringRef Query,
StringRef Category) const {
|
vitalybuka
added a commit
to vitalybuka/llvm-project
that referenced
this pull request
Nov 10, 2025
Currently SpecialCaseList created at least twice, one on by `Driver`, for diagnostics only, and then the real one by the `ASTContext`. Also, deppending on enabled sanitizers, not all sections will be used. In both cases there is unnecessary RadixTree construction. This patch changes `GlobMatcher` to do initialization lazily only when needed. And remove empty one from `RegexMatcher`. This saves saves 0.5% of clang time building large project. Pull Request: llvm#167281
thurstond
approved these changes
Nov 10, 2025
Created using spr 1.3.7 [skip ci]
Created using spr 1.3.7 [skip ci]
vitalybuka
added a commit
to vitalybuka/llvm-project
that referenced
this pull request
Nov 12, 2025
Currently SpecialCaseList created at least twice, one on by `Driver`, for diagnostics only, and then the real one by the `ASTContext`. Also, deppending on enabled sanitizers, not all sections will be used. In both cases there is unnecessary RadixTree construction. This patch changes `GlobMatcher` to do initialization lazily only when needed. And remove empty one from `RegexMatcher`. This saves saves 0.5% of clang time building large project. Pull Request: llvm#167281
Created using spr 1.3.7 [skip ci]
vitalybuka
changed the base branch from
users/vitalybuka/spr/main.nfcspecialcaselist-convert-preprocess-into-lazyinit
to
main
Copilot finished reviewing on behalf of
vitalybuka
November 14, 2025 23:05
Contributor
There was a problem hiding this comment.
Pull Request Overview
This PR converts the preprocess() method in SpecialCaseList to use lazy initialization (LazyInit), aiming to improve performance by avoiding unnecessary RadixTree construction. The change eliminates eager preprocessing when SpecialCaseList instances are created but not fully utilized (e.g., when created by Driver for diagnostics only, or when certain sections are never queried).
Key changes:
- Removed
preprocess()methods from RegexMatcher, GlobMatcher, Matcher, and Section::SectionImpl classes - Implemented lazy initialization in GlobMatcher via a new
LazyInit()const method - Made RadixTree members and the
Initializedflag mutable to support lazy initialization from const methods
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
vitalybuka
deleted the
users/vitalybuka/spr/nfcspecialcaselist-convert-preprocess-into-lazyinit
branch
Collaborator
|
LLVM Buildbot has detected a new failure on builder Full details are available at: https://lab.llvm.org/buildbot/#/builders/10/builds/17365 Here is the relevant piece of the build log for the reference |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Currently SpecialCaseList created at least twice,
one on by
Driver, for diagnostics only, and thenthe real one by the
ASTContext.Also, deppending on enabled sanitizers, not all
sections will be used.
In both cases there is unnecessary RadixTree
construction.
This patch changes
GlobMatcherto do initializationlazily only when needed.
And remove empty one from
RegexMatcher.This saves saves 0.5% of clang time building large project.