Skip to content

[LAA] Keep pointer checks on partial analysis #139719

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
john-brawn-arm merged 9 commits into from
Jun 4, 2025
Merged

[LAA] Keep pointer checks on partial analysis #139719

Show file tree
Hide file tree
Changes from 2 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
babdff1
[LoopAccessAnalysis] Keep pointer checks on partial analysis
john-brawn-arm Mar 26, 2025
6dc3239
Add extra line to LoopAccessInfo::print when we generate some but not…
john-brawn-arm May 15, 2025
686b462
Merge branch 'main' into loop_access_analysis_partial_result
john-brawn-arm May 19, 2025
d78421a
Add HasCompletePtrRtChecking
john-brawn-arm May 20, 2025
be7173d
Merge branch 'main' into loop_access_analysis_partial_result
john-brawn-arm May 21, 2025
5da2cc9
Merge branch 'main' into HEAD
john-brawn-arm May 30, 2025
a77c6f9
Add an option to control whether partial results are allowed
john-brawn-arm May 21, 2025
32dcdac
Adjust based on review comments.
john-brawn-arm Jun 4, 2025
2b02a5b
Add an extra comment
john-brawn-arm Jun 4, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
9 changes: 7 additions & 2 deletions llvm/include/llvm/Analysis/LoopAccessAnalysis.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -192,7 +192,8 @@ class MemoryDepChecker {
/// of a write access.
void addAccess(LoadInst *LI);

/// Check whether the dependencies between the accesses are safe.
/// Check whether the dependencies between the accesses are safe, and records
/// the dependence information in Dependences if so.
///
/// Only checks sets with elements in \p CheckDeps.
bool areDepsSafe(const DepCandidates &AccessSets,
Expand Down Expand Up @@ -779,10 +780,14 @@ class LoopAccessInfo {

/// We need to check that all of the pointers in this list are disjoint
/// at runtime. Using std::unique_ptr to make using move ctor simpler.
/// This list may contain only partial information when we've failed to
/// analyze all the memory accesses in the loop (i.e. CanVecMem is false).
std::unique_ptr<RuntimePointerChecking> PtrRtChecking;

/// the Memory Dependence Checker which can determine the
/// The Memory Dependence Checker which can determine the
/// loop-independent and loop-carried dependences between memory accesses.
/// This will be empty if we've failed to analyze all the memory access in the
/// loop (i.e. CanVecMem is false).
std::unique_ptr<MemoryDepChecker> DepChecker;

Loop *TheLoop;
Expand Down
21 changes: 11 additions & 10 deletions llvm/lib/Analysis/LoopAccessAnalysis.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -529,8 +529,10 @@ void RuntimePointerChecking::groupChecks(
// equivalence class, the iteration order is deterministic.
for (auto M : DepCands.members(Access)) {
auto PointerI = PositionMap.find(M.getPointer());
assert(PointerI != PositionMap.end() &&
"pointer in equivalence class not found in PositionMap");
// If we can't find the pointer in PositionMap that means we can't
// generate a memcheck for it.
if (PointerI == PositionMap.end())
continue;
for (unsigned Pointer : PointerI->second) {
bool Merged = false;
// Mark this pointer as seen.
Expand Down Expand Up @@ -682,7 +684,9 @@ class AccessAnalysis {
/// non-intersection.
///
/// Returns true if we need no check or if we do and we can generate them
/// (i.e. the pointers have computable bounds).
/// (i.e. the pointers have computable bounds). A return value of false means
/// we couldn't analyze and generate runtime checks for all pointers in the
/// loop, but we will have checks for those pointers we could analyze.
Copy link
Contributor

@artagnon artagnon Jun 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

... if \p AllowPartial is set?

bool canCheckPtrAtRT(RuntimePointerChecking &RtCheck, ScalarEvolution *SE,
Loop *TheLoop,
const DenseMap<Value *, const SCEV *> &Strides,
Expand Down Expand Up @@ -1273,7 +1277,6 @@ bool AccessAnalysis::canCheckPtrAtRT(
/*Assume=*/true)) {
CanDoAliasSetRT = false;
UncomputablePtr = Access.getPointer();
break;
}
}
}
Expand Down Expand Up @@ -1313,7 +1316,7 @@ bool AccessAnalysis::canCheckPtrAtRT(
}
}

if (MayNeedRTCheck && CanDoRT)
if (MayNeedRTCheck)
RtCheck.generateChecks(DepCands, IsDepCheckNeeded);

LLVM_DEBUG(dbgs() << "LAA: We need to do " << RtCheck.getNumberOfChecks()
Expand All @@ -1323,11 +1326,7 @@ bool AccessAnalysis::canCheckPtrAtRT(
// are needed. This can happen when all pointers point to the same underlying
// object for example.
RtCheck.Need = CanDoRT ? RtCheck.getNumberOfChecks() != 0 : MayNeedRTCheck;

Copy link
Contributor

@artagnon artagnon Jun 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Stray newline strip?

bool CanDoRTIfNeeded = !RtCheck.Need || CanDoRT;
if (!CanDoRTIfNeeded)
RtCheck.reset();
return CanDoRTIfNeeded;
return !RtCheck.Need || CanDoRT;
}

void AccessAnalysis::processMemAccesses() {
Expand Down Expand Up @@ -3002,6 +3001,8 @@ void LoopAccessInfo::print(raw_ostream &OS, unsigned Depth) const {

// List the pair of accesses need run-time checks to prove independence.
PtrRtChecking->print(OS, Depth);
if (PtrRtChecking->Need && !CanVecMem)
Copy link
Contributor

@artagnon artagnon May 15, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this check is a bit too lax, and generates spurious messages. Can you check CheckingGroups against Pointers instead?

Copy link
Collaborator Author

@john-brawn-arm john-brawn-arm May 22, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't work, as CheckingGroups is generated from Pointers in groupChecks, so there won't be any Pointer that doesn't have a corresponding CheckingGroup. I've gone instead with recording what the result of canCheckPtrAtRT was.

OS.indent(Depth) << "Generated run-time checks are incomplete\n";
OS << "\n";

OS.indent(Depth)
Expand Down
328 changes: 275 additions & 53 deletions llvm/test/Analysis/LoopAccessAnalysis/forked-pointers.ll
View file
Open in desktop

Large diffs are not rendered by default.

16 changes: 16 additions & 0 deletions llvm/test/Analysis/LoopAccessAnalysis/non-constant-strides-backward.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -237,6 +237,10 @@ define void @different_non_constant_strides_known_backward_via_assume(ptr %A, i6
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP1:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %A High: (1024 + %A))
; CHECK-NEXT: Member: {%A,+,4}<nuw><%loop>
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -271,6 +275,10 @@ define void @different_non_constant_strides_known_backward_via_assume_distance_l
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP2:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %A High: (1024 + %A)<nuw>)
; CHECK-NEXT: Member: {%A,+,4}<nuw><%loop>
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -306,6 +314,10 @@ define void @different_non_constant_strides_known_backward_via_assume_min_distan
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP3:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %A High: (1024 + %A))
; CHECK-NEXT: Member: {%A,+,4}<nuw><%loop>
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -341,6 +353,10 @@ define void @different_non_constant_strides_not_known_backward(ptr %A, i64 %scal
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP4:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %A High: (1024 + %A))
; CHECK-NEXT: Member: {%A,+,4}<nuw><%loop>
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down
12 changes: 12 additions & 0 deletions llvm/test/Analysis/LoopAccessAnalysis/non-constant-strides-forward.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,10 @@ define void @different_non_constant_strides_known_forward_via_assume(ptr %A, i64
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP1:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %A High: (1024 + %A))
; CHECK-NEXT: Member: {%A,+,4}<nuw><%loop>
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -119,6 +123,10 @@ define void @different_non_constant_strides_known_forward_via_assume_min_distanc
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP2:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %A High: (1024 + %A))
; CHECK-NEXT: Member: {%A,+,4}<nuw><%loop>
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -154,6 +162,10 @@ define void @different_non_constant_strides_not_known_forward(ptr %A, i64 %scale
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP3:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %A High: (1024 + %A))
; CHECK-NEXT: Member: {%A,+,4}<nuw><%loop>
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down
1 change: 1 addition & 0 deletions llvm/test/Analysis/LoopAccessAnalysis/nusw-predicates.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -95,6 +95,7 @@ define void @int_and_multiple_pointer_predicates(ptr %v, ptr %w, i32 %N) {
; CHECK-NEXT: Group [[GRP5]]:
; CHECK-NEXT: (Low: %w High: (6 + (4 * (trunc i32 %N to i16)) + %w))
; CHECK-NEXT: Member: {%w,+,4}<%loop>
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down
25 changes: 16 additions & 9 deletions llvm/test/Analysis/LoopAccessAnalysis/pointer-phis.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -287,6 +287,7 @@ define i32 @store_with_pointer_phi_incoming_phi(ptr %A, ptr %B, ptr %C, i1 %c.0,
; CHECK-NEXT: (Low: %A High: (256000 + %A))
; CHECK-NEXT: Member: {%A,+,8}<nuw><%loop.header>
; CHECK-NEXT: Member: %A
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -370,6 +371,7 @@ define i32 @store_with_pointer_phi_incoming_phi_irreducible_cycle(ptr %A, ptr %B
; CHECK-NEXT: (Low: %A High: (256000 + %A))
; CHECK-NEXT: Member: {%A,+,8}<nuw><%loop.header>
; CHECK-NEXT: Member: %A
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -462,6 +464,10 @@ define i32 @store_with_pointer_phi_in_same_bb_use_other_phi(ptr %A, ptr %B, ptr
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP10:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %A High: (256000 + %A))
; CHECK-NEXT: Member: {%A,+,8}<nuw><%loop.header>
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -503,39 +509,40 @@ define void @phi_load_store_memdep_check(i1 %c, ptr %A, ptr %B, ptr %C) {
; CHECK-EMPTY:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Check 0:
; CHECK-NEXT: Comparing group ([[GRP10:0x[0-9a-f]+]]):
; CHECK-NEXT: Comparing group ([[GRP11:0x[0-9a-f]+]]):
; CHECK-NEXT: ptr %A
; CHECK-NEXT: ptr %A
; CHECK-NEXT: Against group ([[GRP11:0x[0-9a-f]+]]):
; CHECK-NEXT: Against group ([[GRP12:0x[0-9a-f]+]]):
; CHECK-NEXT: ptr %C
; CHECK-NEXT: ptr %C
; CHECK-NEXT: Check 1:
; CHECK-NEXT: Comparing group ([[GRP10]]):
; CHECK-NEXT: Comparing group ([[GRP11]]):
; CHECK-NEXT: ptr %A
; CHECK-NEXT: ptr %A
; CHECK-NEXT: Against group ([[GRP12:0x[0-9a-f]+]]):
; CHECK-NEXT: Against group ([[GRP13:0x[0-9a-f]+]]):
; CHECK-NEXT: ptr %B
; CHECK-NEXT: ptr %B
; CHECK-NEXT: Check 2:
; CHECK-NEXT: Comparing group ([[GRP11]]):
; CHECK-NEXT: Comparing group ([[GRP12]]):
; CHECK-NEXT: ptr %C
; CHECK-NEXT: ptr %C
; CHECK-NEXT: Against group ([[GRP12]]):
; CHECK-NEXT: Against group ([[GRP13]]):
; CHECK-NEXT: ptr %B
; CHECK-NEXT: ptr %B
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP10]]:
; CHECK-NEXT: Group [[GRP11]]:
; CHECK-NEXT: (Low: %A High: (2 + %A))
; CHECK-NEXT: Member: %A
; CHECK-NEXT: Member: %A
; CHECK-NEXT: Group [[GRP11]]:
; CHECK-NEXT: Group [[GRP12]]:
; CHECK-NEXT: (Low: %C High: (2 + %C))
; CHECK-NEXT: Member: %C
; CHECK-NEXT: Member: %C
; CHECK-NEXT: Group [[GRP12]]:
; CHECK-NEXT: Group [[GRP13]]:
; CHECK-NEXT: (Low: %B High: (2 + %B))
; CHECK-NEXT: Member: %B
; CHECK-NEXT: Member: %B
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down
16 changes: 16 additions & 0 deletions ...ysis/LoopAccessAnalysis/retry-runtime-checks-after-dependence-analysis-forked-pointers.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -167,6 +167,14 @@ define void @dependency_check_and_runtime_checks_needed_select_of_ptr_add_recs_m
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP9:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: ((4 * %offset) + %a) High: ((4 * %offset) + (4 * %n) + %a))
; CHECK-NEXT: Member: {((4 * %offset) + %a),+,4}<%loop>
; CHECK-NEXT: Group [[GRP10:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %a High: ((4 * %n) + %a))
; CHECK-NEXT: Member: {%a,+,4}<nuw><%loop>
; CHECK-NEXT: Member: {%a,+,4}<nuw><%loop>
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -208,6 +216,14 @@ define void @dependency_check_and_runtime_checks_needed_select_of_ptr_add_recs_m
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP11:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: ((4 * %offset) + %a) High: ((4 * %offset) + (4 * %n) + %a))
; CHECK-NEXT: Member: {((4 * %offset) + %a),+,4}<%loop>
; CHECK-NEXT: Group [[GRP12:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: %a High: ((4 * %n) + %a))
; CHECK-NEXT: Member: {%a,+,4}<nuw><%loop>
; CHECK-NEXT: Member: {%a,+,4}<nuw><%loop>
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down
16 changes: 16 additions & 0 deletions llvm/test/Analysis/LoopAccessAnalysis/underlying-object-loop-varying-phi.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -80,6 +80,10 @@ define void @indirect_ptr_recurrences_read_write_may_alias_no_tbaa(ptr %A, ptr %
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP1:0x[0-9a-f]+]]:
; CHECK-NEXT: (Low: (8 + %B)<nuw> High: (40 + %B))
; CHECK-NEXT: Member: {(8 + %B)<nuw>,+,8}<nuw><%loop>
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down Expand Up @@ -111,7 +115,19 @@ define void @indirect_ptr_recurrences_read_write_may_alias_different_obj(ptr %A,
; CHECK-NEXT: Report: cannot identify array bounds
; CHECK-NEXT: Dependences:
; CHECK-NEXT: Run-time memory checks:
; CHECK-NEXT: Check 0:
; CHECK-NEXT: Comparing group ([[GRP2:0x[0-9a-f]+]]):
; CHECK-NEXT: %gep.C = getelementptr inbounds ptr, ptr %C, i64 %iv
; CHECK-NEXT: Against group ([[GRP3:0x[0-9a-f]+]]):
; CHECK-NEXT: %gep.B = getelementptr inbounds ptr, ptr %B, i64 %iv
; CHECK-NEXT: Grouped accesses:
; CHECK-NEXT: Group [[GRP2]]:
; CHECK-NEXT: (Low: (8 + %C)<nuw> High: (36 + %C))
; CHECK-NEXT: Member: {(8 + %C)<nuw>,+,8}<nuw><%loop>
; CHECK-NEXT: Group [[GRP3]]:
; CHECK-NEXT: (Low: (8 + %B)<nuw> High: (40 + %B))
; CHECK-NEXT: Member: {(8 + %B)<nuw>,+,8}<nuw><%loop>
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down
1 change: 1 addition & 0 deletions llvm/test/Analysis/LoopAccessAnalysis/unsafe-and-rt-checks-convergent.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -40,6 +40,7 @@ define void @rtchecks_needed(ptr %a, ptr %b, ptr %c) {
; CHECK-NEXT: Group [[GRP3]]:
; CHECK-NEXT: (Low: %c High: (40 + %c))
; CHECK-NEXT: Member: {%c,+,2}<nuw><%for.body>
; CHECK-NEXT: Generated run-time checks are incomplete
; CHECK-EMPTY:
; CHECK-NEXT: Non vectorizable stores to invariant address were not found in loop.
; CHECK-NEXT: SCEV assumptions:
Expand Down
Loading