Skip to content

Commit 6bef118

Browse files
committed
[RISCV] Allow hoisting VXRM writes out of loops speculatively
Change the intersect for the anticipated algorithm to ignore unknown when anticipating. This effectively allows VXRM writes speculatively because it could do a VXRM write even when there's branches where VXRM is unneeded. The importance of this change is because VXRM writes causes pipeline flushes in some micro-architectures and so it makes sense to allow more aggressive hoisting even if it causes some degradation for the slow path. An example is this code: ``` typedef unsigned char uint8_t; __attribute__ ((noipa)) void foo (uint8_t *dst, int i_dst_stride, uint8_t *src1, int i_src1_stride, uint8_t *src2, int i_src2_stride, int i_width, int i_height ) { for( int y = 0; y < i_height; y++ ) { for( int x = 0; x < i_width; x++ ) dst[x] = ( src1[x] + src2[x] + 1 ) >> 1; dst += i_dst_stride; src1 += i_src1_stride; src2 += i_src2_stride; } } ``` With this patch, the code above generates a hoisting VXRM writes out of the outer loop.
1 parent 7645d9c commit 6bef118

File tree

3 files changed

+420
-6
lines changed

3 files changed

+420
-6
lines changed

llvm/lib/Target/RISCV/RISCVInsertWriteVXRM.cpp

Lines changed: 30 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -109,6 +109,35 @@ class VXRMInfo {
109109
return VXRMInfo::getUnknown();
110110
}
111111

112+
// Calculate the VXRMInfo visible to a block assuming this and Other
113+
// are both predecessors. To allow speculatively running WriteVXRM
114+
// we will ignore Unknowns if one of this and Other have valid
115+
// WriteVXRM. Rationale: WriteVXRM causes a pipeline flush in some
116+
// uarchs and moving it outside loops is very important for some
117+
// workloads.
118+
VXRMInfo intersectAnticipated(const VXRMInfo &Other) const {
119+
// If the new value isn't valid, ignore it.
120+
if (!Other.isValid())
121+
return *this;
122+
123+
// If this value isn't valid, this must be the first predecessor, use it.
124+
if (!isValid())
125+
return Other;
126+
127+
// If either is unknown, the result is the other one.
128+
if (isUnknown())
129+
return Other;
130+
else if (Other.isUnknown())
131+
return *this;
132+
133+
// If we have an exact match, return this.
134+
if (*this == Other)
135+
return *this;
136+
137+
// Otherwise the result is unknown.
138+
return VXRMInfo::getUnknown();
139+
}
140+
112141
#if !defined(NDEBUG) || defined(LLVM_ENABLE_DUMP)
113142
/// Support for debugging, callable in GDB: V->dump()
114143
LLVM_DUMP_METHOD void dump() const {
@@ -290,7 +319,7 @@ void RISCVInsertWriteVXRM::computeAnticipated(const MachineBasicBlock &MBB) {
290319
} else {
291320
for (const MachineBasicBlock *S : MBB.successors())
292321
Anticipated =
293-
Anticipated.intersect(BlockInfo[S->getNumber()].AnticipatedIn);
322+
Anticipated.intersectAnticipated(BlockInfo[S->getNumber()].AnticipatedIn);
294323
}
295324

296325
// If we don't have any valid anticipated info, wait until we do.

0 commit comments

Comments
 (0)