Skip to content

[SystemZ] Allow forming overflow op for i128 #153557

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
nikic merged 3 commits into from
Aug 14, 2025
Merged

[SystemZ] Allow forming overflow op for i128 #153557

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
9603b72
add tests
nikic Aug 14, 2025
56c3d37
Allow forming overflow op for i128
nikic Aug 14, 2025
6ea88ca
Add check lines without z13
nikic Aug 14, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion llvm/lib/Target/SystemZ/SystemZISelLowering.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -523,7 +523,7 @@ class SystemZTargetLowering : public TargetLowering {
bool MathUsed) const override {
// Form add and sub with overflow intrinsics regardless of any extra
// users of the math result.
return VT == MVT::i32 || VT == MVT::i64;
return VT == MVT::i32 || VT == MVT::i64 || VT == MVT::i128;
Copy link
Member

@uweigand uweigand Aug 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

That looks fine for z13 and higher, but on older machines, i128 is not actually a legal type, and we have no instructions for 128-bit operflowing ops. The common code shouldFormOverflowOp has a !isOperationExpand(Opcode, VT) check - I think we need that here for the i128 case.

Copy link
Contributor Author

@nikic nikic Aug 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I pushed additional check lines without z13. And here is the diff on top if we only do this for legal types: https://gist.github.com/nikic/6f84e1c7090a8c356f93e27fe0023a62

It looks like forming uaddo still results in a much better expansion even if we don't have legal i128. For usubo we get the same expansion either way (possibly there is a missed optimization opportunity in there?)

Copy link
Member

@uweigand uweigand Aug 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Huh, OK. That looks good then. Thanks!

}

bool shouldConsiderGEPOffsetSplit() const override { return true; }
Expand Down
13 changes: 8 additions & 5 deletions llvm/test/CodeGen/SystemZ/atomicrmw-ops-i128.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -363,10 +363,11 @@ define i128 @atomicrmw_uinc_wrap(ptr %src, i128 %b) {
define i128 @atomicrmw_udec_wrap(ptr %src, i128 %b) {
; CHECK-LABEL: atomicrmw_udec_wrap:
; CHECK: # %bb.0:
; CHECK-NEXT: larl %r1, .LCPI12_0
; CHECK-NEXT: vl %v0, 0(%r4), 3
; CHECK-NEXT: vl %v3, 0(%r3), 4
; CHECK-NEXT: vgbm %v1, 65535
; CHECK-NEXT: vgbm %v2, 0
; CHECK-NEXT: vl %v1, 0(%r1), 3
; CHECK-NEXT: vgbm %v2, 65535
; CHECK-NEXT: j .LBB12_2
; CHECK-NEXT: .LBB12_1: # %atomicrmw.start
; CHECK-NEXT: # in Loop: Header=BB12_2 Depth=1
Expand All @@ -379,6 +380,9 @@ define i128 @atomicrmw_udec_wrap(ptr %src, i128 %b) {
; CHECK-NEXT: je .LBB12_8
; CHECK-NEXT: .LBB12_2: # %atomicrmw.start
; CHECK-NEXT: # =>This Inner Loop Header: Depth=1
; CHECK-NEXT: vscbiq %v4, %v3, %v1
; CHECK-NEXT: vlgvf %r0, %v4, 3
; CHECK-NEXT: xilf %r0, 1
; CHECK-NEXT: veclg %v0, %v3
; CHECK-NEXT: jlh .LBB12_4
; CHECK-NEXT: # %bb.3: # %atomicrmw.start
Expand All @@ -390,12 +394,11 @@ define i128 @atomicrmw_udec_wrap(ptr %src, i128 %b) {
; CHECK-NEXT: jl .LBB12_6
; CHECK-NEXT: # %bb.5: # %atomicrmw.start
; CHECK-NEXT: # in Loop: Header=BB12_2 Depth=1
; CHECK-NEXT: vaq %v4, %v3, %v1
; CHECK-NEXT: vaq %v4, %v3, %v2
; CHECK-NEXT: .LBB12_6: # %atomicrmw.start
; CHECK-NEXT: # in Loop: Header=BB12_2 Depth=1
; CHECK-NEXT: vceqgs %v5, %v3, %v2
; CHECK-NEXT: vlr %v5, %v0
; CHECK-NEXT: je .LBB12_1
; CHECK-NEXT: cijlh %r0, 0, .LBB12_1
; CHECK-NEXT: # %bb.7: # %atomicrmw.start
; CHECK-NEXT: # in Loop: Header=BB12_2 Depth=1
; CHECK-NEXT: vlr %v5, %v4
Expand Down
259 changes: 216 additions & 43 deletions llvm/test/CodeGen/SystemZ/int-cmp-65.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,87 +1,260 @@
; NOTE: Assertions have been autogenerated by utils/update_llc_test_checks.py UTC_ARGS: --version 5
; Test usage of VACC/VSCBI.
;
; RUN: llc < %s -mtriple=s390x-linux-gnu -mcpu=z13 | FileCheck %s
; RUN: llc < %s -mtriple=s390x-linux-gnu | FileCheck %s --check-prefix=BASELINE
; RUN: llc < %s -mtriple=s390x-linux-gnu -mcpu=z13 | FileCheck %s --check-prefix=Z13

define i128 @i128_subc_1(i128 %a, i128 %b) unnamed_addr {
; CHECK-LABEL: i128_subc_1:
; CHECK: # %bb.0:
; CHECK-NEXT: vl %v0, 0(%r4), 3
; CHECK-NEXT: vl %v1, 0(%r3), 3
; CHECK-NEXT: vscbiq %v0, %v1, %v0
; CHECK-NEXT: vst %v0, 0(%r2), 3
; CHECK-NEXT: br %r14
; BASELINE-LABEL: i128_subc_1:
; BASELINE: # %bb.0:
; BASELINE-NEXT: stmg %r14, %r15, 112(%r15)
; BASELINE-NEXT: .cfi_offset %r14, -48
; BASELINE-NEXT: .cfi_offset %r15, -40
; BASELINE-NEXT: lg %r5, 0(%r4)
; BASELINE-NEXT: lg %r14, 0(%r3)
; BASELINE-NEXT: lg %r1, 8(%r3)
; BASELINE-NEXT: clgr %r14, %r5
; BASELINE-NEXT: ipm %r0
; BASELINE-NEXT: clg %r1, 8(%r4)
; BASELINE-NEXT: ipm %r1
; BASELINE-NEXT: cgrjlh %r14, %r5, .LBB0_2
; BASELINE-NEXT: # %bb.1:
; BASELINE-NEXT: xilf %r1, 4294967295
; BASELINE-NEXT: risbg %r0, %r1, 63, 191, 36
; BASELINE-NEXT: j .LBB0_3
; BASELINE-NEXT: .LBB0_2:
; BASELINE-NEXT: xilf %r0, 4294967295
; BASELINE-NEXT: risbg %r0, %r0, 63, 191, 36
; BASELINE-NEXT: .LBB0_3:
; BASELINE-NEXT: llgfr %r0, %r0
; BASELINE-NEXT: stg %r0, 8(%r2)
; BASELINE-NEXT: mvghi 0(%r2), 0
; BASELINE-NEXT: lmg %r14, %r15, 112(%r15)
; BASELINE-NEXT: br %r14
;
; Z13-LABEL: i128_subc_1:
; Z13: # %bb.0:
; Z13-NEXT: vl %v0, 0(%r4), 3
; Z13-NEXT: vl %v1, 0(%r3), 3
; Z13-NEXT: vscbiq %v0, %v1, %v0
; Z13-NEXT: vst %v0, 0(%r2), 3
; Z13-NEXT: br %r14
%cmp = icmp uge i128 %a, %b
%ext = zext i1 %cmp to i128
ret i128 %ext
}

define i128 @i128_subc_2(i128 %a, i128 %b) unnamed_addr {
; CHECK-LABEL: i128_subc_2:
; CHECK: # %bb.0:
; CHECK-NEXT: vl %v0, 0(%r3), 3
; CHECK-NEXT: vl %v1, 0(%r4), 3
; CHECK-NEXT: vscbiq %v0, %v1, %v0
; CHECK-NEXT: vst %v0, 0(%r2), 3
; CHECK-NEXT: br %r14
; BASELINE-LABEL: i128_subc_2:
; BASELINE: # %bb.0:
; BASELINE-NEXT: stmg %r14, %r15, 112(%r15)
; BASELINE-NEXT: .cfi_offset %r14, -48
; BASELINE-NEXT: .cfi_offset %r15, -40
; BASELINE-NEXT: lg %r5, 0(%r4)
; BASELINE-NEXT: lg %r14, 0(%r3)
; BASELINE-NEXT: lg %r0, 8(%r3)
; BASELINE-NEXT: clgr %r14, %r5
; BASELINE-NEXT: ipm %r1
; BASELINE-NEXT: clg %r0, 8(%r4)
; BASELINE-NEXT: ipm %r0
; BASELINE-NEXT: cgrjlh %r14, %r5, .LBB1_2
; BASELINE-NEXT: # %bb.1:
; BASELINE-NEXT: afi %r0, -536870912
; BASELINE-NEXT: srl %r0, 31
; BASELINE-NEXT: j .LBB1_3
; BASELINE-NEXT: .LBB1_2:
; BASELINE-NEXT: afi %r1, -536870912
; BASELINE-NEXT: srl %r1, 31
; BASELINE-NEXT: lr %r0, %r1
; BASELINE-NEXT: .LBB1_3:
; BASELINE-NEXT: llgfr %r0, %r0
; BASELINE-NEXT: stg %r0, 8(%r2)
; BASELINE-NEXT: mvghi 0(%r2), 0
; BASELINE-NEXT: lmg %r14, %r15, 112(%r15)
; BASELINE-NEXT: br %r14
;
; Z13-LABEL: i128_subc_2:
; Z13: # %bb.0:
; Z13-NEXT: vl %v0, 0(%r3), 3
; Z13-NEXT: vl %v1, 0(%r4), 3
; Z13-NEXT: vscbiq %v0, %v1, %v0
; Z13-NEXT: vst %v0, 0(%r2), 3
; Z13-NEXT: br %r14
%cmp = icmp ule i128 %a, %b
%ext = zext i1 %cmp to i128
ret i128 %ext
}

define i128 @i128_addc_1(i128 %a, i128 %b) {
; CHECK-LABEL: i128_addc_1:
; CHECK: # %bb.0:
; CHECK-NEXT: vl %v0, 0(%r4), 3
; CHECK-NEXT: vl %v1, 0(%r3), 3
; CHECK-NEXT: vaccq %v0, %v1, %v0
; CHECK-NEXT: vst %v0, 0(%r2), 3
; CHECK-NEXT: br %r14
; BASELINE-LABEL: i128_addc_1:
; BASELINE: # %bb.0:
; BASELINE-NEXT: lg %r0, 8(%r3)
; BASELINE-NEXT: lg %r1, 0(%r3)
; BASELINE-NEXT: alg %r0, 8(%r4)
; BASELINE-NEXT: alcg %r1, 0(%r4)
; BASELINE-NEXT: ipm %r0
; BASELINE-NEXT: risbg %r0, %r0, 63, 191, 35
; BASELINE-NEXT: stg %r0, 8(%r2)
; BASELINE-NEXT: mvghi 0(%r2), 0
; BASELINE-NEXT: br %r14
;
; Z13-LABEL: i128_addc_1:
; Z13: # %bb.0:
; Z13-NEXT: vl %v0, 0(%r4), 3
; Z13-NEXT: vl %v1, 0(%r3), 3
; Z13-NEXT: vaccq %v0, %v1, %v0
; Z13-NEXT: vst %v0, 0(%r2), 3
; Z13-NEXT: br %r14
%sum = add i128 %a, %b
%cmp = icmp ult i128 %sum, %a
%ext = zext i1 %cmp to i128
ret i128 %ext
}

define i128 @i128_addc_2(i128 %a, i128 %b) {
; CHECK-LABEL: i128_addc_2:
; CHECK: # %bb.0:
; CHECK-NEXT: vl %v0, 0(%r4), 3
; CHECK-NEXT: vl %v1, 0(%r3), 3
; CHECK-NEXT: vaccq %v0, %v1, %v0
; CHECK-NEXT: vst %v0, 0(%r2), 3
; CHECK-NEXT: br %r14
; BASELINE-LABEL: i128_addc_2:
; BASELINE: # %bb.0:
; BASELINE-NEXT: lg %r0, 8(%r3)
; BASELINE-NEXT: lg %r1, 0(%r3)
; BASELINE-NEXT: alg %r0, 8(%r4)
; BASELINE-NEXT: alcg %r1, 0(%r4)
; BASELINE-NEXT: ipm %r0
; BASELINE-NEXT: risbg %r0, %r0, 63, 191, 35
; BASELINE-NEXT: stg %r0, 8(%r2)
; BASELINE-NEXT: mvghi 0(%r2), 0
; BASELINE-NEXT: br %r14
;
; Z13-LABEL: i128_addc_2:
; Z13: # %bb.0:
; Z13-NEXT: vl %v0, 0(%r4), 3
; Z13-NEXT: vl %v1, 0(%r3), 3
; Z13-NEXT: vaccq %v0, %v1, %v0
; Z13-NEXT: vst %v0, 0(%r2), 3
; Z13-NEXT: br %r14
%sum = add i128 %a, %b
%cmp = icmp ult i128 %sum, %b
%ext = zext i1 %cmp to i128
ret i128 %ext
}

define i128 @i128_addc_3(i128 %a, i128 %b) {
; CHECK-LABEL: i128_addc_3:
; CHECK: # %bb.0:
; CHECK-NEXT: vl %v0, 0(%r4), 3
; CHECK-NEXT: vl %v1, 0(%r3), 3
; CHECK-NEXT: vaccq %v0, %v1, %v0
; CHECK-NEXT: vst %v0, 0(%r2), 3
; CHECK-NEXT: br %r14
; BASELINE-LABEL: i128_addc_3:
; BASELINE: # %bb.0:
; BASELINE-NEXT: lg %r0, 8(%r3)
; BASELINE-NEXT: lg %r1, 0(%r3)
; BASELINE-NEXT: alg %r0, 8(%r4)
; BASELINE-NEXT: alcg %r1, 0(%r4)
; BASELINE-NEXT: ipm %r0
; BASELINE-NEXT: risbg %r0, %r0, 63, 191, 35
; BASELINE-NEXT: stg %r0, 8(%r2)
; BASELINE-NEXT: mvghi 0(%r2), 0
; BASELINE-NEXT: br %r14
;
; Z13-LABEL: i128_addc_3:
; Z13: # %bb.0:
; Z13-NEXT: vl %v0, 0(%r4), 3
; Z13-NEXT: vl %v1, 0(%r3), 3
; Z13-NEXT: vaccq %v0, %v1, %v0
; Z13-NEXT: vst %v0, 0(%r2), 3
; Z13-NEXT: br %r14
%sum = add i128 %a, %b
%cmp = icmp ugt i128 %a, %sum
%ext = zext i1 %cmp to i128
ret i128 %ext
}

define i128 @i128_addc_4(i128 %a, i128 %b) {
; CHECK-LABEL: i128_addc_4:
; CHECK: # %bb.0:
; CHECK-NEXT: vl %v0, 0(%r4), 3
; CHECK-NEXT: vl %v1, 0(%r3), 3
; CHECK-NEXT: vaccq %v0, %v1, %v0
; CHECK-NEXT: vst %v0, 0(%r2), 3
; CHECK-NEXT: br %r14
; BASELINE-LABEL: i128_addc_4:
; BASELINE: # %bb.0:
; BASELINE-NEXT: lg %r0, 8(%r3)
; BASELINE-NEXT: lg %r1, 0(%r3)
; BASELINE-NEXT: alg %r0, 8(%r4)
; BASELINE-NEXT: alcg %r1, 0(%r4)
; BASELINE-NEXT: ipm %r0
; BASELINE-NEXT: risbg %r0, %r0, 63, 191, 35
; BASELINE-NEXT: stg %r0, 8(%r2)
; BASELINE-NEXT: mvghi 0(%r2), 0
; BASELINE-NEXT: br %r14
;
; Z13-LABEL: i128_addc_4:
; Z13: # %bb.0:
; Z13-NEXT: vl %v0, 0(%r4), 3
; Z13-NEXT: vl %v1, 0(%r3), 3
; Z13-NEXT: vaccq %v0, %v1, %v0
; Z13-NEXT: vst %v0, 0(%r2), 3
; Z13-NEXT: br %r14
%sum = add i128 %a, %b
%cmp = icmp ugt i128 %b, %sum
%ext = zext i1 %cmp to i128
ret i128 %ext
}

define i128 @i128_addc_xor(i128 %a, i128 %b) {
; BASELINE-LABEL: i128_addc_xor:
; BASELINE: # %bb.0:
; BASELINE-NEXT: lg %r0, 8(%r4)
; BASELINE-NEXT: lg %r1, 0(%r4)
; BASELINE-NEXT: alg %r0, 8(%r3)
; BASELINE-NEXT: alcg %r1, 0(%r3)
; BASELINE-NEXT: ipm %r0
; BASELINE-NEXT: risbg %r0, %r0, 63, 191, 35
; BASELINE-NEXT: stg %r0, 8(%r2)
; BASELINE-NEXT: mvghi 0(%r2), 0
; BASELINE-NEXT: br %r14
;
; Z13-LABEL: i128_addc_xor:
; Z13: # %bb.0:
; Z13-NEXT: vl %v0, 0(%r3), 3
; Z13-NEXT: vl %v1, 0(%r4), 3
; Z13-NEXT: vaccq %v0, %v1, %v0
; Z13-NEXT: vst %v0, 0(%r2), 3
; Z13-NEXT: br %r14
%b.not = xor i128 %b, -1
%cmp = icmp ugt i128 %a, %b.not
%ext = zext i1 %cmp to i128
ret i128 %ext
}

define i128 @i128_addc_xor_inv(i128 %a, i128 %b) {
; BASELINE-LABEL: i128_addc_xor_inv:
; BASELINE: # %bb.0:
; BASELINE-NEXT: stmg %r14, %r15, 112(%r15)
; BASELINE-NEXT: .cfi_offset %r14, -48
; BASELINE-NEXT: .cfi_offset %r15, -40
; BASELINE-NEXT: lg %r5, 0(%r3)
; BASELINE-NEXT: lghi %r14, -1
; BASELINE-NEXT: xg %r14, 0(%r4)
; BASELINE-NEXT: lghi %r1, -1
; BASELINE-NEXT: xg %r1, 8(%r4)
; BASELINE-NEXT: clgr %r5, %r14
; BASELINE-NEXT: ipm %r0
; BASELINE-NEXT: clg %r1, 8(%r3)
; BASELINE-NEXT: ipm %r1
; BASELINE-NEXT: cgrjlh %r5, %r14, .LBB7_2
; BASELINE-NEXT: # %bb.1:
; BASELINE-NEXT: xilf %r1, 4294967295
; BASELINE-NEXT: risbg %r0, %r1, 63, 191, 36
; BASELINE-NEXT: j .LBB7_3
; BASELINE-NEXT: .LBB7_2:
; BASELINE-NEXT: afi %r0, -536870912
; BASELINE-NEXT: srl %r0, 31
; BASELINE-NEXT: .LBB7_3:
; BASELINE-NEXT: llgfr %r0, %r0
; BASELINE-NEXT: stg %r0, 8(%r2)
; BASELINE-NEXT: mvghi 0(%r2), 0
; BASELINE-NEXT: lmg %r14, %r15, 112(%r15)
; BASELINE-NEXT: br %r14
;
; Z13-LABEL: i128_addc_xor_inv:
; Z13: # %bb.0:
; Z13-NEXT: vl %v1, 0(%r4), 3
; Z13-NEXT: vl %v0, 0(%r3), 3
; Z13-NEXT: vno %v1, %v1, %v1
; Z13-NEXT: vscbiq %v0, %v1, %v0
; Z13-NEXT: vst %v0, 0(%r2), 3
; Z13-NEXT: br %r14
%b.not = xor i128 %b, -1
%cmp = icmp ule i128 %a, %b.not
%ext = zext i1 %cmp to i128
ret i128 %ext
}