Skip to content

[mlir][rocdl] Add readfirstlane intrinsic #152551

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 3 commits into from
Aug 7, 2025
Merged

[mlir][rocdl] Add readfirstlane intrinsic #152551

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
59ec7ab
[mlir][rocdl] Add `readfirstlane` intrinsic
Hardcode84 Aug 7, 2025
53fc37b
drop pure
Hardcode84 Aug 7, 2025
c62230b
drop pure
Hardcode84 Aug 7, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
18 changes: 16 additions & 2 deletions mlir/include/mlir/Dialect/LLVMIR/ROCDLOps.td
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -189,7 +189,21 @@ def ROCDL_BallotOp :
let assemblyFormat = "$pred attr-dict `:` type($res)";
}

def ROCDL_ReadlaneOp : ROCDL_IntrOp<"readlane", [], [0], [AllTypesMatch<["res", "src0"]>], 1>,
def ROCDL_ReadfirstlaneOp : ROCDL_IntrOp<"readfirstlane", [], [0], [AllTypesMatch<["res", "src"]>, Pure], 1>,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IIUC, this relies on the execution mask to determine the first active lane. Do we really want this to be pure and enable speculation/hoisting, since hoisting outside of ifs/loops can affect active lanes?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok, making it loop-hoistable was my exact motivation, any ideas how to handle it?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I guess, I can make it non-pure for now and hoist manually on Wave side.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you can only hoist it if you can prove the control flow is uniform

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I see. Can we hoist readlane?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think structured control flow guarantees that your parents are at least as active as you, so this seems safe for me

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, this isn't hoistable in general, but you can slide out of a for loop or some other thing like that. That is, this isn't speculatable, but it does have no memory effects

Arguments<(ins LLVM_Type:$src)> {
let results = (outs LLVM_Type:$res);
let summary = "Get the value in first active lane.";

let description = [{
Returns the value in the lowest active lane of the input operand.
}];

let assemblyFormat = [{
$src attr-dict `:` type($res)
}];
}

def ROCDL_ReadlaneOp : ROCDL_IntrOp<"readlane", [], [0], [AllTypesMatch<["res", "src0"]>, Pure], 1>,
Arguments<(ins LLVM_Type:$src0,
I32:$src1)> {
let results = (outs LLVM_Type:$res);
Expand All @@ -201,7 +215,7 @@ def ROCDL_ReadlaneOp : ROCDL_IntrOp<"readlane", [], [0], [AllTypesMatch<["res",

let assemblyFormat = [{
$src0 `,` $src1 attr-dict `:` `(` type($src0) `,` type($src1) `)` `->` type($res)
}];
}];
}

//===----------------------------------------------------------------------===//
Expand Down
7 changes: 7 additions & 0 deletions mlir/test/Dialect/LLVMIR/rocdl.mlir
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -981,6 +981,13 @@ llvm.func @rocdl.s.wait.expcnt() {

// -----

llvm.func @rocdl.readfirstlane(%src : f32) -> f32 {
// CHECK-LABEL: rocdl.readfirstlane
// CHECK: rocdl.readfirstlane %{{.*}} : f32
%ret = rocdl.readfirstlane %src : f32
llvm.return %ret : f32
}

llvm.func @rocdl.readlane(%src : f32) -> f32 {
%cst0 = llvm.mlir.constant(0 : i32) : i32

Expand Down
17 changes: 17 additions & 0 deletions mlir/test/Target/LLVMIR/rocdl.mlir
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -125,6 +125,23 @@ llvm.func @rocdl.ballot64(%pred : i1) -> i64 {
llvm.return %0 : i64
}

llvm.func @rocdl.readfirstlane(%src0 : f32, %src1: f64, %src2: i32, %src3: vector<2 x f32>) -> f32 {
// CHECK-LABEL: rocdl.readfirstlane
// CHECK: call float @llvm.amdgcn.readfirstlane.f32(float %{{.*}})
%0 = rocdl.readfirstlane %src0 : f32

// CHECK: call double @llvm.amdgcn.readfirstlane.f64(double %{{.*}})
%1 = rocdl.readfirstlane %src1 : f64

// CHECK: call i32 @llvm.amdgcn.readfirstlane.i32(i32 %{{.*}})
%2 = rocdl.readfirstlane %src2 : i32

// CHECK: call <2 x float> @llvm.amdgcn.readfirstlane.v2f32(<2 x float> %{{.*}})
%3 = rocdl.readfirstlane %src3 : vector<2 x f32>

llvm.return %0 : f32
}

llvm.func @rocdl.readlane(%src0 : f32, %src1: f64, %src2: i32, %src3: vector<2 x f32>) -> f32 {
%idx = llvm.mlir.constant(0 : i32) : i32

Expand Down