Skip to content

[RISCV][CostModel] Add cost for fabs/fsqrt of type bf16/f16 #118608

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 9 commits into from
Jan 10, 2025
Merged

[RISCV][CostModel] Add cost for fabs/fsqrt of type bf16/f16 #118608

Show file tree
Hide file tree
Changes from 6 commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
48dfe67
[RISCV][CostModel] Add cost for fabs/fsqrt of type bf16/f16
LiqinWeng Dec 4, 2024
e682e00
Remove the condition for sqrts
LiqinWeng Dec 9, 2024
86bd2f7
Promote the bf16/f16 to fp32 to calculate the cost
LiqinWeng Dec 16, 2024
0b5f44e
Merge branch 'main' into fabs-fsqrt-f16-bf16-cost
LiqinWeng Dec 23, 2024
1fe4655
Add cost for fsqrt of type nxv32[b]f16
LiqinWeng Dec 27, 2024
b7c046f
update nxv32[b]f16 to nxv16f32
LiqinWeng Dec 29, 2024
3bb8859
Merge branch 'main' into fabs-fsqrt-f16-bf16-cost
LiqinWeng Jan 5, 2025
1a8c67e
fix the coments and clang-format for test
LiqinWeng Jan 6, 2025
5d2432f
fix the vtype for fcvt
LiqinWeng Jan 8, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
63 changes: 52 additions & 11 deletions llvm/lib/Target/RISCV/RISCVTargetTransformInfo.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,7 @@
#include "llvm/CodeGen/BasicTTIImpl.h"
#include "llvm/CodeGen/CostTable.h"
#include "llvm/CodeGen/TargetLowering.h"
#include "llvm/CodeGen/ValueTypes.h"
#include "llvm/IR/Instructions.h"
#include "llvm/IR/PatternMatch.h"
#include <cmath>
Expand Down Expand Up @@ -1035,21 +1036,61 @@ RISCVTTIImpl::getIntrinsicInstrCost(const IntrinsicCostAttributes &ICA,
}
break;
}
case Intrinsic::fabs:
case Intrinsic::fabs: {
auto LT = getTypeLegalizationCost(RetTy);
if (ST->hasVInstructions() && LT.second.isVector()) {
// lui a0, 8
// addi a0, a0, -1
// vsetvli a1, zero, e16, m1, ta, ma
// vand.vx v8, v8, a0
// f16 with zvfhmin and bf16 with zvfhbmin
if (LT.second.getVectorElementType() == MVT::bf16 ||
(LT.second.getVectorElementType() == MVT::f16 &&
!ST->hasVInstructionsF16()))
return LT.first * getRISCVInstructionCost(RISCV::VAND_VX, LT.second,
CostKind) +
2;
else
return LT.first *
getRISCVInstructionCost(RISCV::VFSGNJX_VV, LT.second, CostKind);
}
break;
}
case Intrinsic::sqrt: {
auto LT = getTypeLegalizationCost(RetTy);
// TODO: add f16/bf16, bf16 with zvfbfmin && f16 with zvfhmin
auto NVT = LT.second;
if (ST->hasVInstructions() && LT.second.isVector()) {
unsigned Op;
switch (ICA.getID()) {
case Intrinsic::fabs:
Op = RISCV::VFSGNJX_VV;
break;
case Intrinsic::sqrt:
Op = RISCV::VFSQRT_V;
break;
SmallVector<unsigned, 3> Opcodes;
// f16 with zvfhmin and bf16 with zvfbfmin and the type of nxv32[b]f16
// will be spilt.
if (LT.second.getVectorElementType() == MVT::bf16) {
if (LT.second == MVT::nxv32bf16) {
Opcodes = {RISCV::VFWCVTBF16_F_F_V, RISCV::VFWCVTBF16_F_F_V,
RISCV::VFSQRT_V, RISCV::VFSQRT_V,
RISCV::VFNCVTBF16_F_F_W, RISCV::VFNCVTBF16_F_F_W};
NVT = TLI->getTypeToPromoteTo(ISD::FSQRT,
NVT.getHalfNumVectorElementsVT());
} else {
Opcodes = {RISCV::VFWCVTBF16_F_F_V, RISCV::VFSQRT_V,
RISCV::VFNCVTBF16_F_F_W};
NVT = TLI->getTypeToPromoteTo(ISD::FSQRT, NVT);
}
} else if (LT.second.getVectorElementType() == MVT::f16 &&
!ST->hasVInstructionsF16()) {
if (LT.second == MVT::nxv32f16) {
Opcodes = {RISCV::VFWCVT_F_F_V, RISCV::VFWCVT_F_F_V,
RISCV::VFSQRT_V, RISCV::VFSQRT_V,
Copy link
Collaborator

@topperc topperc Dec 27, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NVT needs to be changed to v16f32.

Copy link
Contributor Author

@LiqinWeng LiqinWeng Dec 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

Copy link
Contributor Author

@LiqinWeng LiqinWeng Dec 29, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you help me see the code check? I feel that there is no problem with the code, but the code check fails.

Copy link
Contributor

@arcbbb arcbbb Dec 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LiqinWeng Are you trying to compute the cost for ISAs without f16 support as (vfwcvt, nxvf16) + (vfsqrt, nxvf32) + (vfncvt, nxv*f16)? From the current code, it doesn’t seem like that’s what’s being implemented.

Copy link
Contributor Author

@LiqinWeng LiqinWeng Dec 31, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Under what circumstances will it occur: (vfwcvt, nxvf16) + (vfsqrt, nxvf32) + (vfncvt, nxv*f16)？？？I don't quite understand ‘ISAs without f16’？ do you means the mattr only support zvfh？？？

Copy link
Contributor Author

@LiqinWeng LiqinWeng Jan 5, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's a vfwcvt.f.f.v from EEW=16 EMUL=m1 to EEW=32 EMUL=m2; Shouldn't the type of vfwcvt be nxvf32, the dest eew is 32

Copy link
Contributor

@arcbbb arcbbb Jan 7, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@LiqinWeng vfwcvt is a widening arithmetic instruction. Based on the spec description, the destination EEW is 2 * SEW, so the vtype is nxvf16 and the destination is nxvf32.

10.2. Widening Vector Arithmetic Instructions
A few vector arithmetic instructions are defined to be widening operations where the destination vector register group has EEW=2*SEW and EMUL=2*LMUL.

On the other hand, for narrowing arithmetic instruction such as vfncvt, the destination EEW is SEW, so both the vtype and the destination are nxvf16.

Copy link
Contributor Author

@LiqinWeng LiqinWeng Jan 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this the case with this instruction(vfwcvtbf16.f.f.v )?

Copy link
Contributor

@arcbbb arcbbb Jan 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm.. I think vfwcvtbf16.f.f.v is the same with vfwcvt.f.f.v.

4.4. vfwcvtbf16.f.f.v
Synopsis
  Vector convert BF16 to FP32
Description
  This instruction is similar to vfwcvt.f.f.v which converts a floating-point value in an SEW-width
format into a 2*SEW-width format. However, here the SEW-width format is limited to BF16.

Copy link
Contributor Author

@LiqinWeng LiqinWeng Jan 8, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fixed

RISCV::VFNCVT_F_F_W, RISCV::VFNCVT_F_F_W};
NVT = TLI->getTypeToPromoteTo(ISD::FSQRT,
Copy link
Collaborator

@topperc topperc Jan 2, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can't use getTypeToPromoteTo. That only works when LegalizeVectorOps/LegalizeDAG does the promotion via setOperationAction(ISD::FSQRT, VT, Promote). This case uses setOperationAction(ISD::FSQRT, MVT::nxv32f16, Custom) and there is custom code in RISCVISelLowering.cpp to do the splitting. You'll need to hardcode the type has MVT::nxv16f32.

Copy link
Contributor Author

@LiqinWeng LiqinWeng Jan 6, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

done

NVT.getHalfNumVectorElementsVT());
} else {
Opcodes = {RISCV::VFWCVT_F_F_V, RISCV::VFSQRT_V, RISCV::VFNCVT_F_F_W};
NVT = TLI->getTypeToPromoteTo(ISD::FSQRT, NVT);
}
} else {
Opcodes = {RISCV::VFSQRT_V};
}
return LT.first * getRISCVInstructionCost(Op, LT.second, CostKind);
return LT.first * getRISCVInstructionCost(Opcodes, NVT, CostKind);
}
break;
}
Expand Down
50 changes: 31 additions & 19 deletions llvm/test/Analysis/CostModel/RISCV/fp-min-max-abs.ll
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,14 +5,14 @@
define void @fabs() {
; CHECK-LABEL: 'fabs'
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %1 = call bfloat @llvm.fabs.bf16(bfloat undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %2 = call <2 x bfloat> @llvm.fabs.v2bf16(<2 x bfloat> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %3 = call <4 x bfloat> @llvm.fabs.v4bf16(<4 x bfloat> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %4 = call <8 x bfloat> @llvm.fabs.v8bf16(<8 x bfloat> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %5 = call <16 x bfloat> @llvm.fabs.v16bf16(<16 x bfloat> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %6 = call <vscale x 2 x bfloat> @llvm.fabs.nxv2bf16(<vscale x 2 x bfloat> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %7 = call <vscale x 4 x bfloat> @llvm.fabs.nxv4bf16(<vscale x 4 x bfloat> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %8 = call <vscale x 8 x bfloat> @llvm.fabs.nxv8bf16(<vscale x 8 x bfloat> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %9 = call <vscale x 16 x bfloat> @llvm.fabs.nxv16bf16(<vscale x 16 x bfloat> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %2 = call <2 x bfloat> @llvm.fabs.v2bf16(<2 x bfloat> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %3 = call <4 x bfloat> @llvm.fabs.v4bf16(<4 x bfloat> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %4 = call <8 x bfloat> @llvm.fabs.v8bf16(<8 x bfloat> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %5 = call <16 x bfloat> @llvm.fabs.v16bf16(<16 x bfloat> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %6 = call <vscale x 2 x bfloat> @llvm.fabs.nxv2bf16(<vscale x 2 x bfloat> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %7 = call <vscale x 4 x bfloat> @llvm.fabs.nxv4bf16(<vscale x 4 x bfloat> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %8 = call <vscale x 8 x bfloat> @llvm.fabs.nxv8bf16(<vscale x 8 x bfloat> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %9 = call <vscale x 16 x bfloat> @llvm.fabs.nxv16bf16(<vscale x 16 x bfloat> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %10 = call float @llvm.fabs.f32(float undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %11 = call <2 x float> @llvm.fabs.v2f32(<2 x float> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %12 = call <4 x float> @llvm.fabs.v4f32(<4 x float> undef)
Expand Down Expand Up @@ -66,17 +66,29 @@ define void @fabs() {
}

define void @fabs_f16() {
; CHECK-LABEL: 'fabs_f16'
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %1 = call half @llvm.fabs.f16(half undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %2 = call <2 x half> @llvm.fabs.v2f16(<2 x half> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %3 = call <4 x half> @llvm.fabs.v4f16(<4 x half> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %4 = call <8 x half> @llvm.fabs.v8f16(<8 x half> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %5 = call <16 x half> @llvm.fabs.v16f16(<16 x half> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %6 = call <vscale x 2 x half> @llvm.fabs.nxv2f16(<vscale x 2 x half> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %7 = call <vscale x 4 x half> @llvm.fabs.nxv4f16(<vscale x 4 x half> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %8 = call <vscale x 8 x half> @llvm.fabs.nxv8f16(<vscale x 8 x half> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %9 = call <vscale x 16 x half> @llvm.fabs.nxv16f16(<vscale x 16 x half> undef)
; CHECK-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
; ZVFH-LABEL: 'fabs_f16'
; ZVFH-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %1 = call half @llvm.fabs.f16(half undef)
; ZVFH-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %2 = call <2 x half> @llvm.fabs.v2f16(<2 x half> undef)
; ZVFH-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %3 = call <4 x half> @llvm.fabs.v4f16(<4 x half> undef)
; ZVFH-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %4 = call <8 x half> @llvm.fabs.v8f16(<8 x half> undef)
; ZVFH-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %5 = call <16 x half> @llvm.fabs.v16f16(<16 x half> undef)
; ZVFH-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %6 = call <vscale x 2 x half> @llvm.fabs.nxv2f16(<vscale x 2 x half> undef)
; ZVFH-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %7 = call <vscale x 4 x half> @llvm.fabs.nxv4f16(<vscale x 4 x half> undef)
; ZVFH-NEXT: Cost Model: Found an estimated cost of 2 for instruction: %8 = call <vscale x 8 x half> @llvm.fabs.nxv8f16(<vscale x 8 x half> undef)
; ZVFH-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %9 = call <vscale x 16 x half> @llvm.fabs.nxv16f16(<vscale x 16 x half> undef)
; ZVFH-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
;
; ZVFHMIN-LABEL: 'fabs_f16'
; ZVFHMIN-NEXT: Cost Model: Found an estimated cost of 1 for instruction: %1 = call half @llvm.fabs.f16(half undef)
; ZVFHMIN-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %2 = call <2 x half> @llvm.fabs.v2f16(<2 x half> undef)
; ZVFHMIN-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %3 = call <4 x half> @llvm.fabs.v4f16(<4 x half> undef)
; ZVFHMIN-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %4 = call <8 x half> @llvm.fabs.v8f16(<8 x half> undef)
; ZVFHMIN-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %5 = call <16 x half> @llvm.fabs.v16f16(<16 x half> undef)
; ZVFHMIN-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %6 = call <vscale x 2 x half> @llvm.fabs.nxv2f16(<vscale x 2 x half> undef)
; ZVFHMIN-NEXT: Cost Model: Found an estimated cost of 3 for instruction: %7 = call <vscale x 4 x half> @llvm.fabs.nxv4f16(<vscale x 4 x half> undef)
; ZVFHMIN-NEXT: Cost Model: Found an estimated cost of 4 for instruction: %8 = call <vscale x 8 x half> @llvm.fabs.nxv8f16(<vscale x 8 x half> undef)
; ZVFHMIN-NEXT: Cost Model: Found an estimated cost of 6 for instruction: %9 = call <vscale x 16 x half> @llvm.fabs.nxv16f16(<vscale x 16 x half> undef)
; ZVFHMIN-NEXT: Cost Model: Found an estimated cost of 0 for instruction: ret void
;
call half @llvm.fabs.f16(half undef)
call <2 x half> @llvm.fabs.v2f16(<2 x half> undef)
Expand Down
Loading
Loading