Skip to content

[AMDGPU]: Unpack packed instructions overlapped by MFMAs post-RA scheduling #157968

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 18 commits into from
Sep 19, 2025
Merged

[AMDGPU]: Unpack packed instructions overlapped by MFMAs post-RA scheduling #157968

Show file tree
Hide file tree
Changes from 7 commits
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
c15104f
unpack packed instructions overlapped by MFMAs post-RA scheduling
akadutta Sep 10, 2025
fb05da7
format error fix and unpack candidate selection condition change
akadutta Sep 11, 2025
bbefcfb
code comments, variable and function re-naming, minor code optimizations
akadutta Sep 11, 2025
880ed6b
Merge branch 'main' into users/akadutta/amdgpu/unpack_non_coissue_ins…
akadutta Sep 11, 2025
f71631d
MIR tests for pk_add and pk_fma
akadutta Sep 11, 2025
6f5c6a5
check if unpacking can introduce dependencies, improve candidate sele…
akadutta Sep 12, 2025
ea85eb7
update MIR test cmd
akadutta Sep 12, 2025
4995f00
remove use of hard coded operand idx
akadutta Sep 12, 2025
05d0c75
update test
akadutta Sep 12, 2025
88e3c6d
direct and transitive dependency on MFMA def, reg kill logic, new tests
akadutta Sep 16, 2025
56b505f
update condition for MFMA dependence check
akadutta Sep 16, 2025
3ecc5da
update code comments
akadutta Sep 17, 2025
48cf50c
update return conditions, reduce LOC
akadutta Sep 18, 2025
022fa2d
new tests, reduced nesting, cleanup
akadutta Sep 18, 2025
720468f
update tests, update unpacking selection condition
akadutta Sep 18, 2025
4685911
minor fixes
akadutta Sep 18, 2025
3e016ca
update failing test
akadutta Sep 18, 2025
1b2bf71
clang-formatted
akadutta Sep 18, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
60 changes: 60 additions & 0 deletions llvm/lib/Target/AMDGPU/SIInstrInfo.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6359,6 +6359,66 @@ bool SIInstrInfo::isOperandLegal(const MachineInstr &MI, unsigned OpIdx,
return isImmOperandLegal(MI, OpIdx, *MO);
}

bool SIInstrInfo::isNeverCoissue(MachineInstr &MI) const {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it possible to extract this information from the scheduler model without maintaining this list of instructions? i.e. there should be some execution resource you can check if the instruction uses

bool IsGFX950Only = ST.hasGFX950Insts();
bool IsGFX940Only = ST.hasGFX940Insts();

if (!IsGFX950Only && !IsGFX940Only)
return false;

if (!isVALU(MI))
return false;

// V_COS, V_EXP, V_RCP, etc.
if (isTRANS(MI))
return true;

// DOT2, DOT2C, DOT4, etc.
if (isDOT(MI))
return true;

// MFMA, SMFMA
if (isMFMA(MI))
return true;

unsigned Opcode = MI.getOpcode();
switch (Opcode) {
case AMDGPU::V_CVT_PK_BF8_F32_e64:
case AMDGPU::V_CVT_PK_FP8_F32_e64:
case AMDGPU::V_MQSAD_PK_U16_U8_e64:
case AMDGPU::V_MQSAD_U32_U8_e64:
case AMDGPU::V_PK_ADD_F16:
case AMDGPU::V_PK_ADD_F32:
case AMDGPU::V_PK_ADD_I16:
case AMDGPU::V_PK_ADD_U16:
case AMDGPU::V_PK_ASHRREV_I16:
case AMDGPU::V_PK_FMA_F16:
case AMDGPU::V_PK_FMA_F32:
case AMDGPU::V_PK_FMAC_F16_e32:
case AMDGPU::V_PK_FMAC_F16_e64:
case AMDGPU::V_PK_LSHLREV_B16:
case AMDGPU::V_PK_LSHRREV_B16:
case AMDGPU::V_PK_MAD_I16:
case AMDGPU::V_PK_MAD_U16:
case AMDGPU::V_PK_MAX_F16:
case AMDGPU::V_PK_MAX_I16:
case AMDGPU::V_PK_MAX_U16:
case AMDGPU::V_PK_MIN_F16:
case AMDGPU::V_PK_MIN_I16:
case AMDGPU::V_PK_MIN_U16:
case AMDGPU::V_PK_MOV_B32:
case AMDGPU::V_PK_MUL_F16:
case AMDGPU::V_PK_MUL_F32:
case AMDGPU::V_PK_MUL_LO_U16:
case AMDGPU::V_PK_SUB_I16:
case AMDGPU::V_PK_SUB_U16:
case AMDGPU::V_QSAD_PK_U16_U8_e64:
return true;
default:
return false;
}
}

void SIInstrInfo::legalizeOperandsVOP2(MachineRegisterInfo &MRI,
MachineInstr &MI) const {
unsigned Opc = MI.getOpcode();
Expand Down
2 changes: 2 additions & 0 deletions llvm/lib/Target/AMDGPU/SIInstrInfo.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -1200,6 +1200,8 @@ class SIInstrInfo final : public AMDGPUGenInstrInfo {
return isImmOperandLegal(MI.getDesc(), OpNo, MO);
}

bool isNeverCoissue(MachineInstr &MI) const;

/// Check if this immediate value can be used for AV_MOV_B64_IMM_PSEUDO.
bool isLegalAV64PseudoImm(uint64_t Imm) const;

Expand Down
Loading