[AMDGPU] Unexpected occupancy calculation w.r.t. LDS size

[lucas-rami]

I am struggling to understand the logic behind the computation of the occupancy with respect to the LDS size for a given function, in particular how it fits in the overall occupancy computation for the function. In particular, it seems to me that there is a discrepancy in the meaning of the achievable occupancy number computed with respect to the number of used SGPRs/VGPRs as opposed to LDS size. Current implementation of GCNSubtarget::computeOccupancy pasted below for future reference.

unsigned GCNSubtarget::computeOccupancy(const Function &F, unsigned LDSSize,
                                        unsigned NumSGPRs,
                                        unsigned NumVGPRs) const {
  unsigned Occupancy =
      std::min(getMaxWavesPerEU(), getOccupancyWithLocalMemSize(LDSSize, F));
  if (NumSGPRs)
    Occupancy = std::min(Occupancy, getOccupancyWithNumSGPRs(NumSGPRs));
  if (NumVGPRs)
    Occupancy = std::min(Occupancy, getOccupancyWithNumVGPRs(NumVGPRs));
  return Occupancy;
}

On gfx908 for example, getMaxWavesPerEU() == 10. If the number of SGPRs and VGPRs used in the function is low enough to support maximum occupancy, getOccupancyWithNumSGPRs(NumSGPRs) == getOccupancyWithNumVGPRs(NumVGPRs) == 10 as well, which makes sense to me. However, even if LDSSize == 0 (or any low enough number so that the LDS should not restrict occupancy), getOccupancyWithLocalMemSize(LDSSize, F) == 8, and as a consequence GCNSubtarget::computeOccupancy returns 8. This seems to be because getOccupancyWithLocalMemSize performs a semantically different calculation that the two other occupancy-computing methods (which as the FIXMEs in the method suggest is perhaps not the one we should do). I would have assumed that no or low LDS usage would return the same number as getMaxWavesPerEU().

I just wanted to check if I am missing something here or if there is indeed a problem in this context.

[llvmbot]

@llvm/issue-subscribers-backend-amdgpu

Author: Lucas Ramirez (lucas-rami)

I am struggling to understand the logic behind the computation of the occupancy with respect to the LDS size for a given function, in particular how it fits in the overall occupancy computation for the function. In particular, it seems to me that there is a discrepancy in the meaning of the achievable occupancy number computed with respect to the number of used SGPRs/VGPRs as opposed to LDS size. Current implementation of `GCNSubtarget::computeOccupancy` pasted below for future reference.

unsigned GCNSubtarget::computeOccupancy(const Function &F, unsigned LDSSize,
                                        unsigned NumSGPRs,
                                        unsigned NumVGPRs) const {
  unsigned Occupancy =
      std::min(getMaxWavesPerEU(), getOccupancyWithLocalMemSize(LDSSize, F));
  if (NumSGPRs)
    Occupancy = std::min(Occupancy, getOccupancyWithNumSGPRs(NumSGPRs));
  if (NumVGPRs)
    Occupancy = std::min(Occupancy, getOccupancyWithNumVGPRs(NumVGPRs));
  return Occupancy;
}

On gfx908 for example, getMaxWavesPerEU() == 10. If the number of SGPRs and VGPRs used in the function is low enough to support maximum occupancy, getOccupancyWithNumSGPRs(NumSGPRs) == getOccupancyWithNumVGPRs(NumVGPRs) == 10 as well, which makes sense to me. However, even if LDSSize == 0 (or any low enough number so that the LDS should not restrict occupancy), getOccupancyWithLocalMemSize(LDSSize, F) == 8, and as a consequence GCNSubtarget::computeOccupancy returns 8. This seems to be because getOccupancyWithLocalMemSize performs a semantically different calculation that the two other occupancy-computing methods (which as the FIXMEs in the method suggest is perhaps not the one we should do). I would have assumed that no or low LDS usage would return the same number as getMaxWavesPerEU().

I just wanted to check if I am missing something here or if there is indeed a problem in this context.

[arsenm]

An empty kernel receives an occupancy of 8, so something seems wrong

With the default 1024 workgroup size. If you use amdgpu-flat-work-group-size to restrict the max group size to 512 or less, you get 10. So it's the group size that's restricting the occupancy, and this should still be expressed as a range. It appears to be returning the maximum, but computeOccupancy seems to be reporting the minimum

[lucas-rami]

Thanks a lot for the insights, they helped clarify the intent of the calculation. I am working on a fix that will make both the getOccupancyWithLocalMemSize and computeOccupancy functions return a range, and hopefully clear up whether we want to use the minimum or maximum estimated occupancy in all uses of these functions.

[shiltian]

@lucas-rami are you working on this one? If not, can I pick it up?

[lucas-rami]

Yes I am actually nearing a patch on this one, I should be able to submit it later today.