Skip to content

Conversation

@goplanid
Copy link
Contributor

@goplanid goplanid commented Jan 23, 2025

This change modifies the packing condition in TT and TN small gemm kernels and helps to improve the performance for rectangular matrix where M, N are large, and K is small. These shapes are widely used in ML and DL workloads.

  1. ~20% gain seen for shapes M=N=128, K=1…7.
  2. This modification has no degradation in smaller rectangular and square matrix shapes.

image
image
image
image

@martin-frbg martin-frbg added this to the 0.3.30 milestone Jan 24, 2025
@martin-frbg martin-frbg merged commit 876ba58 into OpenMathLib:develop Jan 24, 2025
83 of 85 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants