GEMM+GEMM migraphx integration #1791

dhernandez0 · 2025-03-24T16:10:45Z

This PR depends on #1774, that PR needs to be merged first.

In this PR we add migraphx integration and end-to-end tests in migraphx dialect. The end-to-end tests are based on attention tests.

TODO:

Merge Add Gemm+Elementwise+Gemm support #1774 first

codecov · 2025-03-26T14:20:30Z

Codecov Report

Attention: Patch coverage is 86.94030% with 35 lines in your changes missing coverage. Please review.

Project coverage is 78.49%. Comparing base (1fb495e) to head (ad9a71b).
Report is 6 commits behind head on develop.

Files with missing lines	Patch %	Lines
...ect/Rock/Transforms/SortDimensionsMemoryLayout.cpp	73.03%	12 Missing and 12 partials ⚠️
mlir/lib/Conversion/TosaToRock/TosaToRock.cpp	94.70%	0 Missing and 9 partials ⚠️
mlir/lib/Conversion/TosaToRock/TosaToRockPass.cpp	60.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@             Coverage Diff             @@
##           develop    #1791      +/-   ##
===========================================
- Coverage    78.60%   78.49%   -0.11%     
===========================================
  Files           99      100       +1     
  Lines        29389    29885     +496     
  Branches      4379     4451      +72     
===========================================
+ Hits         23100    23458     +358     
- Misses        4492     4591      +99     
- Partials      1797     1836      +39

Flag	Coverage Δ
mfma	`78.49% <86.94%> (-0.11%)`	⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

umangyadav · 2025-03-28T19:27:03Z

mlir/test/fusion/pr-e2e/gemm-gemm/mixr-gemm-gemm.mlir

+// ALLOW_RETRIES: 2
+// CHECK: [1 1 1]
+module {
+  func.func private @mlir_gemm_gemm(%arg0: !migraphx.shaped<1x64x64xf32, 4096x64x1> {mhal.read_access}, %arg1: !migraphx.shaped<1x64x64xf32, 4096x64x1> {mhal.read_access}, %arg2: !migraphx.shaped<1x64x64xf32, 4096x64x1> {mhal.read_access}) -> (!migraphx.shaped<1x64x64xf32, 4096x64x1> {mhal.write_access}) {


For the Fp32 i think if you just use --clone-harness it should work without running into any accuracy issues.

yes, the reason it's like this is because I copied the tests from mixr-attention.mlir. I'll change it.

umangyadav · 2025-03-28T19:27:39Z

mlir/test/fusion/pr-e2e/gemm-gemm/mixr-gemm-gemm-where.mlir

@@ -0,0 +1,14 @@
+// RUN: rocmlir-gen -fut mlir_gemm_gemm_where --arch %arch --clone-harness %s | rocmlir-driver -kernel-pipeline=migraphx | rocmlir-driver -host-pipeline=migraphx,highlevel | rocmlir-gen -ph -rand 1 -rand_type float -fut mlir_gemm_gemm_where_wrapper --verifier clone - | rocmlir-driver -host-pipeline mhal -kernel-pipeline full | xmir-runner --shared-libs=%linalg_test_lib_dir/libmlir_rocm_runtime%shlibext,%conv_validation_wrapper_library_dir/libconv-validation-wrappers%shlibext,%linalg_test_lib_dir/libmlir_runner_utils%shlibext,%linalg_test_lib_dir/libmlir_float16_utils%shlibext,%linalg_test_lib_dir/libmlir_c_runner_utils%shlibext,%linalg_test_lib_dir/libmlir_async_runtime%shlibext --entry-point-result=void | FileCheck %s


nit: it is More like gemm_where_gemm and not gemm_gemm_where

umangyadav · 2025-03-28T19:28:07Z

mlir/test/fusion/pr-e2e/gemm-gemm/mixr-gemm-gemm-scale-reshape-4d.mlir

+// ALLOW_RETRIES: 2
+// CHECK: [1 1 1]
+module {
+  func.func private @mlir_gemm_gemm(%arg0: !migraphx.shaped<1x4x32x32xf32, 4096x1024x32x1> {mhal.read_access}, 


I don't see why you need to attach mhal.read_access etc. if you are using clone-harness

umangyadav · 2025-03-28T19:28:36Z

mlir/test/fusion/pr-e2e/gemm-gemm/mixr-gemm-gemm-padded.mlir

+// ALLOW_RETRIES: 2
+// CHECK: [1 1 1]
+module {
+  func.func private @mlir_gemm_gemm(%arg0: !migraphx.shaped<1x7x3xf32, 21x3x1> {mhal.read_access}, %arg1: !migraphx.shaped<1x3x7xf32, 21x7x1> {mhal.read_access}, %arg2: !migraphx.shaped<1x7x3xf32, 21x3x1> {mhal.read_access}) -> (!migraphx.shaped<1x7x3xf32, 21x3x1> {mhal.write_access}) {


Why are you creating wrapper spearately and not using --clone-harness ?

same here, just copied some existing attention tests and adapted to gemm+gemm. I'll change it.

umangyadav · 2025-04-08T12:01:51Z

mlir/lib/Dialect/Rock/Transforms/SortDimensionsMemoryLayout.cpp

  }
 };

+struct GemmElementwiseGemmRewritePattern


Is there opportunity to use GemmGemmWrapperInterface here ?

yes, I was about to push some changes re this

umangyadav · 2025-04-08T12:02:40Z

mlir/test/fusion/mixr-gemm-gemm-problem-key.mlir

+  func.func private @mlir_gemm_gemm(%arg0: !migraphx.shaped<1x7x3xf32, 21x3x1> {mhal.read_access},
+                                    %arg1: !migraphx.shaped<1x3x7xf32, 21x7x1> {mhal.read_access},
+                                    %arg2: !migraphx.shaped<1x7x3xf32, 21x3x1> {mhal.read_access},
+                                    %arg3: !migraphx.shaped<1x7x7xf32, 49x7x1> {mhal.read_access}) 


Do you need mhal stuff here ?

dhernandez0 self-assigned this Mar 24, 2025

dhernandez0 requested a review from causten as a code owner March 24, 2025 16:10

dhernandez0 changed the base branch from develop to 1704-gemm-elementwise-gemm March 24, 2025 16:11

dhernandez0 requested review from djramic, dorde-antic, mirza-halilcevic, stefankoncarevic and umangyadav March 24, 2025 16:15

dhernandez0 force-pushed the 1704-gemm-elementwise-gemm branch 2 times, most recently from bd2c24e to 54237b0 Compare March 26, 2025 07:53

dhernandez0 force-pushed the gemm_gemm_migraphx_integration branch 2 times, most recently from 2b1e306 to 5fb22e3 Compare March 26, 2025 08:28

dhernandez0 mentioned this pull request Mar 28, 2025

gemm+gemm: extend allowed types #1795

Merged

2 tasks

umangyadav reviewed Mar 28, 2025

View reviewed changes

dhernandez0 force-pushed the 1704-gemm-elementwise-gemm branch 2 times, most recently from 3f02a22 to d7ad776 Compare April 8, 2025 10:09

dhernandez0 force-pushed the gemm_gemm_migraphx_integration branch from 5fb22e3 to 0fe28f6 Compare April 8, 2025 10:11

dhernandez0 force-pushed the 1704-gemm-elementwise-gemm branch from 4821b84 to 8d27a5c Compare April 8, 2025 10:41

dhernandez0 force-pushed the gemm_gemm_migraphx_integration branch from 0fe28f6 to ef9080c Compare April 8, 2025 10:42

umangyadav reviewed Apr 8, 2025

View reviewed changes

umangyadav approved these changes Apr 8, 2025

View reviewed changes

Base automatically changed from 1704-gemm-elementwise-gemm to develop April 8, 2025 13:45

dhernandez0 force-pushed the gemm_gemm_migraphx_integration branch from d3372a5 to b04068b Compare April 8, 2025 14:10

GEMM elementwise GEMM migraphx integration

ad9a71b

dhernandez0 force-pushed the gemm_gemm_migraphx_integration branch from b04068b to ad9a71b Compare April 8, 2025 14:12

dhernandez0 merged commit 2079c51 into develop Apr 9, 2025
11 of 25 checks passed

dhernandez0 deleted the gemm_gemm_migraphx_integration branch April 9, 2025 15:08

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

GEMM+GEMM migraphx integration #1791

GEMM+GEMM migraphx integration #1791

Uh oh!

dhernandez0 commented Mar 24, 2025 •

edited

Loading

Uh oh!

codecov bot commented Mar 26, 2025 •

edited

Loading

Uh oh!

umangyadav Mar 28, 2025

Uh oh!

dhernandez0 Apr 8, 2025

Uh oh!

umangyadav Mar 28, 2025

Uh oh!

umangyadav Mar 28, 2025

Uh oh!

umangyadav Mar 28, 2025

Uh oh!

dhernandez0 Apr 8, 2025

Uh oh!

umangyadav Apr 8, 2025

Uh oh!

dhernandez0 Apr 8, 2025

Uh oh!

umangyadav Apr 8, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

		@@ -0,0 +1,14 @@
		// RUN: rocmlir-gen -fut mlir_gemm_gemm_where --arch %arch --clone-harness %s \| rocmlir-driver -kernel-pipeline=migraphx \| rocmlir-driver -host-pipeline=migraphx,highlevel \| rocmlir-gen -ph -rand 1 -rand_type float -fut mlir_gemm_gemm_where_wrapper --verifier clone - \| rocmlir-driver -host-pipeline mhal -kernel-pipeline full \| xmir-runner --shared-libs=%linalg_test_lib_dir/libmlir_rocm_runtime%shlibext,%conv_validation_wrapper_library_dir/libconv-validation-wrappers%shlibext,%linalg_test_lib_dir/libmlir_runner_utils%shlibext,%linalg_test_lib_dir/libmlir_float16_utils%shlibext,%linalg_test_lib_dir/libmlir_c_runner_utils%shlibext,%linalg_test_lib_dir/libmlir_async_runtime%shlibext --entry-point-result=void \| FileCheck %s

GEMM+GEMM migraphx integration #1791

GEMM+GEMM migraphx integration #1791

Uh oh!

Conversation

dhernandez0 commented Mar 24, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

codecov bot commented Mar 26, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

dhernandez0 commented Mar 24, 2025 •

edited

Loading

codecov bot commented Mar 26, 2025 •

edited

Loading