Add support for bias in optimized op_linear.cpp. #11210

hsharma35 · 2025-05-29T05:04:44Z

Summary: Diff uses op_add_sub_impl to add bias after optimized gemm call.

Differential Revision: D75491158

pytorch-bot · 2025-05-29T05:04:48Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/11210

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit c6d1d2a with merge base 6875c8e ():
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

facebook-github-bot · 2025-05-29T05:04:56Z

This pull request was exported from Phabricator. Differential Revision: D75491158

Summary: Diff uses `op_add_sub_impl` to add bias after optimized gemm call. Reviewed By: zonglinpeng Differential Revision: D75491158

facebook-github-bot · 2025-05-29T17:33:04Z

This pull request was exported from Phabricator. Differential Revision: D75491158

kernels/optimized/cpu/op_linear.cpp

shim_et/xplat/executorch/kernels/optimized/op_registration_util.bzl

kernels/optimized/cpu/op_linear.cpp

Summary: Diff uses `op_add_sub_impl` to add bias after optimized gemm call. Reviewed By: zonglinpeng Differential Revision: D75491158

facebook-github-bot · 2025-05-30T00:06:03Z

This pull request was exported from Phabricator. Differential Revision: D75491158

facebook-github-bot · 2025-05-30T00:09:58Z

This pull request was exported from Phabricator. Differential Revision: D75491158

Summary: Pull Request resolved: pytorch#11210 Diff uses `op_add_sub_impl` to add bias after optimized gemm call. Reviewed By: zonglinpeng Differential Revision: D75491158

Summary: Diff uses `op_add_sub_impl` to add bias after optimized gemm call. Reviewed By: zonglinpeng Differential Revision: D75491158

facebook-github-bot · 2025-05-30T04:17:27Z

This pull request was exported from Phabricator. Differential Revision: D75491158

Summary: Diff uses `op_add_sub_impl` to add bias after optimized gemm call. Reviewed By: zonglinpeng Differential Revision: D75491158

facebook-github-bot · 2025-05-30T04:19:38Z

This pull request was exported from Phabricator. Differential Revision: D75491158

Summary: Diff uses `op_add_sub_impl` to add bias after optimized gemm call. Reviewed By: zonglinpeng Differential Revision: D75491158

Summary: Diff initializes the output tensor before calling gemm with beta=1 when bias is non-nullopt. Reviewed By: larryliu0820, zonglinpeng Differential Revision: D75491158

Summary: Pull Request resolved: pytorch#11210 Diff initializes the output tensor before calling gemm with beta=1 when bias is non-nullopt. Reviewed By: larryliu0820, zonglinpeng Differential Revision: D75491158

facebook-github-bot · 2025-05-30T20:05:10Z

This pull request was exported from Phabricator. Differential Revision: D75491158

kimishpatel

I want to block this for now in favor of landing the other fix for bias. mm.out -> linear.out is the right change though

kernels/test/op_linear_test.cpp

kimishpatel

ok so this diff does incorporate "idea" from the other diff so seems ok to me. but would like the test case to be updated

Summary: Diff initializes the output tensor before calling gemm with beta=1 when bias is non-nullopt. Reviewed By: larryliu0820, zonglinpeng Differential Revision: D75491158

facebook-github-bot · 2025-05-31T01:11:47Z

This pull request was exported from Phabricator. Differential Revision: D75491158

Summary: Pull Request resolved: pytorch#11210 Diff initializes the output tensor before calling gemm with beta=1 when bias is non-nullopt. Reviewed By: larryliu0820, zonglinpeng Differential Revision: D75491158

Summary: Diff initializes the output tensor before calling gemm with beta=1 when bias is non-nullopt. Reviewed By: larryliu0820, zonglinpeng Differential Revision: D75491158

facebook-github-bot · 2025-05-31T03:15:26Z

This pull request was exported from Phabricator. Differential Revision: D75491158

Summary: Diff initializes the output tensor before calling gemm with beta=1 when bias is non-nullopt. Reviewed By: larryliu0820, zonglinpeng Differential Revision: D75491158

facebook-github-bot · 2025-05-31T03:43:53Z

This pull request was exported from Phabricator. Differential Revision: D75491158

Summary: Diff initializes the output tensor before calling gemm with beta=1 when bias is non-nullopt. Reviewed By: larryliu0820, zonglinpeng Differential Revision: D75491158

facebook-github-bot · 2025-05-31T03:47:51Z

This pull request was exported from Phabricator. Differential Revision: D75491158

kimishpatel · 2025-05-31T16:37:28Z

kernels/optimized/cpu/op_linear.cpp

+  // Output is a n x m x scalar_t, while bias is m x scalar_t.
+  const size_t row_size = static_cast<size_t>(m) * sizeof(scalar_t);
+  for (const auto col : c10::irange(n)) {
+    std::memcpy(


To handle 2d bias, you need to fix this. Bias pointer is not advancing

hmm, it's more complicated than that. bias can be 1xm, nx1, or nxm.

kimishpatel

I am accepting this but do note that you want either some random values or "not-all-outputs-are-same" values. The test as is, is quite weak IMO.

Summary: Diff initializes the output tensor before calling gemm with beta=1 when bias is non-nullopt. Reviewed By: larryliu0820, zonglinpeng, kimishpatel Differential Revision: D75491158

facebook-github-bot · 2025-05-31T17:00:47Z

This pull request was exported from Phabricator. Differential Revision: D75491158

hsharma35 requested review from manuelcandales and swolchok as code owners May 29, 2025 05:04

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 29, 2025

facebook-github-bot added the fb-exported label May 29, 2025

hsharma35 added the release notes: none Do not include this in the release notes label May 29, 2025

zonglinpeng approved these changes May 29, 2025

View reviewed changes

hsharma35 force-pushed the export-D75491158 branch from 6739c91 to 7205931 Compare May 29, 2025 17:32

swolchok reviewed May 29, 2025

View reviewed changes

kernels/optimized/cpu/op_linear.cpp Outdated Show resolved Hide resolved

shim_et/xplat/executorch/kernels/optimized/op_registration_util.bzl Outdated Show resolved Hide resolved

kernels/optimized/cpu/op_linear.cpp Show resolved Hide resolved

hsharma35 requested review from digantdesai and kimishpatel May 29, 2025 20:10

hsharma35 force-pushed the export-D75491158 branch from 7205931 to c7022c9 Compare May 30, 2025 00:05

hsharma35 force-pushed the export-D75491158 branch from c7022c9 to 306ee28 Compare May 30, 2025 00:05

hsharma35 force-pushed the export-D75491158 branch from 306ee28 to d625780 Compare May 30, 2025 00:10

hsharma35 force-pushed the export-D75491158 branch from d625780 to 2b84698 Compare May 30, 2025 04:17

hsharma35 force-pushed the export-D75491158 branch from 2b84698 to 70c406c Compare May 30, 2025 04:19

hsharma35 force-pushed the export-D75491158 branch from 70c406c to ab44a56 Compare May 30, 2025 05:37

hsharma35 force-pushed the export-D75491158 branch from 1eacff3 to 7f7ff17 Compare May 30, 2025 20:00

hsharma35 force-pushed the export-D75491158 branch from 7f7ff17 to 0b33f00 Compare May 30, 2025 20:01

hsharma35 force-pushed the export-D75491158 branch from 0b33f00 to 9205582 Compare May 30, 2025 20:05

kimishpatel requested changes May 30, 2025

View reviewed changes

kimishpatel reviewed May 30, 2025

View reviewed changes

kernels/test/op_linear_test.cpp Outdated Show resolved Hide resolved

kimishpatel requested changes May 30, 2025

View reviewed changes

hsharma35 force-pushed the export-D75491158 branch from 9205582 to b14f114 Compare May 31, 2025 00:56

hsharma35 force-pushed the export-D75491158 branch from b14f114 to b38dfa3 Compare May 31, 2025 01:11

hsharma35 force-pushed the export-D75491158 branch from b38dfa3 to 499fb97 Compare May 31, 2025 03:14

hsharma35 force-pushed the export-D75491158 branch from 499fb97 to 7f303be Compare May 31, 2025 03:43

hsharma35 force-pushed the export-D75491158 branch from 7f303be to 49b3b4a Compare May 31, 2025 03:47

hsharma35 requested a review from kimishpatel May 31, 2025 15:48

kimishpatel reviewed May 31, 2025

View reviewed changes

kimishpatel approved these changes May 31, 2025

View reviewed changes

Add support for bias in optimized op_linear.cpp. (pytorch#11210)

c6d1d2a

Summary: Diff initializes the output tensor before calling gemm with beta=1 when bias is non-nullopt. Reviewed By: larryliu0820, zonglinpeng, kimishpatel Differential Revision: D75491158

hsharma35 force-pushed the export-D75491158 branch from 49b3b4a to c6d1d2a Compare May 31, 2025 17:00

facebook-github-bot merged commit 95a1db5 into pytorch:main May 31, 2025
95 of 98 checks passed

Add support for bias in optimized op_linear.cpp. #11210

Add support for bias in optimized op_linear.cpp. #11210

Uh oh!

Conversation

hsharma35 commented May 29, 2025

Uh oh!

pytorch-bot bot commented May 29, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/11210

✅ No Failures

Uh oh!

facebook-github-bot commented May 29, 2025

Uh oh!

facebook-github-bot commented May 29, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

facebook-github-bot commented May 30, 2025

Uh oh!

facebook-github-bot commented May 30, 2025

Uh oh!

facebook-github-bot commented May 30, 2025

Uh oh!

facebook-github-bot commented May 30, 2025

Uh oh!

facebook-github-bot commented May 30, 2025

Uh oh!

kimishpatel left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

kimishpatel left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented May 31, 2025

Uh oh!

facebook-github-bot commented May 31, 2025

Uh oh!

facebook-github-bot commented May 31, 2025

Uh oh!

facebook-github-bot commented May 31, 2025

Uh oh!

kimishpatel May 31, 2025

Choose a reason for hiding this comment

Uh oh!

hsharma35 Jun 1, 2025

Choose a reason for hiding this comment

Uh oh!

kimishpatel left a comment

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented May 31, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

pytorch-bot bot commented May 29, 2025 •

edited

Loading