Source transform custom SDPA for TorchTune

[jackzhxng]

Summary

[PLEASE REMOVE] See CONTRIBUTING.md's Pull Requests for ExecuTorch PR guidelines.

[PLEASE REMOVE] If this PR closes an issue, please add a Fixes #<issue-id> line.

[PLEASE REMOVE] If this PR introduces a fix or feature that should be the upcoming release notes, please add a "Release notes: " label. For a list of available release notes labels, check out CONTRIBUTING.md's Pull Requests.

Test plan

[PLEASE REMOVE] How did you test this PR? Please write down any manual commands you used and note down tests that you have written if applicable.

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6617

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 18 New Failures

As of commit 070ddaf with merge base 8f9fb7e ():

NEW FAILURES - The following jobs have failed:

• Lint / lintrunner / linux-job (gh)
  >>> Lint for examples/models/llama3_2_vision/model.py:
• pull / test-eval_llama-mmlu-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 5b581f0b554b4accb682ebf24543c12eba16ca8c6f9056975e1a81e60f71b07c /exec failed with exit code 1
• pull / test-eval_llama-wikitext-linux / linux-job (gh)
  RuntimeError: Command docker exec -t d3a8214ce829585e5617804590df6c2d5170088032621d85d0f804d2f81ec9df /exec failed with exit code 1
• pull / test-llama_runner_eager-linux / linux-job (gh)
  RuntimeError: Command docker exec -t c604472d297bf420449ca9ce9213b99f913b9e34db1057051ebf680c3b3b42a4 /exec failed with exit code 1
• pull / test-llama-runner-linux (bf16, buck2, custom) / linux-job (gh)
  RuntimeError: Command docker exec -t fe6061281c93e8bbfcf8a69086c0db0b7adea958505b08198391b4ba9008c13c /exec failed with exit code 1
• pull / test-llama-runner-linux (bf16, buck2, portable) / linux-job (gh)
  RuntimeError: Command docker exec -t 200b9d2d1ac397b7fd7b188032fb86dbe354883ab84208ae59afd086d278e9f4 /exec failed with exit code 1
• pull / test-llama-runner-linux (bf16, cmake, custom) / linux-job (gh)
  RuntimeError: Command docker exec -t f853a5bba5cc781ca073f35f10d6ad020158a92c6d7997cbfb3ca88e77ebb4be /exec failed with exit code 1
• pull / test-llama-runner-linux (bf16, cmake, portable) / linux-job (gh)
  RuntimeError: Command docker exec -t 0c415f1312989f9da85efde9171327b3dbeabf70a5419dda238311e57a19ee37 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, buck2, portable) / linux-job (gh)
  RuntimeError: Command docker exec -t 028ce16fe13c39f2969590bfcd9211a1d018c31f0f0ca78cbc1a67331e9484dd /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, buck2, xnnpack+custom) / linux-job (gh)
  RuntimeError: Command docker exec -t fe9731a380f37454067e54f708a640ecc3bda012cc314147a44fc1d4c98c5b07 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, buck2, xnnpack+custom+qe) / linux-job (gh)
  RuntimeError: Command docker exec -t 37dcee81a2a1765e2ad6ba8e95e7fc52197f26f8c25fc9fe2530dee9f0185d80 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, cmake, portable) / linux-job (gh)
  RuntimeError: Command docker exec -t 3ddc19f232f2e64f68f18e4b7731ede6e3309a6973789d21c506554d81ffc80b /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, cmake, xnnpack+custom) / linux-job (gh)
  RuntimeError: Command docker exec -t 980b9b7fc0b28f51443533ead62189259d1327a4eeb45d93e2013a8ec509148d /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, cmake, xnnpack+custom+qe) / linux-job (gh)
  RuntimeError: Command docker exec -t 9992f0d5ab50847b433af6878082f332b4e53ae36bf3b163baa0f96c3fb32ac2 /exec failed with exit code 1
• pull / test-llama-runner-qnn-linux (fp32, cmake, qnn) / linux-job (gh)
  RuntimeError: Command docker exec -t 4830838ab182ae172b4df092a517e191328ef60da6d4bf6644163b7b592d1a63 /exec failed with exit code 1
• pull / test-llava-runner-linux / linux-job (gh)
  RuntimeError: Command docker exec -t 27f394428f33bfa8b4c42905d113daa186bc660227592971cdb73080c0e3f327 /exec failed with exit code 1
• pull / unittest / linux / linux-job (gh)
  RuntimeError: Command docker exec -t 1a2547dbd71f9ab37b7fa59db839ec059cd579ab8db2c1a3e911de39b8e31193 /exec failed with exit code 1
• pull / unittest / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.