[Executorch][kv cache] Make quantized cache return only the updated cache portion

[kimishpatel]

Stack from ghstack (oldest at bottom):

• -> [Executorch][kv cache] Make quantized cache return only the updated cache portion #9351
• [Executorch][llama] Change runner to decouple prompt length from sequence length #9350
• [Executorch][SDPA] Fix bug in sdpa #9105

portion

Differential Revision: D69856554

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/9351

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❗ 1 Active SEVs

There are 1 currently active SEVs. If your PR is affected, please view them below:

• Long queue for ROCM runners, also B200 and XPU queueing is observed

❌ 8 New Failures, 5 Unrelated Failures

As of commit 07e9d95 with merge base 2522789 ():

NEW FAILURES - The following jobs have failed:

• Build documentation / build (buck2) / Build doc (gh)
  RuntimeError: Command docker exec -t 44a1d4a2577ac3011ce5465131ffe97c0e85faf93275fb713d252fe6cf7e336e /exec failed with exit code 2
• Check Labels / Check labels (gh)
  RuntimeError: Error checking labels: PR does not have required labels
• Lint / lintrunner / linux-job (gh)
  >>> Lint for examples/models/llama/source_transformation/quantized_kv_cache.py:
• Propose to merge ghstack orig PRs to main / Try to create a PR with ghstack /orig branch (gh)
  Process completed with exit code 1.
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04... / linux-job (gh)
  RuntimeError: Command docker exec -t ce540e85ca882a6db34b4cdc76465440e29ca6e2f098b77396fd4ec9da166c71 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+custom+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu... / linux-job (gh)
  RuntimeError: Command docker exec -t 913ad552c6e4c759712d8ef3623a81eacaa7441b5b385aa66f1707da50539647 /exec failed with exit code 1
• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.2xlarge, executorch-ubuntu-22.04-clang12) / linux-job (gh)
  RuntimeError: Command docker exec -t 94b557f8547e710b909c1c1d806674c892ce0021a6f8b0e993f694e0d8f69133 /exec failed with exit code 1
• pull / unittest-buck / macos / macos-job (gh)
  RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 3

FLAKY - The following job failed but was likely due to flakiness present on trunk:

• pull / test-llama-runner-linux (fp32, xnnpack+quantize_kv, linux.arm64.2xlarge, executorch-ubuntu-22.04-... / linux-job (gh) (matched linux rule in flaky-rules.json)
  The process '/usr/bin/git' failed with exit code 128

BROKEN TRUNK - The following jobs failed but was present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

• pull / unittest / linux / linux-job (gh) (trunk failure)
  exir/backend/test/test_delegate_map_builder.py::TestDelegateMapBuilder::test_reinsert_delegate_debug_identifier
• pull / unittest / macos / macos-job (gh) (trunk failure)
  exir/backend/test/test_delegate_map_builder.py::TestDelegateMapBuilder::test_reinsert_delegate_debug_identifier
• pull / unittest-editable / linux / linux-job (gh) (trunk failure)
  exir/backend/test/test_delegate_map_builder.py::TestDelegateMapBuilder::test_reinsert_delegate_debug_identifier
• pull / unittest-editable / macos / macos-job (gh) (trunk failure)
  exir/backend/test/test_delegate_map_builder.py::TestDelegateMapBuilder::test_reinsert_delegate_debug_identifier

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[facebook-github-bot]

This pull request was exported from Phabricator. Differential Revision: D69856554

[github-actions]

This PR needs a release notes: label

If your changes are user facing and intended to be a part of release notes, please use a label starting with release notes:.

If not, please add the topic: not user facing label.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "topic: not user facing"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[github-actions]

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.

[github-actions]

Looks like this PR hasn't been updated in a while so we're going to go ahead and mark this as Stale.
Feel free to remove the Stale label if you feel this was a mistake.
If you are unable to remove the Stale label please contact a maintainer in order to do so.
If you want the bot to never mark this PR stale again, add the no-stale label.
Stale pull requests will automatically be closed after 30 days of inactivity.