Export a lora model #11045

lucylq · 2025-05-21T17:53:52Z

Stack from ghstack (oldest at bottom):

^

Program+data combined currently, using the lora linear definition.
Issue: #9580

Differential Revision: D75153377

^ Program+data combined currently, using the lora linear definition. Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) [ghstack-poisoned]

pytorch-bot · 2025-05-21T17:53:56Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/11045

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures, 2 Unrelated Failures

As of commit bda599b with merge base 8da2ea6 ():

NEW FAILURES - The following jobs have failed:

pull / unittest / linux / linux-job (gh)
RuntimeError: Command docker exec -t 3151248983e0cc2440e2ca88788d1bce4670595988eb9a82a14fe6d8a7ecdd5b /exec failed with exit code 5
pull / unittest-editable / linux / linux-job (gh)
RuntimeError: Command docker exec -t 3860a3f52d0ab6081a62869ce603d31d524ed8b04849a2c17f6ade019ff4e65f /exec failed with exit code 5

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / unittest / macos / macos-job (gh) (trunk failure)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 5
pull / unittest-editable / macos / macos-job (gh) (trunk failure)
RuntimeError: Command bash /Users/ec2-user/runner/_work/_temp/exec_script failed with exit code 5

This comment was automatically generated by Dr. CI and updates every 15 minutes.

^ Program+data combined currently, using the lora linear definition. Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) ghstack-source-id: 285402970 Pull Request resolved: #11045

facebook-github-bot · 2025-05-21T17:54:15Z

This pull request was exported from Phabricator. Differential Revision: D75153377

digantdesai · 2025-05-22T04:47:46Z

examples/models/llama/model.py

+        adapter_checkpoint = {}
+        adapter_config = {}
+        adapter_checkpoint_path = kwargs.get("adapter_checkpoint", None)
+        if adapter_checkpoint_path:


nit: use os.exists or similar from Path?

digantdesai

tests?

^ Program+data combined currently, using the lora linear definition. Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) [ghstack-poisoned]

Pull Request resolved: #11045 ^ Program+data combined currently, using the lora linear definition. Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) ghstack-source-id: 295432560

lucylq · 2025-07-10T18:13:40Z

tests?

Let me add one. Maybe with phi-3-mini or a smaller model.

extension/llm/export/config/llm_config.py

JacobSzwejbka · 2025-07-18T17:12:22Z

extension/llm/export/config/llm_config.py

        metadata: Json string containing metadata information.
            e.g. '"{\"get_bos_id\":128000, \"get_eos_ids\":[128009, 128001]}"'
-        use_lora: Rank of the LoRA, if set to 0 then this means no LoRA. For use with QAT.
+        use_lora: Only for use with QAT. Rank of the LoRA adapter, disabled


You didnt add this, but why is this boolean named field an int, and why does it correspond with qat?

cc @cccclai ?

Didn't add it myself either, I think it's likely from Lunwen - I believe it's for the llama 3.2 1b QAT checkpoint which include LoRA, so make sure we don't break llama3.2 QAT model if we use this flag somewhere else

^ Program+data combined currently, using the lora linear definition. Issue: #9580 Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) [ghstack-poisoned]

Pull Request resolved: #11045 ^ Program+data combined currently, using the lora linear definition. Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) ghstack-source-id: 295432560

facebook-github-bot · 2025-07-22T17:51:33Z

This pull request was exported from Phabricator. Differential Revision: D75153377

^ Program+data combined currently, using the lora linear definition. Issue: #9580 Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) [ghstack-poisoned]

facebook-github-bot · 2025-07-23T23:57:36Z

This pull request was exported from Phabricator. Differential Revision: D75153377

^ Program+data combined currently, using the lora linear definition. Issue: #9580 Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) [ghstack-poisoned]

Pull Request resolved: #11045 ^ Program+data combined currently, using the lora linear definition. ghstack-source-id: 298641176 Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/)

facebook-github-bot · 2025-07-25T18:41:51Z

This pull request was exported from Phabricator. Differential Revision: D75153377

Export a lora model

aac7fd6

^ Program+data combined currently, using the lora linear definition. Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) [ghstack-poisoned]

lucylq requested review from digantdesai, jackzhxng and mcr229 as code owners May 21, 2025 17:53

lucylq mentioned this pull request May 21, 2025

Add LoRA linear definition #11044

Merged

facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label May 21, 2025

facebook-github-bot added the fb-exported label May 21, 2025

lucylq added the release notes: llm Changes to llm utilities label May 21, 2025

digantdesai reviewed May 22, 2025

View reviewed changes

Update on "Export a lora model"

c272cb3

^ Program+data combined currently, using the lora linear definition. Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) [ghstack-poisoned]

lucylq requested review from larryliu0820, mergennachin and swolchok as code owners July 10, 2025 18:09

jackzhxng approved these changes Jul 11, 2025

View reviewed changes

JacobSzwejbka reviewed Jul 18, 2025

View reviewed changes

extension/llm/export/config/llm_config.py Show resolved Hide resolved

JacobSzwejbka reviewed Jul 18, 2025

View reviewed changes

JacobSzwejbka approved these changes Jul 22, 2025

View reviewed changes

Update on "Export a lora model"

65999f0

^ Program+data combined currently, using the lora linear definition. Issue: #9580 Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) [ghstack-poisoned]

lucylq mentioned this pull request Jul 22, 2025

CI test for lora #12724

Merged

jackzhxng approved these changes Jul 23, 2025

View reviewed changes

Update on "Export a lora model"

3887cad

^ Program+data combined currently, using the lora linear definition. Issue: #9580 Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) [ghstack-poisoned]

lucylq added 6 commits July 24, 2025 12:37

Update on "Export a lora model"

d5e4d36

^ Program+data combined currently, using the lora linear definition. Issue: #9580 Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) [ghstack-poisoned]

Update on "Export a lora model"

deaa4b6

^ Program+data combined currently, using the lora linear definition. Issue: #9580 Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) [ghstack-poisoned]

Update on "Export a lora model"

751286a

^ Program+data combined currently, using the lora linear definition. Issue: #9580 Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) [ghstack-poisoned]

Update on "Export a lora model"

6392f0b

^ Program+data combined currently, using the lora linear definition. Issue: #9580 Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) [ghstack-poisoned]

Update on "Export a lora model"

fc15be6

^ Program+data combined currently, using the lora linear definition. Issue: #9580 Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) [ghstack-poisoned]

Update on "Export a lora model"

bda599b

^ Program+data combined currently, using the lora linear definition. Issue: #9580 Differential Revision: [D75153377](https://our.internmc.facebook.com/intern/diff/D75153377/) [ghstack-poisoned]

facebook-github-bot merged commit 9558550 into gh/lucylq/83/base Jul 26, 2025
95 of 101 checks passed

facebook-github-bot deleted the gh/lucylq/83/head branch July 26, 2025 01:01

facebook-github-bot temporarily deployed to cherry-pick-bot July 26, 2025 01:01 — with GitHub Actions Inactive

pytorchbot mentioned this pull request Jul 26, 2025

Export a lora model #12875

Merged

Export a lora model #11045

Export a lora model #11045

Uh oh!

Conversation

lucylq commented May 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented May 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/11045

❌ 2 New Failures, 2 Unrelated Failures

Uh oh!

facebook-github-bot commented May 21, 2025

Uh oh!

digantdesai May 22, 2025

Choose a reason for hiding this comment

Uh oh!

digantdesai left a comment

Choose a reason for hiding this comment

Uh oh!

lucylq commented Jul 10, 2025

Uh oh!

Uh oh!

JacobSzwejbka Jul 18, 2025

Choose a reason for hiding this comment

Uh oh!

lucylq Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

cccclai Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

cccclai Jul 22, 2025

Choose a reason for hiding this comment

Uh oh!

facebook-github-bot commented Jul 22, 2025

Uh oh!

facebook-github-bot commented Jul 23, 2025

Uh oh!

facebook-github-bot commented Jul 25, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

lucylq commented May 21, 2025 •

edited

Loading

pytorch-bot bot commented May 21, 2025 •

edited

Loading