Skip to content

Conversation

YifanShenSZ
Copy link
Collaborator

@YifanShenSZ YifanShenSZ commented Sep 12, 2024

Enable all iOS 18 features by a single --coreml-ios 18 argument, rather than one arg for each individual feature

Export our current best model by

python -m examples.models.llama2.export_llama \
    -c <download-path>/consolidated.00.pth \
    -p <download-path>/params.json \
    --disable_dynamic_shape -kv \
    --coreml \
    --coreml-ios 18 \
    --coreml-quantize b4w

i.e.

  • FP16 activation
  • Static sequence length
  • In-place KV cache
  • Fused scaled dot product attention
  • 4-bit per-block weight

Copy link

pytorch-bot bot commented Sep 12, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/5319

Note: Links to docs will display an error until the docs builds have been completed.

❌ 3 New Failures

As of commit 7680294 with merge base fe53d41 (image):

NEW FAILURES - The following jobs have failed:

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Sep 12, 2024
@YifanShenSZ YifanShenSZ force-pushed the coreml-llama-ui branch 2 times, most recently from b1009d5 to 0e6e419 Compare September 12, 2024 21:46
@YifanShenSZ YifanShenSZ marked this pull request as ready for review September 12, 2024 21:46
@YifanShenSZ
Copy link
Collaborator Author

@cccclai

Copy link
Contributor

@cccclai cccclai left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you polishing the UX!

@facebook-github-bot
Copy link
Contributor

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@cccclai has imported this pull request. If you are a Meta employee, you can view this diff on Phabricator.

@facebook-github-bot
Copy link
Contributor

@cccclai merged this pull request in c080c48.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ciflow/trunk CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. Merged

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants