Skip to content

Conversation

@kirklandsign
Copy link
Contributor

Summary

Some model E2E are super slow (get idea from https://hud.pytorch.org/pytorch/executorch/commit/6d6630edae0bdc37c8450e55f2fc0ef90cb5f50c). Remove DL3 and EDSR for now.

Test plan

Reduced CPU usage

@pytorch-bot
Copy link

pytorch-bot bot commented Nov 8, 2024

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/6742

Note: Links to docs will display an error until the docs builds have been completed.

✅ No Failures

As of commit 0493d9f with merge base ddc8ea6 (image):
💚 Looks good so far! There are no failures yet. 💚

This comment was automatically generated by Dr. CI and updates every 15 minutes.

@facebook-github-bot facebook-github-bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Nov 8, 2024
@guangy10
Copy link
Contributor

@kirklandsign On what backend are these models running slow? Is it on xnnpack? I observed extreme slowness today when running previously supported HF models on xnnpack. And it produces garbage outputs (see the CI job on your PR: https://github.com/pytorch/executorch/actions/runs/11750351996/job/32738448869?pr=6742#step:14:10567). I'm wondering the slowness you are seeing is same. If yes, we should not remove/disable the CI/tests.

@kirklandsign
Copy link
Contributor Author

@kirklandsign On what backend are these models running slow? Is it on xnnpack? I observed extreme slowness today when running previously supported HF models on xnnpack. And it produces garbage outputs (see the CI job on your PR: https://github.com/pytorch/executorch/actions/runs/11750351996/job/32738448869?pr=6742#step:14:10567). I'm wondering the slowness you are seeing is same. If yes, we should not remove/disable the CI/tests.

These are for XNNPACK and portable. EDSR is very slow. Do you think we want to remove HF models on XNNPACK as well?

@kirklandsign kirklandsign merged commit 10a814e into main Nov 12, 2024
66 checks passed
@kirklandsign kirklandsign deleted the slow-model-ci branch November 12, 2024 18:48
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants