Skip to content

Conversation

@DN6
Copy link
Collaborator

@DN6 DN6 commented Feb 26, 2025

What does this PR do?

Fast GPU on PR model tests are failing because of the size of the models (too small) and default split percents.

  • Update the model init params and split percents so tests pass.
  • Update disk offload test to try and force disk offloading even if the model is small.

Fixes # (issue)

Before submitting

Who can review?

Anyone in the community is free to review the PR once the tests have passed. Feel free to tag
members/contributors who may be interested in your PR.

@DN6 DN6 requested a review from sayakpaul February 26, 2025 17:34
@DN6 DN6 changed the title Fast gpu tests [CI] Fix Fast GPU tests on PR Feb 26, 2025
"num_key_value_heads": 4,
"intermediate_size": 32,
"num_layers": 1,
"num_layers": 20,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Isn’t it a bit too much for CPUs?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall model size is still very small (2.6 MB). Adds 2 seconds to the current CPU tests.

Copy link
Member

@sayakpaul sayakpaul left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Think the CI should be trigged for changes to tests, too:

@DN6 DN6 merged commit e5c43b8 into main Feb 27, 2025
23 of 26 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants