Skip to content

Mark test_layerwise_upcasting as is_flaky #11282

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from
Closed

Mark test_layerwise_upcasting as is_flaky #11282

Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion tests/models/test_modeling_common.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,7 @@
backend_empty_cache,
floats_tensor,
get_python_version,
is_flaky,
is_torch_compile,
numpy_cosine_similarity_distance,
require_peft_backend,
Expand Down Expand Up @@ -1436,6 +1437,7 @@ def test_fn(storage_dtype, compute_dtype):
test_fn(torch.float8_e5m2, torch.float32)
test_fn(torch.float8_e4m3fn, torch.bfloat16)

@is_flaky
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If it's only for test_models_autoencoder_hunyuan_video.py, I would consider subclassing it inside AutoencoderKLHunyuanVideoTests and decorating it with is_flaky.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These tests haven't been running on CI for a while due to the unclip issue so it is potentially flaky for other models too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would prefer confirming that first before marking all tests as flaky when they appear to be not.

def test_layerwise_casting_inference(self):
from diffusers.hooks.layerwise_casting import DEFAULT_SKIP_MODULES_PATTERN, SUPPORTED_PYTORCH_LAYERS

Expand Down Expand Up @@ -1473,7 +1475,8 @@ def test_layerwise_casting(storage_dtype, compute_dtype):

# The precision test is not very important for fast tests. In most cases, the outputs will not be the same.
# We just want to make sure that the layerwise casting is working as expected.
self.assertTrue(numpy_cosine_similarity_distance(base_slice, output) < 1.0)
diff = numpy_cosine_similarity_distance(base_slice, output)
self.assertTrue(diff < 1.0, f"Expected {diff=} < 1.0.")

test_layerwise_casting(torch.float16, torch.float32)
test_layerwise_casting(torch.float8_e4m3fn, torch.float32)
Expand Down
Loading