-
Notifications
You must be signed in to change notification settings - Fork 4
Closed
Description
While testing #92 I came across these aten ops which have errors. I will just stick the logs here. This is running opinfo against aten and torchbench --topn 3 against aten
Ops are:
empty_like.default -> we should actually remove testing for this one
new_empty.default -> we should actually remove testing for this one
bernoulli.default -> We either need a custom function or remove correctness here as well
new_empty_strided -> remove this one as well
opinfo against aten
[2025-08-21 11:45:56][WARNING][eval.py]
Exception raised for empty_like.default:
args: ((T([], bf16),), {})
exc: Scalars are not close!
Expected 2.369355800876173e-38 but got -6.71875.
Absolute difference: 6.71875 (up to 0.01 allowed)
Relative difference: 2.8356863910078207e+38 (up to 0.01 allowed)
[2025-08-21 11:45:56][WARNING][eval.py]
Exception raised for new_empty.default:
args: ((T([5], bf16), (10,),), {'dtype': torch.float64})
exc: Tensor-likes are not close!
Mismatched elements: 9 / 10 (90.0%)
Greatest absolute difference: 223118.2170910577 at index (3,) (up to 0.01 allowed)
Greatest relative difference: 59994.43869459766 at index (4,) (up to 0.01 allowed)
[2025-08-21 11:45:56][WARNING][eval.py]
Exception raised for new_empty.default:
args: ((T([5], bf16), (2, 2, 2),), {'dtype': torch.float64, 'device': 'cpu'})
exc: Tensor-likes are not close!
Mismatched elements: 1 / 8 (12.5%)
Greatest absolute difference: nan at index (1, 1, 0) (up to 0.01 allowed)
Greatest relative difference: nan at index (1, 1, 0) (up to 0.01 allowed)
[2025-08-21 11:45:56][WARNING][eval.py]
Exception raised for new_empty.default:
args: ((T([5], bf16), (7, 2),), {'device': 'cuda'})
exc: Tensor-likes are not close!
Mismatched elements: 13 / 14 (92.9%)
Greatest absolute difference: 1.05403626228257e+36 at index (6, 1) (up to 0.01 allowed)
Greatest relative difference: 2.03797651697492e+35 at index (6, 1) (up to 0.01 allowed)
[2025-08-21 11:45:56][WARNING][eval.py]
Exception raised for new_empty_strided.default:
args: ((T([5], bf16), (2, 2, 2), (20, 10, 5),), {'dtype': torch.float64, 'device': 'cpu'})
exc: Tensor-likes are not close!
Mismatched elements: 1 / 8 (12.5%)
Greatest absolute difference: nan at index (0, 0, 0) (up to 0.01 allowed)
Greatest relative difference: nan at index (0, 0, 0) (up to 0.01 allowed)
[2025-08-21 11:45:56][WARNING][eval.py]
Exception raised for bernoulli.default:
args: ((T([], bf16),), {})
exc: Scalars are not close!
Expected 1.0 but got 0.0.
Absolute difference: 1.0 (up to 0.01 allowed)
Relative difference: 1.0 (up to 0.01 allowed)
torchbench --topn 3 against aten
Exception raised for new_empty.default:
args: ((T([64, 3136, 128], f16), [64, 1, 1],), {'dtype': torch.float16, 'layout': torch.strided, 'device': 'cuda', 'pin_memory': False})
exc: Tensor-likes are not close!
Mismatched elements: 56 / 64 (87.5%)
Greatest absolute difference: 44224.0 at index (58, 0, 0) (up to 0.01 allowed)
Greatest relative difference: inf at index (1, 0, 0) (up to 0.01 allowed)
[2025-08-21 11:47:19][WARNING][eval.py]
Exception raised for new_empty.default:
args: ((T([64, 16, 196, 128], f16), [64, 1, 1, 1],), {'dtype': torch.float16, 'layout': torch.strided, 'device': 'cuda', 'pin_memory': False})
exc: Tensor-likes are not close!
Mismatched elements: 16 / 64 (25.0%)
Greatest absolute difference: inf at index (58, 0, 0, 0) (up to 0.01 allowed)
Greatest relative difference: inf at index (58, 0, 0, 0) (up to 0.01 allowed)
[2025-08-21 11:47:19][WARNING][eval.py]
Exception raised for new_empty_strided.default:
args: ((T([128, 16, 112, 112], f16), [128, 16, 112, 112], [200704, 12544, 112, 1],), {})
exc: Tensor-likes are not close!
Mismatched elements: 5140280 / 25690112 (20.0%)
Greatest absolute difference: 0.99951171875 at index (0, 0, 17, 3) (up to 0.01 allowed)
Greatest relative difference: inf at index (0, 0, 0, 0) (up to 0.01 allowed)
[2025-08-21 11:47:19][WARNING][eval.py]
Exception raised for new_empty_strided.default:
args: ((T([1, 1024, 128, 128], f16), [1, 1024, 128, 128], [16777216, 16384, 128, 1],), {})
exc: Tensor-likes are not close!
Mismatched elements: 5140280 / 16777216 (30.6%)
Greatest absolute difference: 0.99951171875 at index (0, 544, 14, 115) (up to 0.01 allowed)
Greatest relative difference: inf at index (0, 544, 0, 0) (up to 0.01 allowed)
[2025-08-21 11:47:19][WARNING][eval.py]
Exception raised for new_empty_strided.default:
args: ((T([24, 4, 256, 513], f16), [24, 4, 256, 513], [525312, 131328, 513, 1],), {})
exc: Tensor-likes are not close!
Mismatched elements: 5139725 / 12607488 (40.8%)
Greatest absolute difference: 0.99951171875 at index (0, 3, 161, 466) (up to 0.01 allowed)
Greatest relative difference: 1.0 at index (0, 3, 158, 98) (up to 0.01 allowed)
Metadata
Metadata
Assignees
Labels
No labels