Skip to content

Commit 349cad1

Browse files
authored
Update semi_structured_sparse.py
On my environment with an A100, I only see a speedup when enabling the following line SparseSemiStructuredTensor._FORCE_CUTLASS = True
1 parent 3469d47 commit 349cad1

File tree

1 file changed

+3
-0
lines changed

1 file changed

+3
-0
lines changed

advanced_source/semi_structured_sparse.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -55,6 +55,9 @@
5555
from torch.sparse import to_sparse_semi_structured, SparseSemiStructuredTensor
5656
from torch.utils.benchmark import Timer
5757

58+
# the following line may need to be enabled to see a speedup
59+
# SparseSemiStructuredTensor._FORCE_CUTLASS = True
60+
5861
# mask Linear weight to be 2:4 sparse
5962
mask = torch.Tensor([0, 0, 1, 1]).tile((3072, 2560)).cuda().bool()
6063
linear = torch.nn.Linear(10240, 3072).half().cuda().eval()

0 commit comments

Comments
 (0)