Skip to content

Illegal memory access with quantize_4bit #1782

@davidmezzetti

Description

@davidmezzetti

Running the latest version of bitsandbytes with all recent versions of Torch is leading to the following error. I've tried bitsandbytes version from 0.45.5 to the latest (0.48.0) and all lead to the same result.

import torch
from bitsandbytes.functional import quantize_4bit

data = torch.rand(3000000, 384, device="cuda")
quantize_4bit(data)
Error an illegal memory access was encountered at line 57 in file /src/csrc/ops.cu

I'm running this on Python 3.10, Linux and with a RTX 3090. It seems like there is more than enough free GPU memory.

Metadata

Metadata

Labels

CUDAIssues and PRs related to the CUDA backend, excluding installation/support help.

Type

Projects

No projects

Milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions