Feat (quant_tensor): convert from NamedTuple to Tensor subclass - #1498
Open
Giuseppe5 wants to merge 6 commits into
Open
Feat (quant_tensor): convert from NamedTuple to Tensor subclass#1498Giuseppe5 wants to merge 6 commits into
Giuseppe5 wants to merge 6 commits into
Conversation
Giuseppe5
force-pushed
the
quant_tensor_subclass
branch
from
July 20, 2026 09:28
a35d9b6 to
a6bccd8
Compare
Collaborator
Author
|
We should remove quant_tensor.tensor since it was there for backwards compatibility but it was deprecated. |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR converts all
QuantTensortypes (IntQuantTensor,FloatQuantTensor,GroupwiseIntQuantTensor,GroupwiseFloatQuantTensor) fromNamedTuplesubclassesto proper
torch.Tensorsubclasses. The underlying tensor data now is thedequantized
value; quantization metadata (scale, zero_point, bit_width, etc.) arestored as regular Python attributes exposed through property accessors.
Motivation
The previous
NamedTuple-based design had several practical limitations:torch.jit.trace, aQuantTensorcould silently decay to a plain
tuple, requiring a fragileget_quant_tensor_classreconstruction heuristic keyed on tuple length.
isinstance(qt, torch.Tensor)wasFalse, so anyPyTorch API that dispatches on tensor type (e.g.
torch.onnx.exportargumenthandling,
torch.compilegraph tracing) had to be worked around with special cases.torch.compile/ dynamo friction: metadata-only caching required storingNoneas the value placeholder, which is not compatible with tensor graph tracing.
Moving to a
torch.Tensorsubclass resolves all of the above cleanly:QuantTensorinstances are real tensors, participate naturally in PyTorch dispatch,and no reconstruction heuristics are needed in tracing paths.
Changes
Core base class (
src/brevitas/quant_tensor/base_quant_tensor.py)QuantTensornow subclassestorch.Tensordirectly.__new__creates the instance viavalue.as_subclass(cls), preservinggrad_fnand
requires_grad.valueproperty returns a plain-Tensorview ofselfviaTensor.as_subclass(self, Tensor)to prevent infinite recursion in operations._field_to_constructor_paramclass attribute for field-name → constructor-parammapping (used by groupwise types where e.g.
scale_maps toscale)._get_constructor_kwargs()helper that collects current metadata into a dictsuitable for passing back to the constructor.
set()is reimplemented (previously delegated toNamedTuple._replace()); it nowbuilds a fresh instance via
_get_constructor_kwargs()and handles thevalue/value_distinction for groupwise types.detach(),contiguous(),to(),cuda(),cpu()are all reimplemented toiterate over
_get_constructor_kwargs()and return a new instance of the same typewith metadata tensors transformed accordingly.
detach_()callssuper().detach_()in addition to detaching each metadata tensor.__iadd__,__imul__,__isub__(non-mutating; return a new value).NamedTuplebase classes:IntQuantTensorBase,FloatQuantTensorBase,GroupwiseFloatQuantTensorBase,GroupwisIntQuantTensorBase.IntQuantTensor(src/brevitas/quant_tensor/int_quant_tensor.py)IntQuantTensorBase._fields = ('scale', 'zero_point', 'bit_width', 'signed', 'training').Note:
valueis no longer a field — it is the tensor itself.__new__performsvalue.as_subclass(cls); all metadata coercion moves to__init__, which stores_scale,_zero_point,_bit_width,signed_t,training_t.scale,zero_point,bit_widthexposed as property getters/setters.FloatQuantTensor(src/brevitas/quant_tensor/float_quant_tensor.py)FloatQuantTensorBase._fieldsfor all 10 metadata attributes (value excluded).__new__/__init__split pattern asIntQuantTensor.scale,zero_point,exponent_bit_width,mantissa_bit_width,exponent_bias,inf_values,nan_values) exposed asproperty getters/setters.
GroupwiseIntQuantTensor(src/brevitas/quant_tensor/groupwise_int_quant_tensor.py)GroupwisIntQuantTensorBase._fields(8 entries, usingscale_/zero_point_names for the groupedtensors) and
_field_to_constructor_param = {'scale_': 'scale', 'zero_point_': 'zero_point'}._is_groupwise = Trueso thatset()/detach()/etc. in the base class use_value_instead ofvalue.self._value_;self(as a tensor) holds thedequantized/expanded representation used by
.value.group_size,group_dim,bit_width,dequant_shapeexposed as properties.__neg__and__abs__to referenceself.zero_point_/self.scale_instead of the removed
self.zero_point/self.scale.saturatingproperty that was accidentally inherited.GroupwiseFloatQuantTensor(src/brevitas/quant_tensor/groupwise_float_quant_tensor.py)GroupwiseIntQuantTensor._fieldscovers 13 metadata attributes.exponent_bit_width,mantissa_bit_width,exponent_bias,inf_values,nan_values,dequant_shape) exposed as property getters/setters.__neg__and__abs__to useself.zero_point_/self.scale_.__init__.py(src/brevitas/quant_tensor/__init__.py)from .base_quant_tensor import *(which exported the now-deletedNamedTuple base classes) to explicit named imports of
_unpack_quant_tensorandQuantTensor.NN mixin (
src/brevitas/nn/mixin/base.py)get_quant_tensor_class()— the tuple-length heuristic for reconstructinga
QuantTensorfrom a decayed tuple is no longer needed.unpack_input(). SinceQuantTensoris now a realTensor, it does not decay to a plain tuple duringtracing.
FloatQuantTensor,GroupwiseFloatQuantTensor,GroupwiseIntQuantTensorfrom this module.ONNX export (
src/brevitas/export/onnx/manager.py)isinstance(args, tuple)guard: previously it wasisinstance(args, tuple) and not isinstance(args, QuantTensor)becauseQuantTensorwas aNamedTuple(hence atuple). This exclusion is no longerneeded.
Proxy (
src/brevitas/proxy/parameter_quant.py)isinstance(quant_input, Tensor)tonot isinstance(quant_input, IntQuantTensor),since
IntQuantTensoris now itself aTensorsubclass and the old check wouldalways be
True.Caching utilities (
src/brevitas/utils/quant_utils.py)_CachedIO,_CachedIOFloat,_CachedIOGroupwiseFloat,_CachedIOGroupwiseInt:metadata-only caching now stores
torch.empty(0)as the value placeholder insteadof
None. ANonevalue is not valid inside atorch.Tensorsubclass and breakstorch.compilegraph tracing.Notebook (
notebooks/minifloat_mx_tutorial.ipynb)Behavioral / Migration Notes
Code that worked with the old
NamedTuple-basedQuantTensormay need updates inthe following areas:
isinstance(qt, tuple)→Trueisinstance(qt, tuple)→Falseisinstance(qt, torch.Tensor)→Falseisinstance(qt, torch.Tensor)→Truelen(qt._fields) == 6forIntQuantTensorlen(qt._fields) == 5(valueis no longer a field)qt._replace(scale=s)(NamedTuple API)qt.set(scale=s)(already the public API)qt.tensor(alias for.value).valuedirectlyqt.set(value=None)for metadata-only cachingqt.set(value=torch.empty(0))qt[0],qt[1], … index accessQuantTensordecays totupleduring tracingTensorThe public
.value,.scale,.zero_point,.bit_width(and float equivalents)property API is unchanged.
Testing
tests/brevitas/quant_tensor/test_quant_tensor.py— updated assertions:isinstance(qt, torch.Tensor)(wastuple);len(qt._fields) == 5(was 6).tests/brevitas/quant_tensor/test_quant_tensor_subclass.py(new, 513 lines) —comprehensive test suite covering the new subclass behaviour:
TestIsInstance—IntQuantTensor,FloatQuantTensor,GroupwiseFloatQuantTensorare all
isinstanceoftorch.TensorandQuantTensor.TestValueProperty—.valuereturns a plaintorch.Tensor(not a subclass);.valueandselfshare data;grad_fnandrequires_gradare preserved.TestSetMethod—set()replaces fields correctly, preserves type, no-args copy.TestInPlaceOperators—+=,*=,-=return valid results and preserve type.TestTorchFunctionFallback— unhandled functions fall back to plain tensor outputwithout recursion; handled functions (
relu,max_pool2d) route correctly.TestTensorOps—detach(),contiguous(),to(dtype)preserve type and metadata.TestUnpackQuantTensor— plain tensor passthrough; tuple/dict/nested unpacking.TestRightHandOperators—__radd__,__rmul__with plain-tensor LHS.TestFields—_fieldscontents forInt,Float,GroupwiseFloatvariants.TestShapeSizeDim—.shape,.size(),.dim()delegate correctly.TestConstructionFromNonTensor— construction from Python float or list.TestDynamoExportCacheClass—_CachedIOmetadata-only path works end-to-end.