Skip to content

[Feature Request][Help Wanted] Convert AutoAWQ checkpoints to compressed-tensors #2087

@dsikka

Description

@dsikka

Is your feature request related to a problem? Please describe.

Describe the solution you'd like

  • A tool which when given an AutoAWQ checkpoint, produced a compressed-tensors formatted model
  • The model should be able to run in vLLM without any drop in accuracy

Working implementation will have to account for AutoAWQ's unique ordering for packing 8 int4 weight into a single int32. More detail here:
#1909 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    awqFor any issue / PR related to AWQ supportcompressed-tensorsRelates to compressed-tensorsenhancementNew feature or requestgood first issueA good first issue for users wanting to contributegood follow-up issueA good issue for users with some familiarity of the codebasewNa16Anything related to weight-only int-quantized support

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions