Can this framework used for mutil-gpus and work in tensor parallel

[wfloveiu]

Can this framework used for mutil-gpus and work in tensor parallel

[SimJeg]

Hi,

kvpress is compatible with huggingface accelerate for multi-GPU settings:

pipe = pipeline("kv-press-text-generation", model=model, device_map="auto")

I will add it to the README FAQ section