[DOCS] Add `pruna` as optimization framework

[davidberenstein1957]

Feature request

Have a section on Pruna AI within the documentation. We did a similar PR for diffusers and thought it would be nice to show how to optimize transformers models too.
.

Motivation

Have a section on Pruna AI within the documentation to show how to optimize LLMs for inference.

Your contribution

We could do everything for the PR.

[Rocketknight1]

cc @stevhliu

[davidberenstein1957]

@Rocketknight1 @stevhliu, do you feel it makes sense to add a similar section to the transformers documentation too?

[stevhliu]

Hmm I'm not really sure.

Most of the docs we include that come from other packages show how they integrate with Transformers (for example, all the different quantization backends). With Pruna, it seems like a wrapper over an existing Transformer model (from what I've seen on the Diffusers PR) and so not as directly related to using Transformers itself if that makes sense.

What do you think @huggingface/transformers-core-maintainers?

[davidberenstein1957]

Hi @stevhliu, In a way, I agree, but, for example, we implement a lot of the quantisation methods mentioned in the docs, as you can see here. However, on top of that, we also do many other optimisation techniques like caching or factorisation. At the moment, it fits in a bit quirkily in the docs, but having a dedicated optimisation section might also encourage other contributors to optimise models for inference. I can imagine libraries like Optimum, Pruna, https://github.com/Knowledgator/FlashDeBERTa, and others might find a nice place there but happy to hear the thought of the rest of the maintainers too.

[ArthurZucker]

Really don't mind as long as its in a section similar to "integrations" or "community projects based on transformers". We have something like that no @stevhliu

[stevhliu]

We currently don't have a section that like in the docs, closest thing we have is awesome_transformers.md.

If you don't mind though, happy to create a section for "Integrations" in the docs! :)

[davidberenstein1957]

@stevhliu, happy to help on this section. @ArthurZucker, shall I open a PR adding Pruna in the integrations section and see where we go from there?

For reference, some work we did with optimizing models on AWS: https://www.pruna.ai/blog/llama-juiced. And a more end2end tutorial we could add to the integration section: https://docs.pruna.ai/en/stable/docs_pruna/tutorials/llms.html

[ArthurZucker]

Yes! feel free to open a PR!