Refactor openvino inference section (#794)

* refactor inference

* update code snippet

* add references

* add table

* add diffusers install

* documentation

* remove task

* Update docs/source/openvino/inference.mdx

Co-authored-by: Helena Kloosterman <[email protected]>

* Update docs/source/openvino/inference.mdx

Co-authored-by: Helena Kloosterman <[email protected]>

* Update docs/source/openvino/tutorials/diffusers.mdx

Co-authored-by: Helena Kloosterman <[email protected]>

* Update docs/source/openvino/inference.mdx

Co-authored-by: Helena Kloosterman <[email protected]>

* update export doc

* format

---------

Co-authored-by: Helena Kloosterman <[email protected]>

Author: echarlaix

docs/Dockerfile

@@ -25,4 +25,4 @@ RUN npm install [email protected] -g && \
 RUN python3 -m pip install --no-cache-dir --upgrade pip
 RUN python3 -m pip install --no-cache-dir git+https://github.com/huggingface/doc-builder.git
 RUN git clone $clone_url && cd optimum-intel && git checkout $commit_sha
-RUN python3 -m pip install --no-cache-dir ./optimum-intel[neural-compressor,openvino,nncf,quality]
+RUN python3 -m pip install --no-cache-dir ./optimum-intel[neural-compressor,openvino,diffusers,quality]

docs/source/_toctree.yml

@@ -22,6 +22,13 @@
       title: Supported Models
     - local: openvino/reference
       title: Reference
+    - sections:
+      - local: openvino/tutorials/notebooks
+        title: Notebooks
+      - local: openvino/tutorials/diffusers
+        title: Generate images with Diffusion models
+      title: Tutorials
+      isExpanded: false
     title: OpenVINO
   title: Optimum Intel
   isExpanded: false

docs/source/openvino/export.mdx

@@ -14,25 +14,15 @@ specific language governing permissions and limitations under the License.
 To export your model to the [OpenVINO IR](https://docs.openvino.ai/2024/documentation/openvino-ir-format.html) format with the CLI :
 
 ```bash
-optimum-cli export openvino --model gpt2 ov_model/
+optimum-cli export openvino --model meta-llama/Meta-Llama-3-8B ov_model/
 ```
 
 The model argument can either be the model ID of a model hosted on the [Hub](https://huggingface.co/models) or a path to a model hosted locally. For local models, you need to specify the task for which the model should be loaded before export, among the list of the [supported tasks](https://huggingface.co/docs/optimum/main/en/exporters/task_manager).
 
-
 ```bash
-optimum-cli export openvino --model local_model_dir --task text-generation-with-past ov_model/
+optimum-cli export openvino --model local_llama --task text-generation-with-past ov_model/
 ```
 
-The `-with-past` suffix enable the re-use of past keys and values. This allows to avoid recomputing the same intermediate activations during the generation. to export the model without, you will need to remove this suffix.
-
-| With K-V cache                           | Without K-V cache                    |
-|------------------------------------------|--------------------------------------|
-| `text-generation-with-past`              | `text-generation`                    |
-| `text2text-generation-with-past`         | `text2text-generation`               |
-| `automatic-speech-recognition-with-past` | `automatic-speech-recognition`       |
-
-
 Check out the help for more options:
 
 ```bash
@@ -97,7 +87,7 @@ Optional arguments:
 You can also apply fp16, 8-bit or 4-bit weight-only quantization on the Linear, Convolutional and Embedding layers when exporting your model by setting `--weight-format` to respectively `fp16`, `int8` or `int4`:
 
 ```bash
-optimum-cli export openvino --model gpt2 --weight-format int8 ov_model/
+optimum-cli export openvino --model meta-llama/Meta-Llama-3-8B --weight-format int8 ov_model/
 ```
 
 For more information on the quantization parameters checkout the [documentation](inference#weight-only-quantization)
@@ -109,6 +99,33 @@ Models larger than 1 billion parameters are exported to the OpenVINO format with
 
 </Tip>
 
+
+### Decoder models
+
+For models with a decoder, we enable the re-use of past keys and values by default. This allows to avoid recomputing the same intermediate activations at each generation step. To export the model without, you will need to remove the `-with-past` suffix when specifying the task.
+
+| With K-V cache                           | Without K-V cache                    |
+|------------------------------------------|--------------------------------------|
+| `text-generation-with-past`              | `text-generation`                    |
+| `text2text-generation-with-past`         | `text2text-generation`               |
+| `automatic-speech-recognition-with-past` | `automatic-speech-recognition`       |
+
+
+### Diffusion models
+
+When Stable Diffusion models are exported to the OpenVINO format, they are decomposed into different components that are later combined during inference:
+
+* Text encoder(s)
+* U-Net
+* VAE encoder
+* VAE decoder
+
+To export your Stable Diffusion XL model to the OpenVINO IR format with the CLI you can do as follows:
+
+```bash
+optimum-cli export openvino --model stabilityai/stable-diffusion-xl-base-1.0 ov_sdxl/
+```
+
 ## When loading your model
 
 You can also load your PyTorch checkpoint and convert it to the OpenVINO format on-the-fly, by setting `export=True` when loading your model.
@@ -121,7 +138,7 @@ To easily save the resulting model, you can use the `save_pretrained()` method,
 + from optimum.intel import OVModelForCausalLM
   from transformers import AutoTokenizer
 
-  model_id = "gpt2"
+  model_id = "meta-llama/Meta-Llama-3-8B"
 - model = AutoModelForCausalLM.from_pretrained(model_id)
 + model = OVModelForCausalLM.from_pretrained(model_id, export=True)
   tokenizer = AutoTokenizer.from_pretrained(model_id)
@@ -137,7 +154,7 @@ To easily save the resulting model, you can use the `save_pretrained()` method,
 from transformers import AutoModelForCausalLM
 from optimum.exporters.openvino import export_from_model
 
-model = AutoModelForCausalLM.from_pretrained("gpt2")
+model = AutoModelForCausalLM.from_pretrained("meta-llama/Meta-Llama-3-8B")
 export_from_model(model, output="ov_model", task="text-generation-with-past")
 ```