Skip to content

Commit 4c87685

Browse files
committed
Merge branch 'vllm_doc_update_121' of github.com:terrastackai/terratorch into vllm_doc_update_121
2 parents 16d936f + 59781a8 commit 4c87685

File tree

1 file changed

+20
-16
lines changed

1 file changed

+20
-16
lines changed

docs/guide/vllm/intro.md

Lines changed: 20 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -7,24 +7,28 @@ can be served with vLLM.
77

88
TerraTorch models can be served with vLLM in _tensor-to-tensor_ or
99
_image-to-image_ mode. The tensor-to-tensor mode is the default mode and is
10-
natively enabled by vLLM. For the image-to-image mode instead, TerraTorch uses a
11-
feature in vLLM called
10+
natively enabled by vLLM. For the image-to-image mode, TerraTorch uses a feature
11+
in vLLM called
1212
[IOProcessor plugins](https://docs.vllm.ai/en/v0.13.0/design/io_processor_plugins/#writing-an-io-processor-plugin),
1313
enabling processing and generation of data in any modality (e.g., geoTiff). In
14-
TerraTorch we provide pre-defined IOProcessor plugins, check the list
15-
[here](./vllm_io_plugins.md#available-terratorch-ioprocessor-plugins).
14+
TerraTorch, we provide pre-defined IOProcessor plugins. Check the list of
15+
[available plugins](./vllm_io_plugins.md#available-terratorch-ioprocessor-plugins).
1616

17-
To enable your model to be served via vLLM, follow the below steps:
17+
To enable your model to be served via vLLM, follow the steps below:
1818

19-
1. Verify the model you want to serve is either already a core model, or learn
20-
how to [add your model to TerraTorch](../models.md#adding-a-new-model).
21-
2. [Prepare your model for serving with vLLM](./prepare_your_model.md).
22-
3. If serving in image-to-image mode
23-
[Learn about IOProcessor plugins](./vllm_io_plugins.md), identify an existing
24-
one suiting your model or
19+
1. **Ensure TerraTorch Integration**: Verify the model you want to serve is
20+
either already a core model, or learn how to
21+
[add your model to TerraTorch](../models.md#adding-a-new-model).
22+
2. **Create a Model _config.json_**: Create a
23+
[vLLM compatible _config.json_](./prepare_your_model.md).
24+
3. **Determine IOProcessor Plugin Needs**: If serving in image-to-image mode,
25+
identify an [IOProcessor plugin](./vllm_io_plugins.md) that suits your model
26+
or
2527
[build one yourself](https://docs.vllm.ai/en/latest/design/io_processor_plugins/).
26-
4. Ensure your model weights and config.json are either hosted on Hugging Face,
27-
or stored in a local directory and accessible by your vLLM instance.
28-
5. Start a vLLM serving instance that loads your model and perform an inference
29-
in [tensor-to-tensor mode](./serving_a_model_tensor.md) or in
30-
[image-to-image mode](./serving_a_model_image.md).
28+
4. **Make your Model Accessible to vLLM**: Host your model weights and
29+
config.json on Hugging Face, or store them in a local directory accessible by
30+
the vLLM instance.
31+
32+
To validate the steps above, start a vLLM serving instance that loads your model
33+
and perform an inference in [tensor-to-tensor mode](./serving_a_model_tensor.md)
34+
or in [image-to-image mode](./serving_a_model_image.md).

0 commit comments

Comments
 (0)