Skip to content

Commit b1c43db

Browse files
authored
README: add example for using local model wtth docker container (#528)
* add example for using local model wtth docker container * add git lfs install * change git lfs command based on git-lfs.comm
1 parent b17b588 commit b1c43db

File tree

1 file changed

+12
-0
lines changed

1 file changed

+12
-0
lines changed

README.md

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -150,6 +150,18 @@ The cache path inside the docker container is set by the environment variable `H
150150
```
151151
</details>
152152

153+
#### Using local models with Docker container
154+
155+
In order to deploy a local model with a docker container, you need to mount the model inside the container and specify the path in the container to the launch command.
156+
157+
Example:
158+
```bash
159+
git lfs install
160+
cd /tmp
161+
mkdir models && cd models && git clone https://huggingface.co/BAAI/bge-small-en-v1.5
162+
docker run -it -v /tmp/models:/models -p 8081:8081 michaelf34/infinity:latest v2 --model-id "/models/bge-small-en-v1.5" --port 8081
163+
```
164+
153165
#### Advanced CLI usage
154166

155167
<details>

0 commit comments

Comments
 (0)