You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Added workflow to package a safetensors model (docker#100)
* Added workflow to package a safetensors model
* Build llama-converter image
* No need to install git-lfs
* Use linux/amd6 platform
* No need for intermediate image stage llama-converter
* remove duplicated FROM
* Update README.md
* add prerpare-matrix step as gguf workflow
* fix example
* llamacpp tag always came from the input
* Update README.md
Copy file name to clipboardExpand all lines: pkg/distribution/README.md
+75-5Lines changed: 75 additions & 5 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -13,6 +13,8 @@ Model Distribution is a Go library and CLI tool that allows you to package, push
13
13
- Local model storage
14
14
- Model metadata management
15
15
- Command-line interface for all operations
16
+
- GitHub workflows for automated model packaging
17
+
- Support for both GGUF and safetensors model formats
16
18
17
19
## Usage
18
20
@@ -123,27 +125,31 @@ if err != nil {
123
125
124
126
### GitHub Workflows for Model Packaging and Promotion
125
127
126
-
This project provides GitHub workflows to automate the process of packaging GGUF models and promoting them from staging to production environments.
128
+
This project provides GitHub workflows to automate the process of packaging both GGUF and Safetensors models and promoting them from staging to production environments.
127
129
128
130
#### Overview
129
131
130
132
The model promotion process follows a two-step workflow:
131
-
1.**Package and Push to Staging**: Use `package-gguf-model.yml` to download a GGUF model from HuggingFace and push it to the `aistaging` namespace
133
+
1.**Package and Push to Staging**: Use either:
134
+
-`package-gguf-model.yml` to download a pre-built GGUF model and push it to the `aistaging` namespace
135
+
-`package-safetensors-model.yml` to clone a safetensors model from HuggingFace, convert it to GGUF, and push it to the `aistaging` namespace
132
136
2.**Promote to Production**: Use `promote-model-to-production.yml` to copy the model from staging (`aistaging`) to production (`ai`) namespace
133
137
134
138
#### Prerequisites
135
139
136
140
The following GitHub secrets must be configured:
137
141
-`DOCKER_USER`: DockerHub username for production namespace
138
142
-`DOCKER_OAT`: DockerHub access token for production namespace
139
-
-`DOCKER_USER_STAGING`: DockerHub username for staging namespace (typically `aistaging`)
143
+
-`DOCKER_USER_STAGING`: DockerHub username for staging namespace (`aistaging`)
140
144
-`DOCKER_OAT_STAGING`: DockerHub access token for staging namespace
141
145
142
146
**Note**: The current secrets are configured to write to the `ai` production namespace. If you need to write to a different namespace, you'll need to update the `DOCKERHUB_USERNAME` and `DOCKERHUB_TOKEN` secrets accordingly.
143
147
144
148
#### Step 1: Package Model to Staging
145
149
146
-
Use the **Package GGUF model** workflow to download a model from HuggingFace and push it to the staging environment.
150
+
##### Option A: Package GGUF Model
151
+
152
+
Use the **Package GGUF model** workflow to download a pre-built GGUF model and push it to the staging environment.
147
153
148
154
**Single Model Example:**
149
155
1. Go to Actions → Package GGUF model → Run workflow
@@ -174,6 +180,43 @@ For packaging multiple models at once, use the `models_json` input:
174
180
]
175
181
```
176
182
183
+
##### Option B: Package Safetensors Model
184
+
185
+
Use the **Package Safetensors model** workflow to clone a Safetensors model from HuggingFace, convert it to GGUF format, and push it to the staging environment.
186
+
187
+
**Single Model Example:**
188
+
1. Go to Actions → Package Safetensors model → Run workflow
0 commit comments