|
1 | 1 | # modctl |
2 | 2 |
|
3 | | -Modctl is a user-friendly CLI tool for managing OCI model artifacts, which are bundled based on [Model Spec V1](https://github.com/CloudNativeAI/model-spec). |
4 | | -It offers commands such as `build`, `pull`, `push`, and more, making it easy for users to convert their AI models into OCI artifacts. |
5 | | - |
6 | | -## Usage |
7 | | - |
8 | | -### Installation |
9 | | - |
10 | | -#### Build from source |
11 | | - |
12 | | -```shell |
13 | | -$ git clone https://github.com/CloudNativeAI/modctl.git |
14 | | -$ make |
15 | | -$ ./output/modctl -h |
16 | | -``` |
17 | | - |
18 | | -### Build |
19 | | - |
20 | | -Build the model artifact you need to prepare a Modelfile describe your expected layout of the model artifact in your model repo. |
21 | | - |
22 | | -Example of Modelfile: |
23 | | - |
24 | | -```shell |
25 | | -# Model name (string), such as llama3-8b-instruct, gpt2-xl, qwen2-vl-72b-instruct, etc. |
26 | | -NAME gemma-2b |
27 | | - |
28 | | -# Model architecture (string), such as transformer, cnn, rnn, etc. |
29 | | -ARCH transformer |
30 | | - |
31 | | -# Model family (string), such as llama3, gpt2, qwen2, etc. |
32 | | -FAMILY gemma |
33 | | - |
34 | | -# Model format (string), such as onnx, tensorflow, pytorch, etc. |
35 | | -FORMAT safetensors |
36 | | - |
37 | | -# Number of parameters in the model (integer). |
38 | | -PARAMSIZE 16 |
39 | | - |
40 | | -# Model precision (string), such as bf16, fp16, int8, etc. |
41 | | -PRECISION bf16 |
42 | | - |
43 | | -# Model quantization (string), such as awq, gptq, etc. |
44 | | -QUANTIZATION awq |
45 | | - |
46 | | -# Specify model configuration file. |
47 | | -CONFIG config.json |
| 3 | +[](https://github.com/CloudNativeAI/modctl/releases) |
| 4 | +[](https://github.com/CloudNativeAI/modctl/blob/main/LICENSE) |
48 | 5 |
|
49 | | -# Specify model configuration file. |
50 | | -CONFIG generation_config.json |
51 | | - |
52 | | -# Model weight. |
53 | | -MODEL \.safetensors$ |
54 | | -``` |
55 | | - |
56 | | -Then run the following command to build the model artifact: |
57 | | - |
58 | | -```shell |
59 | | -$ modctl build -t 127.0.0.1/library/test-models:v1 -f Modelfile . |
60 | | -``` |
61 | | - |
62 | | -### Pull & Push |
63 | | - |
64 | | -Before the `pull` or `push` command, you need to login the registry: |
65 | | - |
66 | | -```shell |
67 | | -$ modctl login -u username -p password example.registry.com |
68 | | -``` |
69 | | - |
70 | | -Pull the model artifact from the registry: |
71 | | - |
72 | | -```shell |
73 | | -$ modctl pull 127.0.0.1/library/test-models:v1 |
74 | | -``` |
75 | | - |
76 | | -Push the model artifact to the registry: |
77 | | - |
78 | | -```shell |
79 | | -$ modctl push 127.0.0.1/library/test-models:v1 |
80 | | -``` |
81 | | - |
82 | | -### List |
83 | | - |
84 | | -List the model artifacts in the local storage: |
85 | | - |
86 | | -```shell |
87 | | -$ modctl ls |
88 | | -``` |
89 | | - |
90 | | -### Cleanup |
| 6 | +Modctl is a user-friendly CLI tool for managing OCI model artifacts, which are bundled based on [Model Spec](https://github.com/CloudNativeAI/model-spec). |
| 7 | +It offers commands such as `build`, `pull`, `push`, and more, making it easy for users to convert their AI models into OCI artifacts. |
91 | 8 |
|
92 | | -Delete the model artifact in the local storage: |
| 9 | +## Documentation |
93 | 10 |
|
94 | | -```shell |
95 | | -$ modctl rm 127.0.0.1/library/test-models:v1 |
96 | | -``` |
| 11 | +You can find the full documentation on the [getting started](./docs/getting-started.md). |
97 | 12 |
|
98 | | -Finally, you can use `purge` command to to remove all unnecessary blobs to free up the storage space: |
| 13 | +## LICENSE |
99 | 14 |
|
100 | | -```shell |
101 | | -$ modctl purge |
102 | | -``` |
| 15 | +Apache 2.0 License. Please see [LICENSE](LICENSE) for more information. |
0 commit comments