You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
In order to run GGUF models, the model needs to be converted to Big-Endian. You can achieve this in three cases:
69
+
All models need to be converted to Big-Endian. You can achieve this in three cases:
70
70
71
71
1. Use pre-converted models verified for use on IBM Z & LinuxONE (easiest)
72
72
73
-
You can find popular models pre-converted and verified at [s390x Ready Models](hf.co/collections/taronaeo/s390x-ready-models-672765393af438d0ccb72a08).
73
+
You can find popular models pre-converted and verified at [s390x Ready Models](hf.co/collections/taronaeo/s390x-ready-models-672765393af438d0ccb72a08).
74
74
75
-
These models and their respective tokenizers are verified to run correctly on IBM Z & LinuxONE.
75
+
These models and their respective tokenizers are verified to run correctly on IBM Z & LinuxONE.
76
76
77
77
2. Convert safetensors model to GGUF Big-Endian directly (recommended)
78
78
79
-
```bash
80
-
python3 convert_hf_to_gguf.py \
81
-
--outfile model-name-be.f16.gguf \
82
-
--outtype f16 \
83
-
--bigendian \
84
-
model-directory/
85
-
```
86
-
87
-
For example,
79
+
```bash
80
+
python3 convert_hf_to_gguf.py \
81
+
--outfile model-name-be.f16.gguf \
82
+
--outtype f16 \
83
+
--bigendian \
84
+
model-directory/
85
+
```
88
86
89
-
```bash
90
-
python3 convert_hf_to_gguf.py \
91
-
--outfile granite-3.3-2b-instruct-be.f16.gguf \
92
-
--outtype f16 \
93
-
--bigendian \
94
-
granite-3.3-2b-instruct/
95
-
```
87
+
For example,
88
+
89
+
```bash
90
+
python3 convert_hf_to_gguf.py \
91
+
--outfile granite-3.3-2b-instruct-be.f16.gguf \
92
+
--outtype f16 \
93
+
--bigendian \
94
+
granite-3.3-2b-instruct/
95
+
```
96
96
97
97
3. Convert existing GGUF Little-Endian model to Big-Endian
98
98
99
-
```bash
100
-
python3 gguf-py/gguf/scripts/gguf_convert_endian.py model-name.f16.gguf BIG
101
-
```
102
-
103
-
For example,
104
-
```bash
105
-
python3 gguf-py/gguf/scripts/gguf_convert_endian.py granite-3.3-2b-instruct-le.f16.gguf BIG
- The GGUF endian conversion script may not support all data types at the moment and may fail for some models/quantizations. When that happens, please try manually converting the safetensors model to GGUF Big-Endian via Step 2.
99
+
```bash
100
+
python3 gguf-py/gguf/scripts/gguf_convert_endian.py model-name.f16.gguf BIG
101
+
```
102
+
103
+
For example,
104
+
```bash
105
+
python3 gguf-py/gguf/scripts/gguf_convert_endian.py granite-3.3-2b-instruct-le.f16.gguf BIG
- The GGUF endian conversion script may not support all data types at the moment and may fail for some models/quantizations. When that happens, please try manually converting the safetensors model to GGUF Big-Endian via Step 2.
0 commit comments