@@ -9,8 +9,8 @@ To use, simply change the `model_args` to reflect the arguments you want to pass
9
9
10
10
``` bash
11
11
lighteval vllm \
12
- " model_name=HuggingFaceH4/zephyr-7b-beta,dtype=float16 " \
13
- " leaderboard|truthfulqa:mc |0|0"
12
+ " model_name=HuggingFaceH4/zephyr-7b-beta" \
13
+ " extended|ifeval |0|0"
14
14
```
15
15
16
16
` vllm ` is able to distribute the model across multiple GPUs using data
@@ -21,16 +21,16 @@ For example if you have 4 GPUs you can split it across using `tensor_parallelism
21
21
22
22
``` bash
23
23
export VLLM_WORKER_MULTIPROC_METHOD=spawn && lighteval vllm \
24
- " model_name=HuggingFaceH4/zephyr-7b-beta,dtype=float16, tensor_parallel_size=4" \
25
- " leaderboard|truthfulqa:mc |0|0"
24
+ " model_name=HuggingFaceH4/zephyr-7b-beta,tensor_parallel_size=4" \
25
+ " extended|ifeval |0|0"
26
26
```
27
27
28
28
Or, if your model fits on a single GPU, you can use ` data_parallelism ` to speed up the evaluation:
29
29
30
30
``` bash
31
- lighteval vllm \
32
- " model_name=HuggingFaceH4/zephyr-7b-beta,dtype=float16, data_parallel_size=4" \
33
- " leaderboard|truthfulqa:mc |0|0"
31
+ export VLLM_WORKER_MULTIPROC_METHOD=spawn && lighteval vllm \
32
+ " model_name=HuggingFaceH4/zephyr-7b-beta,data_parallel_size=4" \
33
+ " extended|ifeval |0|0"
34
34
```
35
35
36
36
## Use a config file
@@ -41,7 +41,7 @@ An example of a config file is shown below and can be found at `examples/model_c
41
41
``` bash
42
42
lighteval vllm \
43
43
" examples/model_configs/vllm_model_config.yaml" \
44
- " leaderboard|truthfulqa:mc |0|0"
44
+ " extended|ifeval |0|0"
45
45
```
46
46
47
47
``` yaml
0 commit comments