You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+19-12Lines changed: 19 additions & 12 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,7 +28,7 @@ The package doesn't have the dataset, it is stored on our [HuggingFace page](htt
28
28
29
29
### This package contains
30
30
- Support for modern LLMs.
31
-
- Tools for **evaluation**, **inference**, and **finetuning**.
31
+
- Tools for **inference** and **evaluation**.
32
32
- Support for Hugging Face models out-of-the-box.
33
33
- Structured for reproducibility and benchmarking.
34
34
@@ -46,10 +46,10 @@ We therefore recommend that most users:
46
46
- Evaluate results against the benchmark with the [`llmsql.LLMSQLEvaluator`](./llmsql/evaluation/evaluator.py) evaluator class.
47
47
48
48
2.**Optional finetuning**:
49
-
- For research or domain adaptation, we provide finetuning script for HF models. Use `llmsql finetune --help` or read [Finetune Readme](./llmsql/finetune/README.md) to find more about finetuning.
49
+
- For research or domain adaptation, we provide finetuning version for HF models. Use [Finetune Ready](https://huggingface.co/datasets/llmsql-bench/llmsql-benchmark-finetune-ready) dataset from HuggingFace.
50
50
51
51
> [!Tip]
52
-
> You can find additional manuals in the README files of each folder([Inferece Readme](./llmsql/inference/README.md), [Evaluation Readme](./llmsql/evaluation/README.md), [Finetune Readme](./llmsql/finetune/README.md))
52
+
> You can find additional manuals in the README files of each folder([Inferece Readme](./llmsql/inference/README.md), [Evaluation Readme](./llmsql/evaluation/README.md))
53
53
54
54
> [!Tip]
55
55
> vllm based inference require vllm optional dependency group installed: `pip install llmsql[vllm]`
@@ -61,9 +61,7 @@ We therefore recommend that most users:
61
61
62
62
llmsql/
63
63
├── evaluation/ # Scripts for downloading DB + evaluating predictions
64
-
├── inference/ # Generate SQL queries with your LLM
65
-
└── finetune/ # Fine-tuning with TRL's SFTTrainer
66
-
64
+
└── inference/ # Generate SQL queries with your LLM
67
65
```
68
66
69
67
@@ -111,21 +109,30 @@ print(report)
111
109
112
110
113
111
114
-
## Finetuning (Optional)
115
-
116
-
If you want to adapt a base model on LLMSQL:
112
+
## Vllm inference (Recommended)
117
113
114
+
To speed up your inference we recommend using vllm inference. You can do it with optional llmsql[vllm] dependency group
This will train a model on the train/val splits with the parameters provided in the config file. You can find example config file [here](./examples/example_finetune_args.yaml).
119
+
After that run
120
+
```python
121
+
from llmsql import inference_vllm
122
+
results = inference_vllm(
123
+
"Qwen/Qwen2.5-1.5B-Instruct",
124
+
"test_results.jsonl",
125
+
do_sample=False,
126
+
batch_size=20000
127
+
)
128
+
```
129
+
for fast inference.
123
130
124
131
125
132
126
133
## Suggested Workflow
127
134
128
-
***Primary**: Run inference on `dataset/questions.jsonl` → Evaluate with `evaluation/`.
135
+
***Primary**: Run inference on `dataset/questions.jsonl`with vllm → Evaluate with `evaluation/`.
129
136
***Secondary (optional)**: Fine-tune on `train/val` → Test on `test_questions.jsonl`.
0 commit comments