You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: clients/python/llmengine/fine_tuning.py
+6-4Lines changed: 6 additions & 4 deletions
Original file line number
Diff line number
Diff line change
@@ -38,8 +38,10 @@ def create(
38
38
This API can be used to fine-tune a model. The _model_ is the name of base model
39
39
([Model Zoo](../../model_zoo) for available models) to fine-tune. The training
40
40
and validation files should consist of prompt and response pairs. `training_file`
41
-
and `validation_file` must be publicly accessible HTTP or HTTPS URLs to a CSV file
42
-
that includes two columns: `prompt` and `response`. A maximum of 100,000 rows of data is
41
+
and `validation_file` must be either publicly accessible HTTP or HTTPS URLs, or
42
+
file IDs of files uploaded to LLM Engine's [Files API](./#llmengine.File) (these
43
+
will have the `file-` prefix). The referenced files must be CSV files that include
44
+
two columns: `prompt` and `response`. A maximum of 100,000 rows of data is
43
45
currently supported. At least 200 rows of data is recommended to start to see benefits from
44
46
fine-tuning. For sequences longer than the native `max_seq_length` of the model, the sequences
45
47
will be truncated.
@@ -52,10 +54,10 @@ def create(
52
54
The name of the base model to fine-tune. See [Model Zoo](../../model_zoo) for the list of available models to fine-tune.
53
55
54
56
training_file (`str`):
55
-
Publicly accessible URL to a CSV file for training. When no validation_file is provided, one will automatically be created using a 10% split of the training_file data.
57
+
Publicly accessible URL or file ID referencing a CSV file for training. When no validation_file is provided, one will automatically be created using a 10% split of the training_file data.
56
58
57
59
validation_file (`Optional[str]`):
58
-
Publicly accessible URL to a CSV file for validation. The validation file is used to compute metrics which let LLM Engine pick the best fine-tuned checkpoint, which will be used for inference when fine-tuning is complete.
60
+
Publicly accessible URL or file ID referencing a CSV file for validation. The validation file is used to compute metrics which let LLM Engine pick the best fine-tuned checkpoint, which will be used for inference when fine-tuning is complete.
59
61
60
62
hyperparameters (`Optional[Dict[str, str]]`):
61
63
A dict of hyperparameters to customize fine-tuning behavior.
0 commit comments