Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions docs/datasets.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,13 @@ GuideLLM supports several types of datasets, each with its own advantages and us

Synthetic datasets allow you to generate data on the fly with customizable parameters. This is useful for controlled experiments, stress testing, and simulating specific scenarios. For example, you might want to evaluate how a model handles long prompts or generates outputs with specific characteristics.

For different use cases, here are the recommended dataset profiles to pass as arguments in GuideLLM:

- **Chat**: `--data "prompt_tokens=512,output_tokens=256"`
- **RAG**: `--data "prompt_tokens=4096,output_tokens=512"`
- **Summarization**: `--data "prompt_tokens=1024,output_tokens=256"`
- **Code Generation**: `--data "prompt_tokens=512,output_tokens=512"`

#### Example Commands

```bash
Expand Down
1 change: 0 additions & 1 deletion src/guidellm/utils/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,6 @@ def convert(self, value, param, ctx):

self.fail("; ".join(fails) or f"Invalid value: {value}") # noqa: RET503


def get_metavar(self, param: click.Parameter) -> str:
def get_choices(t: click.ParamType) -> str:
meta = t.get_metavar(param)
Expand Down