From ea8985d2eb27ffc7abfe81ecdd5472d1fc0392a0 Mon Sep 17 00:00:00 2001 From: Jenny Yang Yi Date: Fri, 9 May 2025 16:32:51 -0400 Subject: [PATCH 1/3] Update datasets.md Added recommended dataset profiles for four usecases --- docs/datasets.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/datasets.md b/docs/datasets.md index 86aa3af2..07b951cf 100644 --- a/docs/datasets.md +++ b/docs/datasets.md @@ -39,6 +39,12 @@ GuideLLM supports several types of datasets, each with its own advantages and us Synthetic datasets allow you to generate data on the fly with customizable parameters. This is useful for controlled experiments, stress testing, and simulating specific scenarios. For example, you might want to evaluate how a model handles long prompts or generates outputs with specific characteristics. +For different use cases, here are the recommendeded dataset profiles that can be passed in as arguments in GuideLLM: +- Chat: --data "prompt_tokens=512,output_tokens=256" +- RAG: --data "prompt_tokens=4096,output_tokens=512" +- Summarization: --data "prompt_tokens=1024,output_tokens=256" +- Code Generation: --data "prompt_tokens=512,output_tokens=512" + #### Example Commands ```bash From f4705a41ab3ef6adeeea64065998e6e805062bd2 Mon Sep 17 00:00:00 2001 From: Jenny Yang Yi Date: Tue, 13 May 2025 12:44:26 -0400 Subject: [PATCH 2/3] Update datasets.md --- docs/datasets.md | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-) diff --git a/docs/datasets.md b/docs/datasets.md index 07b951cf..3ba05e0d 100644 --- a/docs/datasets.md +++ b/docs/datasets.md @@ -39,11 +39,13 @@ GuideLLM supports several types of datasets, each with its own advantages and us Synthetic datasets allow you to generate data on the fly with customizable parameters. This is useful for controlled experiments, stress testing, and simulating specific scenarios. For example, you might want to evaluate how a model handles long prompts or generates outputs with specific characteristics. -For different use cases, here are the recommendeded dataset profiles that can be passed in as arguments in GuideLLM: -- Chat: --data "prompt_tokens=512,output_tokens=256" -- RAG: --data "prompt_tokens=4096,output_tokens=512" -- Summarization: --data "prompt_tokens=1024,output_tokens=256" -- Code Generation: --data "prompt_tokens=512,output_tokens=512" +For different use cases, here are the recommended dataset profiles to pass as arguments in GuideLLM: + +- **Chat**: `--data "prompt_tokens=512,output_tokens=256"` +- **RAG**: `--data "prompt_tokens=4096,output_tokens=512"` +- **Summarization**: `--data "prompt_tokens=1024,output_tokens=256"` +- **Code Generation**: `--data "prompt_tokens=512,output_tokens=512"` + #### Example Commands From e90693a0d3f8f2e2b1e48cc2a28470eaa8404b4b Mon Sep 17 00:00:00 2001 From: Mark Kurtz Date: Tue, 24 Jun 2025 13:00:40 +0000 Subject: [PATCH 3/3] Fixes for styling --- docs/datasets.md | 1 - src/guidellm/utils/cli.py | 1 - 2 files changed, 2 deletions(-) diff --git a/docs/datasets.md b/docs/datasets.md index 3ba05e0d..a5d0aa4e 100644 --- a/docs/datasets.md +++ b/docs/datasets.md @@ -46,7 +46,6 @@ For different use cases, here are the recommended dataset profiles to pass as ar - **Summarization**: `--data "prompt_tokens=1024,output_tokens=256"` - **Code Generation**: `--data "prompt_tokens=512,output_tokens=512"` - #### Example Commands ```bash diff --git a/src/guidellm/utils/cli.py b/src/guidellm/utils/cli.py index 9af6359b..4d83526a 100644 --- a/src/guidellm/utils/cli.py +++ b/src/guidellm/utils/cli.py @@ -46,7 +46,6 @@ def convert(self, value, param, ctx): self.fail("; ".join(fails) or f"Invalid value: {value}") # noqa: RET503 - def get_metavar(self, param: click.Parameter) -> str: def get_choices(t: click.ParamType) -> str: meta = t.get_metavar(param)