Skip to content

Commit 465884d

Browse files
committed
Add multi-turn documentation to readme
1 parent f142b04 commit 465884d

File tree

2 files changed

+10
-5
lines changed

2 files changed

+10
-5
lines changed

README.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -145,7 +145,8 @@ The `guidellm benchmark` command is used to run benchmarks against a generative
145145

146146
- `prompt_tokens`: Average number of tokens for prompts.
147147
- `output_tokens`: Average number of tokens for outputs.
148-
- `TYPE_stdev`, `TYPE_min`, `TYPE_max`: Standard deviation, minimum, and maximum values for the specified type (e.g., `prompt_tokens`, `output_tokens`). If not provided, will use the provided tokens value only.
148+
- `turns`: Average number of request-response pairs per sample. Values above `1` result in a multi-turn[^1] benchmark.
149+
- `TYPE_stdev`, `TYPE_min`, `TYPE_max`: Standard deviation, minimum, and maximum values for the specified type (e.g., `prompt_tokens`, `output_tokens`, `turns`). If not provided, will use the provided tokens value only.
149150
- `samples`: Number of samples to generate, defaults to 1000.
150151
- `source`: Source text data for generation, defaults to a local copy of Pride and Prejudice.
151152

@@ -261,3 +262,7 @@ If you find GuideLLM helpful in your research or projects, please consider citin
261262
howpublished={\url{https://github.com/vllm-project/guidellm}},
262263
}
263264
```
265+
266+
- - -
267+
268+
[^1]: Multi-turn refers to a benchmark where each dataset row represents a series of sequential requests, with each subsequent request building upon the context of the previous ones.

src/guidellm/benchmark/entrypoints.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -90,11 +90,11 @@ async def benchmark_generative_text(
9090
),
9191
random_seed=random_seed,
9292
)
93-
unique_requests = request_loader.num_unique_items(raise_err=False)
93+
unique_samples = request_loader.num_unique_items(raise_err=False)
9494
console.print_line(
95-
f"Created loader with {unique_requests} unique requests from {data}.\n\n"
96-
if unique_requests > 0
97-
else f"Created loader with unknown number unique requests from {data}.\n\n"
95+
f"Created loader with {unique_samples} unique samples from {data}.\n\n"
96+
if unique_samples > 0
97+
else f"Created loader with unknown number unique samples from {data}.\n\n"
9898
)
9999

100100
profile = create_profile(rate_type=rate_type, rate=rate)

0 commit comments

Comments
 (0)