Skip to content

Commit 94890cd

Browse files
committed
Add multi-turn documentation to readme
1 parent 7192781 commit 94890cd

File tree

2 files changed

+10
-5
lines changed

2 files changed

+10
-5
lines changed

README.md

Lines changed: 6 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -126,7 +126,8 @@ The `guidellm benchmark` command is used to run benchmarks against a generative
126126

127127
- `prompt_tokens`: Average number of tokens for prompts.
128128
- `output_tokens`: Average number of tokens for outputs.
129-
- `TYPE_stdev`, `TYPE_min`, `TYPE_max`: Standard deviation, minimum, and maximum values for the specified type (e.g., `prompt_tokens`, `output_tokens`). If not provided, will use the provided tokens value only.
129+
- `turns`: Average number of request-response pairs per sample. Values above `1` result in a multi-turn[^1] benchmark.
130+
- `TYPE_stdev`, `TYPE_min`, `TYPE_max`: Standard deviation, minimum, and maximum values for the specified type (e.g., `prompt_tokens`, `output_tokens`, `turns`). If not provided, will use the provided tokens value only.
130131
- `samples`: Number of samples to generate, defaults to 1000.
131132
- `source`: Source text data for generation, defaults to a local copy of Pride and Prejudice.
132133

@@ -208,3 +209,7 @@ If you find GuideLLM helpful in your research or projects, please consider citin
208209
howpublished={\url{https://github.com/vllm-project/guidellm}},
209210
}
210211
```
212+
213+
- - -
214+
215+
[^1]: Multi-turn refers to a benchmark where each dataset row represents a series of sequential requests, with each subsequent request building upon the context of the previous ones.

src/guidellm/benchmark/entrypoints.py

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -90,11 +90,11 @@ async def benchmark_generative_text(
9090
),
9191
random_seed=random_seed,
9292
)
93-
unique_requests = request_loader.num_unique_items(raise_err=False)
93+
unique_samples = request_loader.num_unique_items(raise_err=False)
9494
console.print_line(
95-
f"Created loader with {unique_requests} unique requests from {data}.\n\n"
96-
if unique_requests > 0
97-
else f"Created loader with unknown number unique requests from {data}.\n\n"
95+
f"Created loader with {unique_samples} unique samples from {data}.\n\n"
96+
if unique_samples > 0
97+
else f"Created loader with unknown number unique samples from {data}.\n\n"
9898
)
9999

100100
profile = create_profile(rate_type=rate_type, rate=rate)

0 commit comments

Comments
 (0)