feat: add conversation context mode for multi-turn history control

Introduce ConversationContextMode enum (accumulate_all, drop_responses,
standalone) to control how prior turns are accumulated in multi-turn
conversations. Modes resolve with conversation > dataset default >
accumulate_all precedence. Standalone replaces turn_list with only the
current turn; drop_responses skips storing assistant responses.

Signed-off-by: Anthony Casagrande <acasagrande@nvidia.com>

Author: ajcasagrande

README.md

@@ -74,6 +74,7 @@ aiperf profile \
 - [Reproducibility](docs/reproducibility.md) - Deterministic datasets with `--random-seed`
 - [Template Endpoint](docs/tutorials/template-endpoint.md) - Custom Jinja2 request templates
 - [Multi-Turn Conversations](docs/tutorials/multi-turn.md) - Multi-turn conversation benchmarking
+- [Conversation Context Mode](docs/tutorials/conversation-context-mode.md) - Control how conversation history accumulates
 - [Local Tokenizer](docs/tutorials/local-tokenizer.md) - Use local tokenizers without HuggingFace
 
 ### Endpoint Types

docs/tutorials/conversation-context-mode.md

@@ -0,0 +1,97 @@
+<!--
+# SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.
+# SPDX-License-Identifier: Apache-2.0
+-->
+# Conversation Context Mode
+
+Conversation context mode controls how prior turns are accumulated when building multi-turn chat requests. Different dataset formats imply different accumulation strategies, and AIPerf automatically selects the right one based on your data.
+
+## Modes
+
+### `accumulate_all`
+
+Standard multi-turn chat. The live inference response is stored and included in subsequent requests.
+
+**Dataset:**
+```
+Turn 1: {"role": "user", "content": "What is ML?"}
+Turn 2: {"role": "user", "content": "Give an example"}
+Turn 3: {"role": "user", "content": "How does it differ from traditional programming?"}
+```
+
+**Replay:**
+```
+Request 1: [User "What is ML?"]
+  → Server responds with A1
+
+Request 2: [User "What is ML?", Assistant A1, User "Give an example"]
+  → Server responds with A2
+
+Request 3: [User "What is ML?", Assistant A1, User "Give an example", Assistant A2, User "How does it differ..."]
+  → Server responds with A3
+```
+
+Default for:
+- Synthetic datasets
+- Multi-turn JSONL
+- ShareGPT
+- Mooncake traces with `hash_ids`
+
+### `drop_responses`
+
+Delta-compressed prompts. Each dataset turn only contains the *new* messages since the previous turn. AIPerf accumulates these deltas to reconstruct the full conversation. The live inference response is only used for measurement and discarded -- the pre-canned assistant responses in the dataset are used instead.
+
+**Dataset (each turn is a delta):**
+```
+Turn 1: [{"role": "user", "content": "What is ML?"}]
+Turn 2: [{"role": "assistant", "content": "ML is..."}, {"role": "user", "content": "Give an example"}]
+Turn 3: [{"role": "assistant", "content": "Sure..."}, {"role": "user", "content": "How does it differ..."}]
+```
+
+**Replay (deltas accumulated):**
+```
+Request 1: [User "What is ML?"]
+  → Live response discarded
+
+Request 2: [User "What is ML?"] + [Assistant "ML is...", User "Give an example"]
+  → Live response discarded
+
+Request 3: [User "What is ML?"] + [Assistant "ML is...", User "Give an example"] + [Assistant "Sure...", User "How does it differ..."]
+  → Live response discarded
+```
+
+Default for:
+- N/A (no built-in loader defaults to this mode yet)
+
+### `standalone`
+
+Self-contained prompts. Each turn already contains its full context. No session accumulation.
+
+**Dataset:**
+```
+Turn 1: [{"role": "user", "content": "What is ML?"}]
+Turn 2: [{"role": "user", "content": "What is ML?"}, {"role": "assistant", "content": "ML is..."}, {"role": "user", "content": "Give an example"}]
+Turn 3: [{"role": "user", "content": "What is ML?"}, {"role": "assistant", "content": "ML is..."}, {"role": "user", "content": "Give an example"}, {"role": "assistant", "content": "Sure..."}, {"role": "user", "content": "How does it differ..."}]
+```
+
+**Replay:**
+```
+Request 1: sends Turn 1 as-is
+Request 2: sends Turn 2 as-is
+Request 3: sends Turn 3 as-is
+```
+
+Each turn is sent exactly as it appears in the dataset.
+
+Default for:
+- Mooncake traces with pre-built `messages` arrays
+
+## How It Works
+
+Context mode is resolved through a priority chain:
+
+1. **Per-conversation override** -- A conversation in the dataset can specify its own `context_mode`
+2. **Loader default** -- The dataset loader can declare a default based on dataset format semantics
+3. **Global fallback** -- `accumulate_all`
+
+This means most users never need to think about context mode. The loader picks the right default, and individual conversations can override it when needed.

docs/tutorials/multi-turn.md

@@ -442,3 +442,6 @@ The delays between turns are controlled by:
 - Consider using `--request-rate` to control conversation start rate for more predictable load
 - Use `--random-seed` for reproducible conversation patterns
 
+**See also:**
+- [Conversation Context Mode](conversation-context-mode.md) — Control how conversation history accumulates (delta-compressed, standalone, etc.)
+

src/aiperf/common/enums/__init__.py

@@ -13,6 +13,7 @@
     CommandResponseStatus,
     CommandType,
     ConnectionReuseStrategy,
+    ConversationContextMode,
     CreditPhase,
     ExportLevel,
     GPUTelemetryMode,
@@ -77,6 +78,7 @@
     "CommandResponseStatus",
     "CommandType",
     "ConnectionReuseStrategy",
+    "ConversationContextMode",
     "CreditPhase",
     "EnergyMetricUnit",
     "EnergyMetricUnitInfo",

src/aiperf/common/enums/enums.py

@@ -96,6 +96,23 @@ class CommandResponseStatus(CaseInsensitiveStrEnum):
     UNHANDLED = "unhandled"  # The command was received but not handled by any hook
 
 
+class ConversationContextMode(CaseInsensitiveStrEnum):
+    """Controls how prior turns are accumulated in multi-turn conversations.
+
+    The context mode is a property of how the dataset was constructed.
+    It determines what conversation history is included in each request.
+    """
+
+    ACCUMULATE_ALL = "accumulate_all"
+    """Standard multi-turn chat. Both user and assistant turns are kept in history."""
+
+    DROP_RESPONSES = "drop_responses"
+    """Delta-compressed prompts. Dataset turns accumulate but live inference responses are discarded."""
+
+    STANDALONE = "standalone"
+    """Self-contained prompts. Each turn already has full context; no prior turns included."""
+
+
 class ConnectionReuseStrategy(CaseInsensitiveStrEnum):
     """Transport connection reuse strategy. Controls how and when connections are reused across requests."""
 

src/aiperf/common/models/dataset_models.py

@@ -7,7 +7,7 @@
 
 from pydantic import Field
 
-from aiperf.common.enums import MediaType
+from aiperf.common.enums import ConversationContextMode, MediaType
 from aiperf.common.models.base_models import AIPerfBaseModel
 from aiperf.common.types import MediaTypeT
 from aiperf.plugin.enums import DatasetClientStoreType, DatasetSamplingStrategy
@@ -246,6 +246,12 @@ class DatasetMetadata(AIPerfBaseModel):
         default=False,
         description="Whether the dataset has timing data (timestamps/delays in turns).",
     )
+    default_context_mode: ConversationContextMode | None = Field(
+        default=None,
+        description="Dataset-level default for how prior turns are accumulated. "
+        "Set by the loader based on dataset format semantics. "
+        "Individual conversations can override this via their own context_mode field.",
+    )
 
     @cached_property
     def total_turn_count(self) -> int:
@@ -270,6 +276,11 @@ class Conversation(AIPerfBaseModel):
     session_id: str = Field(
         default="", description="Unique identifier for the conversation."
     )
+    context_mode: ConversationContextMode | None = Field(
+        default=None,
+        description="How prior turns are accumulated for this conversation. "
+        "When None, inherits the dataset-level default.",
+    )
     turns: list[Turn] = Field(
         default=[], description="List of turns in the conversation."
     )

src/aiperf/dataset/composer/custom.py

@@ -10,6 +10,7 @@
 from aiperf.common.tokenizer import Tokenizer
 from aiperf.common.utils import load_json_str
 from aiperf.dataset.composer.base import BaseDatasetComposer
+from aiperf.dataset.loader.base_loader import BaseLoader
 from aiperf.dataset.utils import check_file_exists
 from aiperf.plugin import plugins
 from aiperf.plugin.enums import CustomDatasetType, PluginType
@@ -18,6 +19,7 @@
 class CustomDatasetComposer(BaseDatasetComposer):
     def __init__(self, config: UserConfig, tokenizer: Tokenizer | None):
         super().__init__(config, tokenizer)
+        self.loader: BaseLoader | None = None
 
     def create_dataset(self) -> list[Conversation]:
         """Create conversations from a file or directory.

src/aiperf/dataset/dataset_manager.py

@@ -14,6 +14,7 @@
 from aiperf.common.enums import (
     CommAddress,
     CommandType,
+    ConversationContextMode,
     CreditPhase,
     MessageType,
     PublicDatasetType,
@@ -108,6 +109,7 @@ def __init__(
             compress_only=self._compress_only,
         )
         self._dataset_client: DatasetClientStoreProtocol | None = None
+        self._default_context_mode: ConversationContextMode | None = None
 
     @on_command(CommandType.PROFILE_CONFIGURE)
     async def _profile_configure_command(
@@ -291,14 +293,21 @@ async def _load_public_dataset(self) -> list[Conversation]:
             self.user_config.input.dataset_sampling_strategy = (
                 loader.get_recommended_sampling_strategy()
             )
+        self._default_context_mode = loader.get_default_context_mode()
         return await loader.convert_to_conversations(dataset)
 
     def _load_custom_dataset(self) -> list[Conversation]:
         ComposerClass = plugins.get_class(
             PluginType.DATASET_COMPOSER, ComposerType.CUSTOM
         )
         composer = ComposerClass(config=self.user_config, tokenizer=self.tokenizer)
-        return composer.create_dataset()
+        conversations = composer.create_dataset()
+        self._default_context_mode = (
+            composer.loader.get_default_context_mode()
+            if composer.loader is not None
+            else None
+        )
+        return conversations
 
     def _is_rankings_endpoint(self, endpoint_type: str) -> bool:
         return "rankings" in endpoint_type.lower()
@@ -321,6 +330,7 @@ async def _configure_dataset(self) -> None:
 
         self.dataset_configured.clear()
 
+        self._default_context_mode = None
         if self.user_config.input.public_dataset is not None:
             conversations = await self._load_public_dataset()
         elif (
@@ -364,6 +374,7 @@ async def _configure_dataset(self) -> None:
         self.dataset_metadata = DatasetMetadata(
             conversations=[conversation.metadata() for conversation in conversations],
             sampling_strategy=self.user_config.input.dataset_sampling_strategy,
+            default_context_mode=self._default_context_mode,
         )
         self.info(
             f"sampling strategy: {self.dataset_metadata.sampling_strategy}, "

src/aiperf/dataset/loader/base_loader.py

@@ -4,6 +4,7 @@
 from abc import ABC, abstractmethod
 
 from aiperf.common.config.user_config import UserConfig
+from aiperf.common.enums import ConversationContextMode
 from aiperf.common.mixins import AIPerfLoggerMixin
 from aiperf.common.models import Conversation
 from aiperf.common.session_id_generator import SessionIDGenerator
@@ -31,6 +32,15 @@ def __init__(self, *, user_config: UserConfig, **kwargs):
             seed=user_config.input.random_seed
         )
 
+    @classmethod
+    def get_default_context_mode(cls) -> ConversationContextMode | None:
+        """Dataset-level default context mode for conversations without an explicit one.
+
+        Override in subclasses when the dataset format implies a specific mode.
+        Returns None to fall through to the global ACCUMULATE_ALL default.
+        """
+        return None
+
     @abstractmethod
     def load_dataset(self) -> dict[str, list[CustomDatasetT]]: ...
 

src/aiperf/dataset/loader/base_trace_loader.py

@@ -6,6 +6,7 @@
 
 from aiperf.common.config.config_defaults import InputTokensDefaults
 from aiperf.common.config.user_config import UserConfig
+from aiperf.common.enums import ConversationContextMode
 from aiperf.common.models import Conversation, Text, Turn
 from aiperf.dataset.generator.parallel_decode import parallel_decode
 from aiperf.dataset.generator.prompt import PromptGenerator
@@ -213,6 +214,16 @@ def _get_text_input(self, trace: TraceT) -> str | None:
         """
         return getattr(trace, "text_input", None)
 
+    def _infer_context_mode(
+        self, traces: list[TraceT]
+    ) -> ConversationContextMode | None:
+        """Infer context_mode from trace data when not explicitly set.
+
+        Override in subclasses to auto-detect based on trace content.
+        Default returns None (falls through to global ACCUMULATE_ALL default).
+        """
+        return None
+
     def _build_turn(self, trace: TraceT, prompt: str) -> Turn:
         """Build a :class:`Turn` from trace data and a generated prompt.
 
@@ -292,7 +303,12 @@ def convert_to_conversations(
         # Phase 3: Build final conversation objects
         conversations: list[Conversation] = []
         for session_id, trace_prompt_pairs in conversations_data.items():
-            conversation = Conversation(session_id=session_id)
+            traces_in_session = [trace for trace, _ in trace_prompt_pairs]
+            context_mode = self._infer_context_mode(traces_in_session)
+
+            conversation = Conversation(
+                session_id=session_id, context_mode=context_mode
+            )
             for trace, prompt in trace_prompt_pairs:
                 conversation.turns.append(self._build_turn(trace, prompt))
             conversations.append(conversation)