feat: add data_model parameter to .to_solver() for structured output in evals (#264)

* feat: add data_model parameter to to_solver() for structured output in evals

When a data_model (Pydantic model) is provided to .to_solver(), the solver
uses .chat_structured_async() instead of .chat_async() to generate responses.
The resulting Pydantic model instance is serialized to JSON and set as the
completion text in state.output.completion.

This allows using chatlas for structured data extraction tasks in Inspect AI
evaluations, where scorers can parse and validate the JSON output.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>

* docs: add PR number to CHANGELOG entry

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>

* refactor: consolidate completion logic in to_solver()

Move all completion text determination to one place, right before
setting state.output.

🤖 Generated with [Claude Code](https://claude.com/claude-code)

Co-Authored-By: Claude Opus 4.5 <[email protected]>

* Apply suggestions from code review

* Update chatlas/_chat.py

---------

Co-authored-by: Claude Opus 4.5 <[email protected]>

Author: cpsievert

CHANGELOG.md

@@ -12,6 +12,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 ### New features
 
 * `.stream()` and `.stream_async()` now support a `data_model` parameter for structured data extraction while streaming. (#262)
+* `.to_solver()` now supports a `data_model` parameter for structured data extraction in evals. When provided, the solver uses `.chat_structured()` instead of `.chat()` and outputs JSON-serialized data. (#264)
 
 ## [0.15.0] - 2026-01-06
 

chatlas/_chat.py

@@ -836,6 +836,7 @@ def to_solver(
         *,
         include_system_prompt: bool = False,
         include_turns: bool = False,
+        data_model: type[BaseModel] | None = None,
     ):
         """
         Create an InspectAI solver from this chat.
@@ -847,6 +848,11 @@ def to_solver(
 
         Parameters
         ----------
+        data_model
+            A Pydantic model describing the structure of the data to extract.
+            When provided, the solver will use `.chat_structured()` instead of
+            `.chat()` to generate responses, and the output completion will be
+            JSON serialized from the model instance.
         include_system_prompt
             Whether to include the system prompt in the solver's starting
             messages.
@@ -977,8 +983,14 @@ async def solve(state: InspectTaskState, generate):
                     input_content = [input_content]
                 input_content = [inspect_content_as_chatlas(x) for x in input_content]
 
-                # Generate the response (this can generate multiple turns!)
-                await chat_instance.chat_async(*input_content, echo="none")
+                # Generate the response
+                structured_result: BaseModel | None = None
+                if data_model is not None:
+                    structured_result = await chat_instance.chat_structured_async(
+                        *input_content, data_model=data_model, echo="none"
+                    )
+                else:
+                    await chat_instance.chat_async(*input_content, echo="none")
 
                 # Map change in chatlas Turn state back to Inspect message.state
                 # (Note: we skip the user prompt turn since it's already included)
@@ -1001,10 +1013,15 @@ async def solve(state: InspectTaskState, generate):
                         "Expected the last message in InspectAI state to be an assistant message"
                     )
 
+                if structured_result is not None:
+                    completion = structured_result.model_dump_json()
+                else:
+                    completion = turns[-1].text
+
                 state.output = imodel.ModelOutput(
                     model=model,
                     choices=[imodel.ChatCompletionChoice(message=last_message)],
-                    completion=turns[-1].text,
+                    completion=completion,
                     usage=usage,
                     time=time.perf_counter() - start_time,
                 )