Skip to content

Commit c2d281e

Browse files
authored
chore(replay): improve replay robustness with un-validated construction (#3414)
# What does this PR do? some providers do not produce spec compliant outputs. when this happens the replay infra will fail to construct the proper types and will return a dict to the client. the client likely does not expect a dict. this was discovered with tgi, which returns finish_reason="" when valid values are "stop", "length" or "content_filter" ## Test Plan ci
1 parent 2838d5a commit c2d281e

File tree

1 file changed

+6
-2
lines changed

1 file changed

+6
-2
lines changed

llama_stack/testing/inference_recorder.py

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -105,8 +105,12 @@ def _deserialize_response(data: dict[str, Any]) -> Any:
105105

106106
return cls.model_validate(data["__data__"])
107107
except (ImportError, AttributeError, TypeError, ValueError) as e:
108-
logger.warning(f"Failed to deserialize object of type {data['__type__']}: {e}")
109-
return data["__data__"]
108+
logger.warning(f"Failed to deserialize object of type {data['__type__']} with model_validate: {e}")
109+
try:
110+
return cls.model_construct(**data["__data__"])
111+
except Exception as e:
112+
logger.warning(f"Failed to deserialize object of type {data['__type__']} with model_construct: {e}")
113+
return data["__data__"]
110114

111115
return data
112116

0 commit comments

Comments
 (0)