Skip to content

Commit 00b0b6c

Browse files
committed
Align SDK 0.1.5 with TRACE 0.5.x runtime + handler
This is the alignment audit follow-up: the SDK now matches the canonical TRACE wire schema and the RunPod handler envelope field-for-field on every shape that production callers consume (groundedness scoring, redaction, compression, memory, rollup). Key fixes that callers see immediately: * RunPod handler error envelope ({error_code, error, hint, status_code, docs_url}) is now decoded into the structured _Envelope used by LatenceTraceAPIError. Prior versions only matched {code, message} and silently degraded RunPod failures to "HTTP {status}". * GroundednessRequest accepts `language` as the canonical field (with `locale` as a deprecated alias). The server reads `language` to pick the per-class calibration bundle; older SDK calls that set `locale` were silently dropped on the server. * RunPod transports now default response_format=canonical so the typed GroundednessResponse populates nested scores / runtime_decision blocks instead of receiving the flat compact dict (which left most typed fields at None). * score_groundedness exposes profile, response_format, include_triangular_diagnostics, evidence_limit, heatmap_format, auto_decide as first-class kwargs alongside the existing extra fall-through. * rollup() normalizes the RunPod {success, action, rollup: {...}} envelope into the same metrics dict REST returns, and gains an as_model=True flag returning the new typed RollupResponse. Typed model parity bumps: * GroundednessScores gains nli_aggregate, primary_score/name, reverse_context*, semantic_entropy_*, literal_*, structured_*, context_*_ratio, support_units_usage_*, and the AST/phantom channels emitted by the code lane. * GroundednessResponse adds top-level band/score/groundedness_v2, corpus_route, heatmap, warnings, scoring_mode, profile, effective_profile, session_id, *_diagnostics, and the next_session_state / next_memory_state lanes. * RuntimeDecision adds band, policy_version/policy_sha256, head_registry_sha256/head_version, reason_codes, evidence, unsupported_spans, allow/block_disabled, allow/block_threshold, rollback_safe -- covering the full RuntimeDecisionRecord, including the new calibration_band_coercion reason code. * MemoryUpdateResponse.hot_context becomes required (server contract); CompressionResponse.provider stays Optional[str]. * RollupResponse mirrors the FastAPI shape so as_model=True works for both REST and RunPod transports. Helper updates: * GroundednessResponse.fill_native_risk_band promotes risk_band from scores.risk_band -> top-level band -> runtime_decision.band (in that priority order), matching the new backend coercion. * integrations._band_utils.resolve_band/resolve_score prefer the calibration risk_band / trace_score over the head-driven runtime_decision.band / decision.score, so adapter consumers render the same band the event log shows. Deferred to a separate release: typed RunPod action mappings for the nine /v1/trace/sessions* paths. Those require typed session models and live outside the groundedness/rollup hot path. Bumps version 0.1.4 -> 0.1.5 (pyproject.toml + DEFAULT_USER_AGENT). All 41 unit tests green (27 existing + 14 new alignment regressions).
1 parent 36b1380 commit 00b0b6c

9 files changed

Lines changed: 1113 additions & 140 deletions

File tree

pyproject.toml

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@ build-backend = "hatchling.build"
44

55
[project]
66
name = "latence"
7-
version = "0.1.4"
7+
version = "0.1.5"
88
description = "Thin Python SDK for Latence TRACE."
99
readme = "README.md"
1010
requires-python = ">=3.10"

src/latence/__init__.py

Lines changed: 15 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -32,12 +32,19 @@
3232
ComplianceRedactionResponse,
3333
ComplianceUsage,
3434
CompressionResponse,
35+
CorpusRoute,
3536
GroundednessRequest,
3637
GroundednessResponse,
38+
GroundednessScores,
3739
MemoryUpdateResponse,
3840
NLIVerdict,
3941
RiskBand,
42+
RollupDriftTrend,
43+
RollupResponse,
44+
RollupTopDeadFile,
4045
RuntimeDecision,
46+
RuntimeEvidenceUnit,
47+
RuntimeUnsupportedSpan,
4148
SupportUnit,
4249
TokenScore,
4350
)
@@ -49,7 +56,7 @@
4956
TraceSessionSnapshot,
5057
)
5158

52-
__version__ = "0.1.4"
59+
__version__ = "0.1.5"
5360

5461
__all__ = [
5562
"AsyncLatence",
@@ -64,9 +71,11 @@
6471
"ComplianceRedactionResponse",
6572
"ComplianceUsage",
6673
"CompressionResponse",
74+
"CorpusRoute",
6775
"FileSessionStorage",
6876
"GroundednessRequest",
6977
"GroundednessResponse",
78+
"GroundednessScores",
7079
"InMemorySessionStorage",
7180
"Latence",
7281
"LatenceTraceAPIError",
@@ -79,7 +88,12 @@
7988
"MemoryUpdateResponse",
8089
"NLIVerdict",
8190
"RiskBand",
91+
"RollupDriftTrend",
92+
"RollupResponse",
93+
"RollupTopDeadFile",
8294
"RuntimeDecision",
95+
"RuntimeEvidenceUnit",
96+
"RuntimeUnsupportedSpan",
8397
"SessionStorage",
8498
"SupportUnit",
8599
"TokenScore",

src/latence/_transport.py

Lines changed: 60 additions & 11 deletions
Original file line numberDiff line numberDiff line change
@@ -25,7 +25,7 @@
2525
_Envelope,
2626
)
2727

28-
DEFAULT_USER_AGENT = "latence/0.1.4"
28+
DEFAULT_USER_AGENT = "latence/0.1.5"
2929
DEFAULT_TIMEOUT_SECONDS = 30.0
3030
DEFAULT_MAX_RETRIES = 4
3131
RETRYABLE_STATUS = frozenset({429, 500, 502, 503, 504})
@@ -106,7 +106,24 @@ def parse_retry_after(value: str | None) -> float | None:
106106

107107
def decode_error(status: int, body: Any, request_id: str | None) -> LatenceTraceAPIError:
108108
envelope = _envelope_from_body(body)
109-
message = envelope.message if envelope and envelope.message else f"HTTP {status}"
109+
if envelope and envelope.message:
110+
message = envelope.message
111+
elif status == 422 and isinstance(body, Mapping) and isinstance(body.get("detail"), list):
112+
# FastAPI validator errors — the detail is a list of error dicts.
113+
# Surface a useful message instead of the generic ``HTTP 422``.
114+
details = body["detail"]
115+
head = details[0] if details else None
116+
if isinstance(head, Mapping):
117+
loc = head.get("loc")
118+
msg = head.get("msg") or "validation error"
119+
location = ".".join(str(item) for item in loc) if isinstance(loc, list) else None
120+
message = (
121+
f"validation error at {location}: {msg}" if location else f"validation error: {msg}"
122+
)
123+
else:
124+
message = f"validation error: {details!r}"
125+
else:
126+
message = f"HTTP {status}"
110127
if status in (401, 402, 403):
111128
return LatenceTraceAuthError(
112129
message,
@@ -138,27 +155,59 @@ def decode_error(status: int, body: Any, request_id: str | None) -> LatenceTrace
138155
return LatenceTraceAPIError(message, status=status, envelope=envelope, request_id=request_id)
139156

140157

158+
_ERROR_ENVELOPE_RESERVED = frozenset(
159+
{"code", "message", "hint", "docs_url", "error_code", "error", "status_code"}
160+
)
161+
162+
141163
def _envelope_from_body(body: Any) -> _Envelope | None:
164+
"""Coerce a server error body into a structured :class:`_Envelope`.
165+
166+
Recognises three wire shapes:
167+
168+
1. FastAPI ``{"detail": {"code": ..., "message": ..., "hint": ...}}``
169+
— the canonical REST error envelope.
170+
2. RunPod handler ``{"success": false, "error_code": ..., "error":
171+
..., "hint": ..., "status_code": ..., "docs_url": ...}`` —
172+
emitted by ``runpod/handler.py::_service_error_payload`` when a
173+
job fails. The handler does not use ``code``/``message``;
174+
prior SDK versions would silently fall through to the
175+
``HTTP {status}`` placeholder.
176+
3. FastAPI ``{"detail": "<plain string>"}`` — degraded path; we
177+
keep ``code=None`` and stuff the string into ``message`` so
178+
callers still get a useful description.
179+
"""
180+
142181
if not isinstance(body, Mapping):
143182
return None
144183
detail = body.get("detail")
145184
if isinstance(detail, Mapping):
146185
body = detail
186+
elif isinstance(detail, str) and detail:
187+
return _Envelope(code="error", message=detail, hint=None, docs_url=None, extra={})
147188
if not isinstance(body, Mapping):
148189
return None
149-
code = body.get("code")
150-
if not isinstance(code, str):
190+
191+
# Accept either ``code`` (REST) or ``error_code`` (RunPod handler).
192+
code_value = body.get("code")
193+
if not isinstance(code_value, str):
194+
code_value = body.get("error_code")
195+
# Accept either ``message`` (REST) or ``error`` (RunPod handler).
196+
message_value = body.get("message")
197+
if not isinstance(message_value, str) or not message_value:
198+
message_value = body.get("error")
199+
200+
# Bail when neither code nor message is present — there is no
201+
# structured envelope to surface.
202+
if not isinstance(code_value, str) and not isinstance(message_value, str):
151203
return None
204+
152205
return _Envelope(
153-
code=code,
154-
message=str(body.get("message") or ""),
206+
code=str(code_value) if isinstance(code_value, str) else "error",
207+
message=str(message_value) if isinstance(message_value, str) else "",
155208
hint=body.get("hint") if isinstance(body.get("hint"), str) else None,
156209
docs_url=body.get("docs_url") if isinstance(body.get("docs_url"), str) else None,
157-
extra={
158-
k: v
159-
for k, v in body.items()
160-
if k not in {"code", "message", "hint", "docs_url"}
161-
},
210+
extra={k: v for k, v in body.items() if k not in _ERROR_ENVELOPE_RESERVED},
162211
)
163212

164213

src/latence/async_client.py

Lines changed: 60 additions & 39 deletions
Original file line numberDiff line numberDiff line change
@@ -26,6 +26,7 @@
2626
runpod_request_body,
2727
unwrap_runpod_response,
2828
)
29+
from latence.client import _build_groundedness_payload, _normalize_rollup_body
2930
from latence.errors import (
3031
LatenceTraceAPIError,
3132
LatenceTraceRateLimited,
@@ -41,9 +42,9 @@
4142
ComplianceRedactionRequest,
4243
ComplianceRedactionResponse,
4344
CompressionResponse,
44-
GroundednessRequest,
4545
GroundednessResponse,
4646
MemoryUpdateResponse,
47+
RollupResponse,
4748
SupportUnit,
4849
)
4950
from latence.sessions import (
@@ -344,10 +345,18 @@ async def score_groundedness(
344345
coverage_threshold: float | None = None,
345346
raw_context_chunk_tokens: int | None = None,
346347
response_chunk_tokens: int | None = None,
348+
language: str | None = None,
347349
locale: str | None = None,
348350
context_trust_enabled: bool = True,
349351
runtime_head_features: Mapping[str, float] | None = None,
350352
trajectory_features: Mapping[str, float] | None = None,
353+
profile: str | None = None,
354+
scoring_mode: str | None = None,
355+
response_format: str | None = None,
356+
include_triangular_diagnostics: bool | None = None,
357+
evidence_limit: int | None = None,
358+
heatmap_format: str | None = None,
359+
auto_decide: bool | None = None,
351360
extra: Mapping[str, Any] | None = None,
352361
) -> GroundednessResponse:
353362
payload = self._build_payload(
@@ -361,10 +370,18 @@ async def score_groundedness(
361370
coverage_threshold=coverage_threshold,
362371
raw_context_chunk_tokens=raw_context_chunk_tokens,
363372
response_chunk_tokens=response_chunk_tokens,
373+
language=language,
364374
locale=locale,
365375
context_trust_enabled=context_trust_enabled,
366376
runtime_head_features=runtime_head_features,
367377
trajectory_features=trajectory_features,
378+
profile=profile,
379+
scoring_mode=scoring_mode,
380+
response_format=response_format,
381+
include_triangular_diagnostics=include_triangular_diagnostics,
382+
evidence_limit=evidence_limit,
383+
heatmap_format=heatmap_format,
384+
auto_decide=auto_decide,
368385
extra=extra,
369386
)
370387
return await self._request(
@@ -428,14 +445,20 @@ async def redact_compliance(
428445
async def rollup(
429446
self,
430447
turns: Sequence[Mapping[str, Any]],
448+
*,
449+
as_model: bool = False,
431450
**options: Any,
432-
) -> Mapping[str, Any]:
433-
return await self._request(
451+
) -> RollupResponse | Mapping[str, Any]:
452+
body = await self._request(
434453
"POST",
435454
"/groundedness/rollup",
436455
json={"turns": list(turns), **options},
437456
expected_model=None,
438457
)
458+
normalized = _normalize_rollup_body(body)
459+
if as_model:
460+
return RollupResponse.model_validate({**normalized, "raw": body})
461+
return normalized
439462

440463
def session(
441464
self,
@@ -470,48 +493,46 @@ def _build_payload(
470493
coverage_threshold: float | None,
471494
raw_context_chunk_tokens: int | None,
472495
response_chunk_tokens: int | None,
496+
language: str | None,
473497
locale: str | None,
474498
context_trust_enabled: bool,
475499
runtime_head_features: Mapping[str, float] | None,
476500
trajectory_features: Mapping[str, float] | None,
501+
profile: str | None,
502+
scoring_mode: str | None,
503+
response_format: str | None,
504+
include_triangular_diagnostics: bool | None,
505+
evidence_limit: int | None,
506+
heatmap_format: str | None,
507+
auto_decide: bool | None,
477508
extra: Mapping[str, Any] | None,
478509
) -> dict:
479-
normalised_units: list[dict] | None = None
480-
if support_units:
481-
normalised_units = [
482-
u.model_dump(exclude_none=True) if isinstance(u, SupportUnit) else dict(u)
483-
for u in support_units
484-
]
485-
try:
486-
req = GroundednessRequest(
487-
query_text=query,
488-
response_text=response_text,
489-
chunk_ids=list(chunk_ids) if chunk_ids else None,
490-
raw_context=_coerce_raw_context(raw_context),
491-
support_units=(
492-
[SupportUnit(**u) for u in normalised_units]
493-
if normalised_units
494-
else None
495-
),
496-
attribution_mode=attribution_mode,
497-
primary_metric=primary_metric,
498-
coverage_threshold=coverage_threshold,
499-
raw_context_chunk_tokens=raw_context_chunk_tokens,
500-
response_chunk_tokens=response_chunk_tokens,
501-
locale=locale,
502-
context_trust_enabled=context_trust_enabled,
503-
runtime_head_features=runtime_head_features,
504-
trajectory_features=trajectory_features,
505-
)
506-
except ValidationError as exc:
507-
raise LatenceTraceValidationError(
508-
f"client-side request validation failed: {exc.errors()[:3]}",
509-
status=422,
510-
) from exc
511-
body = req.model_dump(mode="json", exclude_none=True)
512-
if extra:
513-
body.update(dict(extra))
514-
return body
510+
return _build_groundedness_payload(
511+
runpod=self._runpod,
512+
response_text=response_text,
513+
query=query,
514+
chunk_ids=chunk_ids,
515+
raw_context=raw_context,
516+
support_units=support_units,
517+
attribution_mode=attribution_mode,
518+
primary_metric=primary_metric,
519+
coverage_threshold=coverage_threshold,
520+
raw_context_chunk_tokens=raw_context_chunk_tokens,
521+
response_chunk_tokens=response_chunk_tokens,
522+
language=language,
523+
locale=locale,
524+
context_trust_enabled=context_trust_enabled,
525+
runtime_head_features=runtime_head_features,
526+
trajectory_features=trajectory_features,
527+
profile=profile,
528+
scoring_mode=scoring_mode,
529+
response_format=response_format,
530+
include_triangular_diagnostics=include_triangular_diagnostics,
531+
evidence_limit=evidence_limit,
532+
heatmap_format=heatmap_format,
533+
auto_decide=auto_decide,
534+
extra=extra,
535+
)
515536

516537
async def _request(
517538
self,

0 commit comments

Comments
 (0)