feat(ai): systemPrompts accept { content, metadata } for per-provider metadata#575
Merged
Conversation
… bump vite - Exclude testing/** alongside examples/** in build, build:all, test:ci, test:pr, test:lib, test:types, test:eslint, test:build, test:coverage. Internal test harnesses (testing/e2e, testing/panel) had build scripts but no exclude, so they were getting built on every release pipeline. - Add an `ignore` list to .changeset/config.json covering every in-workspace private package. Defends against accidental publication if `"private": true` is ever dropped from a package by mistake. - Bump vite from ^7.2.7 to ^7.3.3 across every declaring package.json. Resolves a transitive version skew (some dev-only plugins pull vite@7.3.x) that was breaking typechecking of vite.config.ts files. - Add `engines` to the root package.json: `node >=24`, `pnpm >=11.0.0`.
…ests - Introduce tsconfig.base.json at the repo root with the shared compilerOptions. Root tsconfig.json now extends it for the small set of root-level files (scripts, config files) we still typecheck. - Migrate every package tsconfig to extend tsconfig.base.json with a consistent shape: only the package's unique compilerOptions overrides (outDir, rootDir where needed, JSX runtime, framework lib) plus a uniform `include: ["src", "tests"]` / `exclude: ["node_modules", "dist"]`. - Drop brittle one-off patterns: per-package duplicated compilerOptions (ai-devtools, ai-isolate-cloudflare), explicit single-test-file includes (ai-anthropic, ai-gemini), and `**/*.config.ts` excludes that were silently hiding test files from `tsc`. - Config files (vite.config.ts, vitest.config.ts) are no longer in the include set. They're typechecked at build time by vite/vitest themselves and the cross-tool type matrix makes them brittle to check via `tsc`. Net effect: `pnpm test:types` now actually typechecks the tests in every package. Prior to this commit several packages were silently excluding tests from the typecheck pipeline.
…interface The old shape kept thinking config in two separate interfaces — one with required `includeThoughts` + optional `thinkingBudget`, the other with `thinkingLevel` — and intersected both into ExternalTextProviderOptions. The intersection produced a `thinkingConfig` shape where neither `includeThoughts`+`thinkingBudget` nor `thinkingLevel` could be passed cleanly, even though the adapter reads both at runtime (see src/adapters/text.ts mapCommonOptionsToGemini). Merge into a single GeminiThinkingOptions with all three fields optional. Drop GeminiThinkingAdvancedOptions entirely. Update model-meta intersections and the sync-provider-models template that references the type. GeminiThinkingOptions is the only thinking type exported from @tanstack/ai-gemini, so no public-API breakage. GeminiThinkingAdvancedOptions was never exported.
`system` was sneaking into modelOptions via `validKeys` and getting spread
over the system block constructed from systemPrompts. That was a leaky
abstraction: it bypassed the cross-provider `systemPrompts` API and forced
users to construct Anthropic's TextBlockParam shape manually to attach
features like cache_control.
- Remove `'system'` from validKeys in adapters/text.ts. The adapter no longer
accepts a system override from modelOptions.
- Keep `system?: string | Array<TextBlockParam>` on InternalTextProviderOptions
(the adapter still constructs it internally from `options.systemPrompts`).
Mark it explicitly as internal in the JSDoc.
- Delete the test that exercised the override behaviour and prune the
`system` field + `& { system: string }` satisfies intersection from the
one other test that was using modelOptions.system.
Regression to be aware of: there is currently no public way to attach
Anthropic `cache_control` to system prompts. A follow-up will extend
`systemPrompts` to accept `{ content, metadata }` with provider-specific
metadata (cache_control for Anthropic, others later).
`Logger` was declared as an `export interface` in src/logger/types.ts but wasn't re-exported from src/adapter-internals.ts. Provider adapter packages consume internals only via the `@tanstack/ai/adapter-internals` subpath, so without this re-export they couldn't reach the `Logger` type without an ambient import that would break the encapsulation boundary. Surfaced by the openai-base tests once `tsc` started checking tests.
…tion
Including tests in `tsc` (see the tsconfig commit) surfaced ~200 pre-existing
type errors across the test suite that had silently rotted as source types
evolved. This commit fixes them and tightens the remaining type assertions.
Patterns fixed:
- Stale model name literals (`claude-3-7-sonnet-20250219` →
`claude-3-7-sonnet`, `grok-4-0709` → `grok-4`, `chatgpt-4.0` →
`chatgpt-4o-latest`, etc.).
- EventType string literals replaced with the enum values from
`@tanstack/ai`/`@ag-ui/core` in groq/grok/openai-base/openai tests.
- Removed/renamed provider option fields (`generationConfig.*` flattened
onto top-level Gemini options; `system` removed from anthropic
modelOptions tests).
- Required fields added to mock generation result objects (`id` on
image/speech/transcription mocks, `id` + `usage` on summarize mocks)
to satisfy GenerationFetcher result types.
- `MakeInputModalitiesTypes<['text', 'image', ...]>` wrappers and
provider-typed content parts (`AnthropicTextPart`, `OpenAITextPart`,
`GeminiImagePart`, etc.) plumbed into the model-meta tests.
- `tools` array entries replaced with typed `AnyClientTool` factories
for the elevenlabs realtime tests.
- Cloudflare worker tests: 7 `as any` casts collapsed into a single
`readJson<T>` helper with one trust-boundary cast and `Extract<>` types
per response status arm.
- ai-react/ai-solid `useChat({ onToolCall })` tests: `onToolCall` is not
a public UseChatOptions field — it's set internally by ChatClient via
the `tools` array. Deleted one test of nonexistent behaviour and
removed dead `onToolCall` options from two `addToolResult` tests.
- ai-react/ai-solid `useGeneration` tests: explicit generic arguments
instead of `onResult ... as any`; `as const` on inline chunk arrays so
StreamChunk literals infer with EventType enum values intact.
- ai-svelte `create-generation` tests: EventType enum + required fields
on AGUIEvent shapes (`threadId` on RUN_STARTED/RUN_FINISHED, `message`
on RUN_ERROR).
- openai-base tests now build a real `new OpenAI({ apiKey })` and
monkey-patch a typed `MockChatCompletionCreate` / `MockResponsesCreate`
signature instead of `as unknown as OpenAI`.
Remaining `as`-style casts in the touched files: 5 total, all at trust
boundaries with justification comments (Response.json() → typed shape;
discriminated-union narrowing for the schemaless useChat branch; AGUI
RUN_ERROR carrying both spec-shape `message` and legacy `error.message`).
`@ts-expect-error` is used in one place (audio-adapter.test.ts) for the
"unsupported model name is rejected" test, replacing an `as never` cast.
No production code changes other than:
- A new export of an existing `Logger` interface (separate commit).
- The anthropic modelOptions.system removal (separate commit).
…ndations - CONTRIBUTING.md covering prereqs (Node 24, pnpm 11), initial setup, repo layout, day-to-day commands, the per-package tsconfig pattern, where to add unit tests, the (mandatory) E2E coverage matrix, changesets, PR flow, and how to add a new provider adapter. Documents the known gaps: .vue/.svelte SFCs are not linted today, and build configs are not in the `tsc` pass. - .editorconfig with the repo's existing conventions (LF, utf-8, 2-space, final newline). - .vscode/extensions.json recommending eslint, prettier, vitest explorer, nx, svelte, vue, editorconfig. The PR template already linked to a missing CONTRIBUTING.md; this fixes that broken link.
… isolated-vm) The previous "Node 24+" claim was overreaching: - Only `@tanstack/ai-isolate-node` (via `isolated-vm`) has a Node version floor. Everything else in the workspace runs fine on older Node. - `isolated-vm`'s actual `engines.node` is `>=22.0.0` (see https://github.com/laverdet/isolated-vm/blob/main/package.json), not 24. Changes: - Root package.json: drop the `node: ">=24"` entry from engines (keep the pnpm constraint). No global Node floor for development. - packages/typescript/ai-isolate-node/package.json: bump `engines.node` from `>=18` to `>=22` to match upstream. - packages/typescript/ai-isolate-node/README.md: same — say `>=22`, and add the `--no-node-snapshot` flag note for Node 20+ runtime usage.
- package.json: relax engines.pnpm from `>=11.0.0` to `>=10.17.0` to match
CONTRIBUTING.md ("pnpm 10.17.0 or newer"). packageManager still pins
pnpm@11.1.1 for corepack users.
- packages/typescript/ai-isolate-node/README.md: rewrite the
`--no-node-snapshot` bullet so it no longer says "Node.js 20.x and later"
alongside the "Node.js >= 22" minimum stated two bullets above. The flag
is required by isolated-vm on every Node version we support.
- packages/typescript/ai-gemini/tests/model-meta.test.ts: the
"thinking models should allow thinkingConfig" type-test was asserting
`stopSequences` (a placeholder swapped in during the typecheck rot
cleanup). Restore the original intent: assert that `thinkingConfig` is
present on `GeminiChatModelProviderOptionsByName['gemini-2.5-pro']`.
- packages/typescript/ai-openai/tests/openai-adapter.test.ts: the test
passes `systemPrompts: ['Stay concise']` but never checked that it
arrived on the outbound payload. Add the missing
`instructions: 'Stay concise'` assertion (the Responses API field the
adapter writes `systemPrompts.join('\n')` into).
Skipped:
- packages/typescript/ai-vue-ui/tsconfig.json (and sibling -ui packages):
CodeRabbit suggested adding `"tests"` to `include`. The *-ui packages
have no test files on disk and use `composite: true` with `rootDir: src`,
so widening include would point at a non-existent directory.
Contributor
📝 WalkthroughWalkthroughThe pull request extends the ChangesSystem Prompts Metadata & Adapter Support
Estimated code review effort🎯 4 (Complex) | ⏱️ ~60 minutes Possibly related PRs
Suggested reviewers
Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
✏️ Tip: You can configure your own custom pre-merge checks in the settings. ✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
Contributor
🚀 Changeset Version Preview8 package(s) bumped directly, 22 bumped as dependents. 🟥 Major bumps
🟨 Minor bumps
🟩 Patch bumps
|
|
View your CI Pipeline Execution ↗ for commit 7a58420
☁️ Nx Cloud last updated this comment at |
@tanstack/ai
@tanstack/ai-anthropic
@tanstack/ai-client
@tanstack/ai-code-mode
@tanstack/ai-code-mode-skills
@tanstack/ai-devtools-core
@tanstack/ai-elevenlabs
@tanstack/ai-event-client
@tanstack/ai-fal
@tanstack/ai-gemini
@tanstack/ai-grok
@tanstack/ai-groq
@tanstack/ai-isolate-cloudflare
@tanstack/ai-isolate-node
@tanstack/ai-isolate-quickjs
@tanstack/ai-ollama
@tanstack/ai-openai
@tanstack/ai-openrouter
@tanstack/ai-preact
@tanstack/ai-react
@tanstack/ai-react-ui
@tanstack/ai-solid
@tanstack/ai-solid-ui
@tanstack/ai-svelte
@tanstack/ai-utils
@tanstack/ai-vue
@tanstack/ai-vue-ui
@tanstack/openai-base
@tanstack/preact-ai-devtools
@tanstack/react-ai-devtools
@tanstack/solid-ai-devtools
commit: |
…erred metadata typing
Extends `chat({ systemPrompts })` to accept either a plain string (existing
shape — backward compatible) or `{ content, metadata }`. The structured
form's `metadata` type is inferred from the adapter at the chat() call site
via a new `TSystemPromptMetadata` generic on `TextAdapter` — no `satisfies`
needed by callers.
- Anthropic declares `AnthropicSystemPromptMetadata` → users get
`cache_control` autocomplete + type-checking.
- Adapters with no per-prompt metadata (OpenAI, Gemini, Ollama,
OpenRouter, openai-base) inherit the default `never`, which makes the
`metadata` field unusable at the call site. Passing metadata to those
adapters is a TypeScript error.
Closes the regression introduced by removing the `modelOptions.system`
escape hatch in the audit PR — there is now a public, typed path for
attaching Anthropic `cache_control` to system prompts.
Plumbing
- New `TSystemPromptMetadata = never` generic on `TextAdapter` /
`BaseTextAdapter`, surfaced via `'~types'['systemPromptMetadata']`.
`TextActivityOptions.systemPrompts` is now
`Array<SystemPrompt<TAdapter['~types']['systemPromptMetadata']>>`.
- `AnyTextAdapter` extended to 7 generic slots.
- `TextOptions.systemPrompts` (the wide internal shape adapters receive)
is `Array<SystemPrompt>`; adapters call `normalizeSystemPrompts<...>()`
to narrow.
- Chat engine + middleware context/config widened to carry
`Array<SystemPrompt>` so metadata flows through to the adapter.
- OpenTelemetry middleware extracts `.content` for span events;
per-prompt metadata is dropped from spans.
- `@tanstack/ai-event-client` mirrors the `SystemPrompt` shape locally
(avoids a circular import with `@tanstack/ai`) and projects metadata
away on the devtools wire.
Adapter mappings
- Anthropic reads `metadata.cache_control` and attaches it to the
matching `TextBlockParam`.
- OpenAI / Gemini / Ollama / OpenRouter / openai-base call
`normalizeSystemPrompts()` and join `.content` for their respective
`instructions` / `system` / `systemInstruction` fields. (Their
metadata type is `never`, so the field can't be set anyway.)
Tests
- ai-anthropic: new test verifies `cache_control` flows from
`systemPrompts[i].metadata` onto the outbound `TextBlockParam`, and
plain-string entries still produce metadata-less blocks.
- ai-openai: new test verifies mixed string + object-form input
(without metadata) produces the expected joined `instructions`.
AlemTuzlak
force-pushed
the
feat/system-prompts-metadata
branch
from
2cfda79 to
8dc3dd7
Compare
Contributor
tombeckenham
left a comment
tombeckenham
left a comment
There was a problem hiding this comment.
PR Review — feat/system-prompts-metadata
Scope: 25 files, +410/-46, reviewed against chore/monorepo-setup-audit (stacked PR). Type plumbing verified, all unit tests pass, Array<string> backward compatibility confirmed.
Critical (1)
C1 — normalizeSystemPrompts does zero runtime validation
packages/typescript/ai/src/system-prompts.ts:64-68. The cast p as NormalizedSystemPrompt<TMetadata> trusts the TS boundary. { metadata: {…} } (missing content) or { content: undefined as any } passes through; every adapter then does prompts.map((p) => p.content).join('\n') and produces a literal "undefined" in the system prompt — no log, no throw, no telemetry. Fix: when the entry is object-form, validate typeof p.content === 'string'; throw with the offending index. This is a public API boundary.
Important (6)
I1 — No E2E coverage; CLAUDE.md mandates it
testing/e2e/ has zero diff. The repo policy requires E2E for behavior changes. testing/e2e/src/routes/api.chat.ts already wires systemPrompts; one Anthropic fixture passing metadata.cache_control would cover the wire path.
I2 — Middleware-context shape widened but only types changed, not behavior
ChatMiddlewareContext.systemPrompts is now Array<SystemPrompt> but every runtime assertion in packages/typescript/ai/tests/middleware.test.ts still uses plain strings. A middleware doing config.systemPrompts.join('\n') would now produce '[object Object]' and no test catches it.
I3 — Per-adapter mapping has no direct coverage for Gemini, Ollama, OpenRouter
Anthropic+OpenAI tests cover the helper indirectly, but per-adapter glue diverges: Gemini joins into systemInstruction (string), Ollama uses spread-or-omit, OpenRouter text.ts:1117-1122 pushes a positional { role: 'system' } message (different shape entirely), OpenRouter responses-text.ts uses yet another SDK. None of these have a direct test.
I4 — OTel middleware drops cache_control from observability
packages/typescript/ai/src/middlewares/otel.ts:374-403 flattens to strings; the fact that cache_control was attached is invisible. For Anthropic prompt-caching users this is the one attribute that explains cache hit/miss. Fix: attach metadata as a tanstack.ai.system_prompt.metadata JSON span attribute when captureContent: true.
I5 — Doc/changeset wording on metadata: never is misleading
Changeset and JSDoc in system-prompts.ts:9-12, types.ts:699-700 claim adapters "silently ignore" foreign metadata. With TSystemPromptMetadata = never default they reject the field at compile time (the adjacent chat/index.ts:112-113 doc correctly says "reject the field entirely"). Also: { content: 'x', metadata: undefined } does typecheck — the "passing it errors at compile time" claim should be tightened to "cannot carry a meaningful value." The @example showing satisfies AnthropicSystemPromptMetadata is contradicted by the test comment saying no satisfies is needed.
I6 — DevtoolsSystemPrompt mirror has no structural-equality guard
packages/typescript/ai-event-client/src/devtools-middleware.ts:15-20 is intentionally duplicated to avoid a circular import. If SystemPrompt gains a third variant, the projection typeof p === 'string' ? p : p.content silently emits undefined. Add a const _check: SystemPrompt = null as unknown as DevtoolsSystemPrompt assertion in a test to force breakage if shapes diverge.
Suggestions (5)
- S1 —
NormalizedSystemPromptis incidental cruft; structurallyExtract<SystemPrompt<T>, object>with a new name. Consider deleting and inlining theExtract. - S2 — Add
Readonly<…>to bothSystemPromptobject arm andNormalizedSystemPrompt. Adapters shouldn't mutate caller-supplied data; cheap, prevents fan-out bugs. - S3 — 7 positional generics on
TextAdapter/BaseTextAdapteris approaching unmanageable. Don't refactor in this PR, but the next addition should consolidate to a singleTConfigobject generic. - S4 —
systemPromptMetadata: undefined as neverin mocks is awkward. Introduce a tinyphantom<T>(): Thelper so the cast lives in one place, and/or add a one-line justifying comment attests/test-utils.ts:175. - S5 — Devtools wire could ship
hasMetadata: booleanper prompt now to allow a future UI badge without a wire-format break.
Strengths
- Clean type chain
~types['systemPromptMetadata']→SystemPrompt<TMetadata>→ call-site narrowing works as advertised. - Backward compat verified:
systemPrompts: ['foo']typechecks and behaves identically on every adapter. - Anthropic test covers the regression most likely to break (
TextBlockParamshape preservation across the rewrite). - IIFE pattern across 6 adapters is consistent and readable; the structural mirror in
ai-event-clientis the right call given the circular-import constraint. - Most new doc comments are genuine WHY/non-obvious (OTel rationale, devtools wire-stability rationale, circular-dep mirror).
- Snake_case
cache_controlis correct for SDK pass-through. - Changeset bump levels are correct.
Recommended Action
- Before merge: fix C1 (runtime validation), address I1 (E2E fixture), tighten I5 (doc/changeset wording).
- Strongly recommended: add direct mapping tests for Gemini/Ollama/OpenRouter (I3) and one middleware-context test with the object form (I2).
- Follow-up acceptable: I4 (OTel attribute), I6 (mirror guard), all suggestions.
Conflicts: - packages/typescript/ai-ollama/src/adapters/text.ts main (#568) moved Ollama system prompts from the `ChatRequest.system` field into a leading `{ role: 'system' }` message in `messages[]` (Ollama's `system` field wasn't being honored). This branch reads via `normalizeSystemPrompts()` to support the new object form. Resolved by keeping main's unshift-into-messages flow and projecting `.content` through `normalizeSystemPrompts()` before joining. - packages/typescript/ai-anthropic/tests/anthropic-adapter.test.ts Two unrelated tests landed at the same insertion point: this branch's `attaches cache_control to system TextBlockParams via systemPrompts metadata` and main's `drops unknown modelOptions keys (e.g. system) and warns via logger.error`. Kept both as independent `it` blocks.
Maps to tombeckenham's review items C1, I1–I6:
- C1 (runtime validation): `normalizeSystemPrompts` throws TypeError naming
the offending index when an object-form entry's content isn't a string,
so stale call sites can't stream a literal "undefined" into the model.
New test file `packages/typescript/ai/tests/system-prompts.test.ts`
covers the happy paths and the throw cases.
- I1 (E2E coverage): `testing/e2e/tests/system-prompt-metadata.spec.ts`
drives the full Anthropic HTTP path with object-form systemPrompts +
metadata.cache_control. Wire-shape assertion stays in the ai-anthropic
unit test (aimock's journal normalises Anthropic into an OpenAI-shaped
request and drops `cache_control`, so a journal-based assertion isn't
reliable — the spec asserts end-to-end success instead). Required a
test-only `systemPromptCacheControl` opt-in on `api.chat.ts` to flip
the system prompt to object form.
- I2 (middleware widened-shape coverage): new
`middleware.test.ts > should preserve object-form systemPrompts through
middleware` asserts the middleware sees `Array<SystemPrompt>`, not a
pre-flattened `Array<string>`, and that mutations preserve metadata
through to the adapter.
- I3 (per-adapter mapping coverage): direct unit tests for
Gemini (`systemInstruction` joined; foreign metadata dropped),
Ollama (`messages.unshift({role:'system'})` joined; foreign metadata
dropped), and OpenRouter (positional `{role:'system'}` message; foreign
metadata dropped).
- I4 (OTel metadata observability): when `captureContent: true` and at
least one entry carries metadata, the iteration span gains a
`tanstack.ai.system_prompt.metadata` JSON attribute (positional
per-prompt array, `null` for plain strings / no metadata). Kept off
span events so the existing one-event-per-message GenAI semconv stays
intact. New tests in `tests/middlewares/otel.test.ts` cover both the
set and the absent path.
- I5 (doc/changeset wording): replaced "silently ignore" with the
accurate "carries no meaningful value … silently dropped, never
written to the wire" framing in `SystemPrompt`'s JSDoc, the
`TextOptions.systemPrompts` JSDoc, and the changeset. Removed the
misleading `satisfies AnthropicSystemPromptMetadata` from the
`@example` (the adapter narrows the metadata field at the call site,
so a `satisfies` cast is contradicted by the tests).
- I6 (devtools structural guard):
`ai-event-client/tests/devtools-middleware-shape.test.ts` re-declares
the local `DevtoolsSystemPrompt` mirror and uses `expectTypeOf` to
assert mutual assignability with `@tanstack/ai`'s `SystemPrompt`. If
the canonical shape gains a third variant the guard breaks at
type-check time, forcing the maintainer to update the mirror in
`devtools-middleware.ts`.
Suggestions S1–S5 deferred per the reviewer's "follow-up acceptable" call.
![coderabbitai[bot]](https://avatars.githubusercontent.com/in/347564?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Actionable comments posted: 2
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@packages/typescript/ai/src/middlewares/otel.ts`:
- Around line 384-393: The JSON.stringify of systemPromptMetadata inside the
safeCall('otel.onConfig', ...) callback can throw and abort the rest of the
onConfig flow (preventing state.iterationCount from incrementing), so wrap the
serialization in a local try-catch: compute a safe string (e.g., try
JSON.stringify(systemPromptMetadata) and on error fall back to a placeholder
like '"<unserializable>"' or an array of nulls) and then call
iterSpan.setAttribute('tanstack.ai.system_prompt.metadata', safeString); ensure
the code references the existing systemPromptMetadata variable and leaves the
rest of the callback (including state.iterationCount += 1) untouched so
iteration state remains consistent.
In `@packages/typescript/ai/src/system-prompts.ts`:
- Around line 70-91: The normalizeSystemPrompts function uses manual typeof/null
checks; replace them with Zod validation at this public boundary: import z from
'zod', define a Zod schema for SystemPrompt (accepting either string or object {
content: string, metadata?: unknown }) and an array wrapper, then validate the
incoming prompts with schema.parse or schema.safeParse inside
normalizeSystemPrompts and map the parsed results to
Array<NormalizedSystemPrompt<TMetadata>>; keep the current error semantics by
throwing on failed validation and ensure you continue to return the typed
NormalizedSystemPrompt<TMetadata> values (referencing normalizeSystemPrompts,
SystemPrompt and NormalizedSystemPrompt).
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: ffd312d6-583d-40c7-b411-bfb275913977
📒 Files selected for processing (33)
.changeset/feat-system-prompts-metadata.mdpackages/typescript/ai-anthropic/src/adapters/text.tspackages/typescript/ai-anthropic/src/index.tspackages/typescript/ai-anthropic/src/text/text-provider-options.tspackages/typescript/ai-anthropic/tests/anthropic-adapter.test.tspackages/typescript/ai-event-client/src/devtools-middleware.tspackages/typescript/ai-event-client/tests/devtools-middleware-shape.test.tspackages/typescript/ai-gemini/src/adapters/text.tspackages/typescript/ai-gemini/tests/gemini-adapter.test.tspackages/typescript/ai-ollama/src/adapters/text.tspackages/typescript/ai-ollama/tests/text-adapter.test.tspackages/typescript/ai-openai/src/adapters/text.tspackages/typescript/ai-openai/tests/openai-adapter.test.tspackages/typescript/ai-openrouter/src/adapters/responses-text.tspackages/typescript/ai-openrouter/src/adapters/text.tspackages/typescript/ai-openrouter/tests/openrouter-adapter.test.tspackages/typescript/ai/src/activities/chat/adapter.tspackages/typescript/ai/src/activities/chat/index.tspackages/typescript/ai/src/activities/chat/middleware/types.tspackages/typescript/ai/src/index.tspackages/typescript/ai/src/middlewares/otel.tspackages/typescript/ai/src/system-prompts.tspackages/typescript/ai/src/types.tspackages/typescript/ai/tests/chat-structured-output-stream.test.tspackages/typescript/ai/tests/middleware.test.tspackages/typescript/ai/tests/middlewares/otel.test.tspackages/typescript/ai/tests/system-prompts.test.tspackages/typescript/ai/tests/test-utils.tspackages/typescript/ai/tests/type-check.test.tspackages/typescript/openai-base/src/adapters/chat-completions-text.tspackages/typescript/openai-base/src/adapters/responses-text.tstesting/e2e/src/routes/api.chat.tstesting/e2e/tests/system-prompt-metadata.spec.ts
Comment on lines
+384
to
+393
| const systemPromptMetadata = config.systemPrompts.map((p) => | ||
| typeof p === 'string' || p.metadata === undefined | ||
| ? null | ||
| : p.metadata, | ||
| ) | ||
| if (systemPromptMetadata.some((m) => m !== null)) { | ||
| iterSpan.setAttribute( | ||
| 'tanstack.ai.system_prompt.metadata', | ||
| JSON.stringify(systemPromptMetadata), | ||
| ) |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🌐 Web query:
In JavaScript, does JSON.stringify throw for BigInt values and circular references?
💡 Result:
Yes. 1) BigInt values: JSON.stringify throws a TypeError when it encounters a BigInt value (unless you provide custom serialization via a replacer / toJSON). [1] Example: JSON.stringify({ x: 2n }) throws “TypeError: BigInt value can't be serialized in JSON”. [1] 2) Circular references: JSON.stringify throws when the object graph contains circular references. [1] MDN describes this as a TypeError for cyclic object values and notes that JSON.stringify “doesn’t try to solve them and fails accordingly.” [2] So, JSON.stringify will throw for BigInt values and will also throw for circular references.
Citations:
- 1: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/JSON/stringify
- 2: https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Errors/Cyclic_object_value
🏁 Script executed:
head -c 50000 packages/typescript/ai/src/middlewares/otel.ts | wc -lRepository: TanStack/ai
Length of output: 59
🏁 Script executed:
wc -l packages/typescript/ai/src/middlewares/otel.tsRepository: TanStack/ai
Length of output: 106
🏁 Script executed:
sed -n '374,454p' packages/typescript/ai/src/middlewares/otel.tsRepository: TanStack/ai
Length of output: 3784
🏁 Script executed:
# Find the safeCall function definition
rg -A 10 'const safeCall|function safeCall' packages/typescript/ai/src/middlewares/otel.tsRepository: TanStack/ai
Length of output: 488
🏁 Script executed:
sed -n '360,380p' packages/typescript/ai/src/middlewares/otel.tsRepository: TanStack/ai
Length of output: 963
🏁 Script executed:
sed -n '450,470p' packages/typescript/ai/src/middlewares/otel.tsRepository: TanStack/ai
Length of output: 557
🏁 Script executed:
# Find the type definition for systemPrompts config
rg -B 5 -A 5 'systemPrompts' packages/typescript/ai/src/middlewares/otel.ts | head -50Repository: TanStack/ai
Length of output: 1062
🏁 Script executed:
# Search for systemPrompts type definition in the broader codebase
rg 'systemPrompts.*:' packages/typescript/ -t ts | head -20Repository: TanStack/ai
Length of output: 2071
🏁 Script executed:
# Find SystemPrompt type definition
rg -A 10 'type SystemPrompt|interface SystemPrompt' packages/typescript/ai/src/types.tsRepository: TanStack/ai
Length of output: 37
🏁 Script executed:
# Check for onConfig callback wrapper boundaries
sed -n '330,365p' packages/typescript/ai/src/middlewares/otel.tsRepository: TanStack/ai
Length of output: 1519
🏁 Script executed:
# Search for onConfig callback definition
rg -B 5 'onConfig:' packages/typescript/ai/src/middlewares/otel.ts | head -20Repository: TanStack/ai
Length of output: 37
🏁 Script executed:
# Search for SystemPrompt type more broadly
fd -t f '\.ts$' packages/typescript/ai/src | xargs rg -l 'SystemPrompt' | head -10Repository: TanStack/ai
Length of output: 341
🏁 Script executed:
# Get the SystemPrompt type from system-prompts.ts
rg -B 2 -A 15 'export.*SystemPrompt' packages/typescript/ai/src/system-prompts.tsRepository: TanStack/ai
Length of output: 1995
🏁 Script executed:
# Search for onConfig handler definition more precisely
rg -B 10 -A 3 'onConfig.*\(' packages/typescript/ai/src/middlewares/otel.ts | head -40Repository: TanStack/ai
Length of output: 481
🏁 Script executed:
# Check if onConfig is wrapped in safeCall
sed -n '300,376p' packages/typescript/ai/src/middlewares/otel.tsRepository: TanStack/ai
Length of output: 2898
🏁 Script executed:
# Check if metadata is validated anywhere or has constraints
rg 'metadata' packages/typescript/ai/src/system-prompts.tsRepository: TanStack/ai
Length of output: 1028
🏁 Script executed:
# Check what type TMetadata is typically bound to in adapters
rg -B 5 -A 5 'SystemPrompt<' packages/typescript/ai/src/activities/chat/ | head -50Repository: TanStack/ai
Length of output: 975
🏁 Script executed:
# Check how iterationCount is used after it's incremented
rg 'iterationCount' packages/typescript/ai/src/middlewares/otel.tsRepository: TanStack/ai
Length of output: 631
🏁 Script executed:
# Check if metadata can realistically contain BigInt or circular refs
# Look at real adapter implementations
fd -t f 'anthropic.*adapter' packages/typescript/ | head -5Repository: TanStack/ai
Length of output: 212
Guard system-prompt metadata serialization to avoid aborting onConfig flow.
At line 392, JSON.stringify(systemPromptMetadata) can throw if the metadata contains non-serializable values (e.g., BigInt, circular references). Because this runs inside safeCall('otel.onConfig', ...), the exception skips the rest of the callback—notably line 454 state.iterationCount += 1—leaving the iteration counter inconsistent and breaking downstream logic that depends on it.
Wrap the serialization in a nested try-catch or guard to ensure the attribute is set (with a fallback value) even if serialization fails, allowing the iteration to complete normally.
💡 Suggested fix
if (systemPromptMetadata.some((m) => m !== null)) {
+ const metadataJson =
+ safeCall('otel.serializeSystemPromptMetadata', () =>
+ JSON.stringify(systemPromptMetadata),
+ ) ?? '[unserializable_system_prompt_metadata]'
iterSpan.setAttribute(
'tanstack.ai.system_prompt.metadata',
- JSON.stringify(systemPromptMetadata),
+ metadataJson,
)
}📝 Committable suggestion
‼️ IMPORTANT
Carefully review the code before committing. Ensure that it accurately replaces the highlighted code, contains no missing lines, and has no issues with indentation. Thoroughly test & benchmark the code to ensure it meets the requirements.
Suggested change
| const systemPromptMetadata = config.systemPrompts.map((p) => | |
| typeof p === 'string' || p.metadata === undefined | |
| ? null | |
| : p.metadata, | |
| ) | |
| if (systemPromptMetadata.some((m) => m !== null)) { | |
| iterSpan.setAttribute( | |
| 'tanstack.ai.system_prompt.metadata', | |
| JSON.stringify(systemPromptMetadata), | |
| ) | |
| const systemPromptMetadata = config.systemPrompts.map((p) => | |
| typeof p === 'string' || p.metadata === undefined | |
| ? null | |
| : p.metadata, | |
| ) | |
| if (systemPromptMetadata.some((m) => m !== null)) { | |
| let metadataJson: string | |
| try { | |
| metadataJson = JSON.stringify(systemPromptMetadata) | |
| } catch { | |
| metadataJson = '[unserializable_system_prompt_metadata]' | |
| } | |
| iterSpan.setAttribute( | |
| 'tanstack.ai.system_prompt.metadata', | |
| metadataJson, | |
| ) |
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/typescript/ai/src/middlewares/otel.ts` around lines 384 - 393, The
JSON.stringify of systemPromptMetadata inside the safeCall('otel.onConfig', ...)
callback can throw and abort the rest of the onConfig flow (preventing
state.iterationCount from incrementing), so wrap the serialization in a local
try-catch: compute a safe string (e.g., try JSON.stringify(systemPromptMetadata)
and on error fall back to a placeholder like '"<unserializable>"' or an array of
nulls) and then call iterSpan.setAttribute('tanstack.ai.system_prompt.metadata',
safeString); ensure the code references the existing systemPromptMetadata
variable and leaves the rest of the callback (including state.iterationCount +=
1) untouched so iteration state remains consistent.
Comment on lines
+70
to
+91
| export function normalizeSystemPrompts<TMetadata = unknown>( | ||
| // Accept the wide public shape (`SystemPrompt<unknown>`) regardless of the | ||
| // caller's `TMetadata`. Adapters know their own metadata shape; the | ||
| // generic narrows the *output* so adapter code can read `p.metadata.X` | ||
| // without an additional cast. | ||
| prompts: ReadonlyArray<SystemPrompt> | undefined, | ||
| ): Array<NormalizedSystemPrompt<TMetadata>> { | ||
| if (!prompts || prompts.length === 0) return [] | ||
| return prompts.map((p, i) => { | ||
| if (typeof p === 'string') return { content: p } | ||
| if (p === null || typeof p !== 'object') { | ||
| throw new TypeError( | ||
| `systemPrompts[${i}]: expected a string or { content, metadata? }, got ${p === null ? 'null' : typeof p}`, | ||
| ) | ||
| } | ||
| if (typeof p.content !== 'string') { | ||
| throw new TypeError( | ||
| `systemPrompts[${i}]: content must be a string, got ${typeof p.content}`, | ||
| ) | ||
| } | ||
| return p as NormalizedSystemPrompt<TMetadata> | ||
| }) |
Contributor
There was a problem hiding this comment.
🧩 Analysis chain
🏁 Script executed:
fd system-prompts.tsRepository: TanStack/ai
Length of output: 100
🏁 Script executed:
head -100 packages/typescript/ai/src/system-prompts.ts | cat -nRepository: TanStack/ai
Length of output: 4359
🏁 Script executed:
wc -l packages/typescript/ai/src/system-prompts.tsRepository: TanStack/ai
Length of output: 103
🏁 Script executed:
# Check if zod is in dependencies
find packages/typescript -name "package.json" -type f | head -5 | xargs grep -l "\"zod\""Repository: TanStack/ai
Length of output: 312
🏁 Script executed:
# Check the entire normalizeSystemPrompts function context
sed -n '60,100p' packages/typescript/ai/src/system-prompts.ts | cat -nRepository: TanStack/ai
Length of output: 1733
🏁 Script executed:
cat packages/typescript/ai/package.json | grep -A 50 '"dependencies"'Repository: TanStack/ai
Length of output: 503
🏁 Script executed:
# Check if zod is used elsewhere in the ai package
rg "from 'zod'" packages/typescript/ai/src/ -A 2 -B 2Repository: TanStack/ai
Length of output: 1822
🏁 Script executed:
# Check what imports are already in system-prompts.ts
head -20 packages/typescript/ai/src/system-prompts.tsRepository: TanStack/ai
Length of output: 1012
🏁 Script executed:
# Check if there are other validation functions in this file
rg "throw new (TypeError|Error)" packages/typescript/ai/src/system-prompts.ts -B 3 -A 1Repository: TanStack/ai
Length of output: 494
Use Zod at the systemPrompts validation boundary.
This function is a public API validation boundary that currently performs manual input validation. Per the coding guidelines for packages/typescript/**/src/**/*.ts, validation should use Zod for consistency. Zod is already available in this package's dependencies and used in similar validation contexts elsewhere (e.g., tool-definition.ts, schema-converter.ts).
♻️ Proposed refactor
+import { z } from 'zod'
+
+const systemPromptSchema = z.union([
+ z.string(),
+ z.object({
+ content: z.string(),
+ metadata: z.unknown().optional(),
+ }),
+])
export function normalizeSystemPrompts<TMetadata = unknown>(
prompts: ReadonlyArray<SystemPrompt> | undefined,
): Array<NormalizedSystemPrompt<TMetadata>> {
if (!prompts || prompts.length === 0) return []
return prompts.map((p, i) => {
- if (typeof p === 'string') return { content: p }
- if (p === null || typeof p !== 'object') {
- throw new TypeError(
- `systemPrompts[${i}]: expected a string or { content, metadata? }, got ${p === null ? 'null' : typeof p}`,
- )
- }
- if (typeof p.content !== 'string') {
- throw new TypeError(
- `systemPrompts[${i}]: content must be a string, got ${typeof p.content}`,
- )
- }
- return p as NormalizedSystemPrompt<TMetadata>
+ const parsed = systemPromptSchema.safeParse(p)
+ if (!parsed.success) {
+ throw new TypeError(
+ `systemPrompts[${i}]: expected a string or { content, metadata? }`,
+ )
+ }
+ return typeof parsed.data === 'string'
+ ? { content: parsed.data }
+ : (parsed.data as NormalizedSystemPrompt<TMetadata>)
})
}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
In `@packages/typescript/ai/src/system-prompts.ts` around lines 70 - 91, The
normalizeSystemPrompts function uses manual typeof/null checks; replace them
with Zod validation at this public boundary: import z from 'zod', define a Zod
schema for SystemPrompt (accepting either string or object { content: string,
metadata?: unknown }) and an array wrapper, then validate the incoming prompts
with schema.parse or schema.safeParse inside normalizeSystemPrompts and map the
parsed results to Array<NormalizedSystemPrompt<TMetadata>>; keep the current
error semantics by throwing on failed validation and ensure you continue to
return the typed NormalizedSystemPrompt<TMetadata> values (referencing
normalizeSystemPrompts, SystemPrompt and NormalizedSystemPrompt).
Two CI failures on the previous review-feedback commit, both follow-ups to the same patch: - `@tanstack/ai-event-client:test:types` failed because the new `devtools-middleware-shape.test.ts` imported `SystemPrompt` from `@tanstack/ai`, but `@tanstack/ai-event-client` only declares `@tanstack/ai` as a peerDependency. Nx's project graph therefore doesn't see a build edge and the `^build` predecessor for test:types never produces the dist files tsc needs. Adding a `workspace:*` devDep would close the loop but reintroduce the circular dev/runtime dep the current architecture is built to avoid. Move the structural guard test into `@tanstack/ai/tests/` instead. `@tanstack/ai` has no edge to ai-event-client's source for this assertion (the mirror gets re-declared inline), and the test now lives alongside the type it's guarding. - `@tanstack/ai:test:eslint` failed on `@typescript-eslint/no-unnecessary-condition` at the C1 runtime checks inside `normalizeSystemPrompts`. TS narrows `p` to the object arm after the string branch, so the `p === null || typeof p !== 'object'` guard looks redundant to the rule even though it's deliberate defence-in-depth at a public API boundary. Cast through `unknown` to drop the narrow and re-validate without disabling the rule. Same treatment for the `p.content` typeof check — read `content` off an `unknown`-typed view of the candidate so the runtime check is visible to the linter without an eslint-disable.
Contributor
There was a problem hiding this comment.
♻️ Duplicate comments (1)
packages/typescript/ai/src/system-prompts.ts (1)
70-98: 🛠️ Refactor suggestion | 🟠 MajorThe past review comment about Zod validation remains applicable.
The previous review correctly identified that this validation boundary should use Zod per the coding guidelines. The current manual type-checking with casts through
unknown(lines 84, 90, 96) is more verbose than a Zod-based approach. As per coding guidelines,packages/typescript/**/src/**/*.tsfiles should use Zod for schema validation.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/typescript/ai/src/system-prompts.ts` around lines 70 - 98, The function normalizeSystemPrompts currently does manual runtime checks; replace them with a Zod schema to validate each SystemPrompt item and the array shape: create a Zod schema for the union (string | object with required string content and optional metadata), use z.array(...) or z.preprocess if needed, call schema.parse/promises to validate the incoming prompts in normalizeSystemPrompts, and then map the parsed values to the returned Array<NormalizedSystemPrompt<TMetadata>>; keep the same function signature (normalizeSystemPrompts) and return shape (NormalizedSystemPrompt<TMetadata>), and ensure Zod validation errors are surfaced instead of the current TypeError messages.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Duplicate comments:
In `@packages/typescript/ai/src/system-prompts.ts`:
- Around line 70-98: The function normalizeSystemPrompts currently does manual
runtime checks; replace them with a Zod schema to validate each SystemPrompt
item and the array shape: create a Zod schema for the union (string | object
with required string content and optional metadata), use z.array(...) or
z.preprocess if needed, call schema.parse/promises to validate the incoming
prompts in normalizeSystemPrompts, and then map the parsed values to the
returned Array<NormalizedSystemPrompt<TMetadata>>; keep the same function
signature (normalizeSystemPrompts) and return shape
(NormalizedSystemPrompt<TMetadata>), and ensure Zod validation errors are
surfaced instead of the current TypeError messages.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 17ec9b4a-48d7-42e5-af09-6318626fd54e
📒 Files selected for processing (2)
packages/typescript/ai/src/system-prompts.tspackages/typescript/ai/tests/devtools-system-prompt-mirror.test.ts
✅ Files skipped from review due to trivial changes (1)
- packages/typescript/ai/tests/devtools-system-prompt-mirror.test.ts
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Restores the ability to attach Anthropic
cache_controlto system prompts — the regression introduced by removing themodelOptions.systemescape hatch in #572 — via a typed, cross-provider mechanism.chat({ systemPrompts })now accepts either a plain string (existing shape — fully backward compatible) or{ content, metadata }. The structured form lets providers attach typed metadata per prompt:Design
@tanstack/aiexportsSystemPrompt<TMetadata = unknown>,NormalizedSystemPrompt, andnormalizeSystemPrompts(). The chat layer carries the wide shape through middleware; each adapter calls the helper at the top of its option-mapping pipeline so the rest of its code only sees{ content, metadata }..contentfor their respectiveinstructions/system/systemInstructionfields.metadata.cache_controland attaches it to the correspondingTextBlockParam. The newAnthropicSystemPromptMetadatainterface is exported from@tanstack/ai-anthropic.Plumbing
TextOptions.systemPromptswidened fromArray<string>toArray<SystemPrompt>.ChatMiddlewareContext.systemPrompts/ChatMiddlewareConfig.systemPromptswidened so middleware can read/mutate the wide shape..contentfor span events; per-prompt metadata is dropped from spans (not useful for observability).@tanstack/ai-event-clientmirrors theSystemPromptshape locally (it intentionally avoids importing@tanstack/aito prevent a circular dep) and projects metadata away on the devtools wire — devtools UI continues to receiveArray<string>.Tests
ai-anthropic: new test verifiescache_controlflows fromsystemPrompts[i].metadataonto the outboundTextBlockParam, and plain-string entries still produce metadata-less blocks.ai-openai: new test verifies mixed string + object-form input produces the expected joinedinstructionsand that provider-foreign metadata is silently dropped.Test plan
pnpm test:ciis green on this branch (33 projects × 8 targets).systemPrompts: ['x']still works for every provider;systemPrompts: [{ content: 'x' }]works for every provider; Anthropicmetadata.cache_controlreaches the wire payload.Notes
@tanstack/ai+@tanstack/ai-anthropicminor; downstream adapters patch).C:\Users\AlemTuzlak\.claude\projects\F--projects-tanstack-ai\design\system-prompts-metadata-2026-05-18.md(outside the repo, not committed).Summary by CodeRabbit
Release Notes
New Features
{ content: string, metadata }for provider-specific configurations.cache_controlmetadata is now supported.Tests