Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
19 changes: 19 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,6 +85,25 @@ Respond with valid JSON matching this schema:

The key insight: **your system prompt stays clean and focused on behavior**, while technical details (tools, schemas) are handled by the API integration layer.

### Prompt Linter (opt-in)

Enable a provider-aware prompt linter without changing your code. It never alters requests; it only logs concise findings. Disabled by default.

```bash
SDK_PROMPT_LINTER=1 \
SDK_PROMPT_LINTER_MODE=warn \
SDK_PROMPT_LINTER_PROVIDER=auto \
SDK_PROMPT_LINTER_RULES='{"core/asks-for-chain-of-thought":"error"}'
```

Examples of checks:

- Missing output contract when extraction implied (warn)
- Undelimited long content (warn)
- Requesting chain-of-thought (warn)
- OpenAI: suggest Structured Outputs if asking for JSON (info)
- Anthropic: suggest simple XML tags for multi-part tasks (info)

## API Reference

### `prompt(systemPrompt?: string)`
Expand Down
55 changes: 54 additions & 1 deletion packages/core/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ A structured, testable, and data-driven prompt engineering SDK for TypeScript/Ja
- **Fluent API**: Chainable methods for building prompts
- **AI SDK-style Tools**: Define tools with Zod schemas and metadata
- **Provider Support**: Compile for OpenAI, Anthropic, or generic formats
- **Built-in Linting**: Automatic validation and token counting
- **Provider-Aware Prompt Linting (opt-in)**: Research-backed checks for OpenAI/Anthropic best practices (off by default)
- **Vercel AI SDK Compatible**: Seamless integration with existing workflows

## Installation
Expand Down Expand Up @@ -121,6 +121,59 @@ const myTool = tool({
- `'openai'` - OpenAI-optimized with separate tools array
- `'anthropic'` - Claude-optimized with XML structure

## Prompt Linter (opt-in)

The SDK includes a lightweight, provider-aware prompt linter you can enable via environment variables. It never changes your request payload and only logs findings. It is disabled by default.

### Enable

```bash
SDK_PROMPT_LINTER=1 # enable (default off)
SDK_PROMPT_LINTER_MODE=warn # info|warn|error threshold (default: warn)
SDK_PROMPT_LINTER_PROVIDER=auto # auto|openai|anthropic (default: auto)
SDK_PROMPT_LINTER_RULES='{"core/asks-for-chain-of-thought":"error"}' # per-rule overrides
```

Findings are logged as single-line messages:

```
core/missing-output-spec (warn): Extraction implied but no output contract found. Add JSON or field list.
```

### What it checks

- Core
- Missing action verb at the start (warn)
- Extraction implied but no output contract (warn)
- Long content lacks fences/delimiters (warn)
- Obvious contradictory instructions (warn)
- Requests chain-of-thought (warn)
- RAG context but missing guardrails (info)
- Unescaped placeholders like `{var}` (info)
- Overlong system prompts (info)
- OpenAI
- Recommend Structured Outputs when asking for JSON (info)
- Durable policy belongs in system message (info)
- Overuse of negative instructions (info)
- Anthropic
- Recommend simple XML tags for multi-part tasks (info)
- Avoid requesting detailed chain-of-thought (warn)
- Put durable policy in system message (info)
- Fence few-shot examples (info)

### Behavior and guarantees

- Default-off; zero behavior change unless enabled
- Purely local; no network calls; never throws
- Non-blocking: logs to console; use MODE to filter
- Fine-grained overrides via `SDK_PROMPT_LINTER_RULES`

### Rationale and sources

- OpenAI: prompt engineering best practices and structured outputs guidance
- Anthropic: prompting docs and XML structuring guidance


## Examples

See the `/examples` directory for complete working examples:
Expand Down
14 changes: 14 additions & 0 deletions packages/core/src/compilers/AnthropicCompiler.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import type { PromptBuilder } from "../PromptBuilder.js";
import { lintPrompt } from "../linter/index.js";
import { extractToolMetadata } from "../tools/tool.js";
import { zodToJsonSchema } from "../schema/zodToJsonSchema.js";

Expand Down Expand Up @@ -31,6 +32,19 @@ ${JSON.stringify(jsonSchema, null, 2)}
};
});

// Lint (default-off, env-controlled). Non-blocking: only logs.
try {
lintPrompt(
{
provider: "anthropic",
system: systemPrompt,
},
console,
);
} catch {
// Never throw from linter
}

return {
system: systemPrompt,
tools: tools.length > 0 ? tools : undefined,
Expand Down
10 changes: 9 additions & 1 deletion packages/core/src/compilers/GenericCompiler.ts
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
import type { PromptBuilder } from "../PromptBuilder.js";
import { lintPrompt } from "../linter/index.js";
import { extractToolMetadata } from "../tools/tool.js";
import { zodToJsonSchema } from "../schema/zodToJsonSchema.js";

Expand Down Expand Up @@ -39,5 +40,12 @@ export function toGeneric(promptBuilder: PromptBuilder): string {
}
}

return parts.join("\n");
const system = parts.join("\n");
// Lint generically using Anthropic-like structuring suggestions as a sensible default
try {
lintPrompt({ provider: "anthropic", system }, console);
} catch {
// Never throw from linter
}
return system;
}
15 changes: 15 additions & 0 deletions packages/core/src/compilers/OpenAICompiler.ts
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
import type { PromptBuilder } from "../PromptBuilder.js";
import type { CoreMessage } from "../types.js";
import { lintPrompt } from "../linter/index.js";
import { extractToolMetadata } from "../tools/tool.js";
import { zodToJsonSchema } from "../schema/zodToJsonSchema.js";

Expand Down Expand Up @@ -49,6 +50,20 @@ export function toOpenAI(
};
}

// Lint (default-off, env-controlled). Non-blocking: only logs.
try {
lintPrompt(
{
provider: "openai",
system: systemMessage.content,
messages: allMessages,
},
console,
);
} catch {
// Never throw from linter
}

return {
messages: allMessages,
tools: tools.length > 0 ? tools : undefined,
Expand Down
3 changes: 3 additions & 0 deletions packages/core/src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,9 @@ export { toGeneric } from "./compilers/GenericCompiler.js";
export { toOpenAI } from "./compilers/OpenAICompiler.js";
export { toAnthropic } from "./compilers/AnthropicCompiler.js";

// Linter exports (types only; functionality auto-runs when enabled via env)
export type { LintFinding, LintSeverity } from "./linter/index.js";

// Schema utilities
export {
zodToJsonSchema,
Expand Down
59 changes: 59 additions & 0 deletions packages/core/src/linter/config.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
// Lightweight linter configuration sourced from environment variables only.
// We avoid filesystem access to keep the SDK browser/bundler-friendly.

export type Provider = "openai" | "anthropic";
export type LintSeverity = "error" | "warn" | "info";

export interface LintRuleOverride {
// Map of rule-id -> severity or "off"
[ruleId: string]: LintSeverity | "off";
}

export interface LinterConfig {
enabled: boolean;
mode: LintSeverity; // threshold: error->only errors, warn->warn+error, info->all
providerOverride?: Provider | "auto";
ruleOverrides?: LintRuleOverride;
}

function normalizeBooleanEnv(value: string | undefined): boolean {
if (!value) return false;
const v = value.toLowerCase();
return v === "1" || v === "true" || v === "on" || v === "yes";
}

function normalizeMode(value: string | undefined): LintSeverity {
if (!value) return "warn";
const v = value.toLowerCase();
if (v === "error" || v === "warn" || v === "info") return v;
return "warn";
}

function parseRuleOverrides(raw: string | undefined): LintRuleOverride | undefined {
if (!raw) return undefined;
try {
const parsed = JSON.parse(raw);
if (parsed && typeof parsed === "object") {
return parsed as LintRuleOverride;
}
} catch {
// Swallow parse errors to avoid breaking user apps
}
return undefined;
}

export function getLinterConfig(): LinterConfig {
// eslint-disable-next-line no-restricted-globals
const env = typeof process !== "undefined" ? process.env : ({} as Record<string, string | undefined>);
return {
enabled: normalizeBooleanEnv(env.SDK_PROMPT_LINTER),
mode: normalizeMode(env.SDK_PROMPT_LINTER_MODE),
providerOverride: (env.SDK_PROMPT_LINTER_PROVIDER as Provider | "auto" | undefined) || "auto",
ruleOverrides: parseRuleOverrides(env.SDK_PROMPT_LINTER_RULES),
};
}

export function severityPassesThreshold(severity: LintSeverity, threshold: LintSeverity): boolean {
const order: Record<LintSeverity, number> = { error: 2, warn: 1, info: 0 };
return order[severity] >= order[threshold];
}
51 changes: 51 additions & 0 deletions packages/core/src/linter/index.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
import { getLinterConfig, severityPassesThreshold, type LinterConfig } from "./config.js";
import { runAnthropicRules, runCoreRules, runOpenAIRules, type LintFinding, type PromptInputs } from "./rules.js";

export { type LintFinding } from "./rules.js";
export { type LintSeverity } from "./config.js";

function applyOverrides(findings: LintFinding[], config: LinterConfig): LintFinding[] {
if (!config.ruleOverrides) return findings;
return findings
.map((f) => {
const override = config.ruleOverrides![f.ruleId];
if (!override) return f;
if (override === "off") return { ...f, severity: "info", message: "(suppressed) " + f.message } as LintFinding;
return { ...f, severity: override } as LintFinding;
})
.filter((f) => f);
}

function filterByMode(findings: LintFinding[], mode: LinterConfig["mode"]): LintFinding[] {
return findings.filter((f) => severityPassesThreshold(f.severity, mode));
}

function formatFinding(f: LintFinding): string {
return `${f.ruleId} (${f.severity}): ${f.message}`;
}

export function lintPrompt(input: PromptInputs, logger?: { warn: (msg: string) => void; error: (msg: string) => void; info?: (msg: string) => void }): LintFinding[] {
const config = getLinterConfig();
if (!config.enabled) return [];

const provider = (config.providerOverride === "auto" || !config.providerOverride) ? input.provider : (config.providerOverride as PromptInputs["provider"]);

const all: LintFinding[] = [
...runCoreRules(input),
...(provider === "openai" ? runOpenAIRules(input) : []),
...(provider === "anthropic" ? runAnthropicRules(input) : []),
];

const overridden = applyOverrides(all, config);
const filtered = filterByMode(overridden, config.mode);

if (filtered.length && logger) {
for (const f of filtered) {
const line = formatFinding(f);
if (f.severity === "error") logger.error(line);
else logger.warn(line);
}
}

return filtered;
}
Loading