|
| 1 | +# Citation Verifier |
| 2 | + |
| 3 | +Verify claims in text against sources using semantic similarity. Supports web search fallback for unverifiable claims. |
| 4 | + |
| 5 | +## Installation |
| 6 | + |
| 7 | +```bash |
| 8 | +npm install @framers/agentos-ext-citation-verifier |
| 9 | +``` |
| 10 | + |
| 11 | +## Usage |
| 12 | + |
| 13 | +```typescript |
| 14 | +import { createExtensionPack } from '@framers/agentos-ext-citation-verifier'; |
| 15 | + |
| 16 | +const pack = createExtensionPack({ |
| 17 | + config: { |
| 18 | + embedFn: async (texts) => myEmbeddingProvider.embed(texts), |
| 19 | + }, |
| 20 | +}); |
| 21 | +``` |
| 22 | + |
| 23 | +## Tool: `verify_citations` |
| 24 | + |
| 25 | +| Parameter | Type | Required | Description | |
| 26 | +|-----------|------|----------|-------------| |
| 27 | +| `text` | string | Yes | Text containing claims to verify | |
| 28 | +| `sources` | array | No | Sources to verify against (`{ title, content, url }`) | |
| 29 | +| `webFallback` | boolean | No | Search the web for unverifiable claims | |
| 30 | + |
| 31 | +### Example |
| 32 | + |
| 33 | +``` |
| 34 | +User: "Verify this research summary" |
| 35 | +Agent: verify_citations({ |
| 36 | + text: "The Earth's core temperature is approximately 5,400°C. Mars has two moons.", |
| 37 | + sources: [ |
| 38 | + { content: "Earth's inner core reaches temperatures of 5,400°C.", url: "https://example.com/earth" } |
| 39 | + ], |
| 40 | + webFallback: true |
| 41 | +}) |
| 42 | +Result: { |
| 43 | + totalClaims: 2, |
| 44 | + supportedCount: 1, // "Earth's core" matched the source |
| 45 | + unverifiableCount: 0, // "Mars has two moons" verified via web |
| 46 | + summary: "2/2 claims verified (100%)" |
| 47 | +} |
| 48 | +``` |
| 49 | + |
| 50 | +## How It Works |
| 51 | + |
| 52 | +1. **Claim extraction** — splits text into atomic factual claims using sentence boundaries |
| 53 | +2. **Batch embedding** — embeds all claims + all sources in one call |
| 54 | +3. **Cosine similarity matrix** — computes claim × source similarity |
| 55 | +4. **Verdict assignment**: |
| 56 | + - `similarity >= 0.6` → **supported** (claim matches a source) |
| 57 | + - `similarity 0.3-0.6` → **weak** (partial match) |
| 58 | + - `similarity < 0.3` → **unverifiable** (no matching source) |
| 59 | +5. **Web fallback** — for unverifiable claims, searches the web via FactCheckTool |
| 60 | + |
| 61 | +## Verdicts |
| 62 | + |
| 63 | +| Verdict | Meaning | Action | |
| 64 | +|---------|---------|--------| |
| 65 | +| `supported` | Claim semantically matches a source | Safe to present | |
| 66 | +| `weak` | Partial match, lower confidence | Present with caveat | |
| 67 | +| `unverifiable` | No source matches | Mark as "[unverified]" or search web | |
| 68 | +| `contradicted` | Source contradicts the claim (via NLI) | Do not present as fact | |
| 69 | + |
| 70 | +## Integration with Deep Research |
| 71 | + |
| 72 | +When using the `deep_research` tool with `depth: "deep"`, citation verification runs automatically on the synthesized output. No explicit `verify_citations` call needed. |
| 73 | + |
| 74 | +Configure in `agent.config.json`: |
| 75 | + |
| 76 | +```json |
| 77 | +{ |
| 78 | + "queryRouter": { |
| 79 | + "verifyCitations": true |
| 80 | + } |
| 81 | +} |
| 82 | +``` |
| 83 | + |
| 84 | +## Environment Variables |
| 85 | + |
| 86 | +| Variable | Required | Description | |
| 87 | +|----------|----------|-------------| |
| 88 | +| `SERPER_API_KEY` | No | Enables web fallback for unverifiable claims | |
| 89 | +| `TAVILY_API_KEY` | No | Alternative search provider for web fallback | |
| 90 | + |
| 91 | +## Related |
| 92 | + |
| 93 | +- **Skill**: `fact-grounding` — instructs the agent to verify claims before presenting |
| 94 | +- **Guardrail**: `grounding-guard` — real-time NLI-based grounding (streaming) |
| 95 | +- **Tool**: `fact_check` — web-based fact verification (single claim) |
| 96 | + |
| 97 | +## License |
| 98 | + |
| 99 | +MIT |
0 commit comments