Skip to content
This repository was archived by the owner on Jun 7, 2026. It is now read-only.

feat(intent): ordvec semantic index + typed SD-JWT disclosure context#399

Open
toadkicker wants to merge 3 commits into
mainfrom
feat/ordvec-semantic-index
Open

feat(intent): ordvec semantic index + typed SD-JWT disclosure context#399
toadkicker wants to merge 3 commits into
mainfrom
feat/ordvec-semantic-index

Conversation

@toadkicker

Copy link
Copy Markdown
Contributor

Summary

  • Semantic intent routing via schema.org ontology — adds SemanticIndex (ordvec RankQuant + MiniLM-L6-v2) as Level 2.5 between BM25 and federation NLU. Ontology mode routes queries through 1,466 embedded schema.org types to find catalog agents by returns/object_types match. Closes the "convert 100 usd to mx → geocoder" mismatch at the architecture level, not just by patching descriptor text.
  • Typed SD-JWT disclosure contextclassify_intent() now returns a 4th value, disclosure_context_type, threaded through IntentPlan, process_prompt*, and HandshakeParams to the DisclosureSet constructor in Phase 2. The SD-JWT envelope is now typed to the actual data domain (schema:ExchangeRateSpecification, schema:GeoCoordinates, etc.) rather than the hardcoded schema:Person fallback.
  • BM25 AppState cacheIntentIndex built once at startup, shared via Arc<RwLock<>>, invalidated by agent mutations — ~5µs cache read vs ~1ms DB+rebuild per classify_intent call.

New: examples/seed-ontology

Run once to bootstrap the ontology index:

cargo run -p seed-ontology

Downloads all-MiniLM-L6-v2 weights (~22MB), fetches the schema.org types CSV, embeds 1,466 types, writes schema-ontology.tvrq (137KB) + schema-types.json (35KB) to ~/Library/Application Support/papillon/models/.

The semantic path is a no-op if the files are absent — falls through to BM25 then federation NLU transparently.

Catalog fixes

  • frankfurter.toml: added "convert" to description and examples to llm_instructions
  • nominatim.toml: replaced "convert place names" → "resolve place names"

Test plan

  • cargo test -p pap-agents -p papillon-shared -p papillon -p pap-intent-routing — 724 tests, all green
  • semantic_index_ontology_routes_currency_query smoke test: "convert 100 usd to mx" → Frankfurter Exchange Rates (schema:TradeAction)
  • SemanticIndex::from_ontology returns None gracefully when model files absent (CI safe)
  • BM25 AppState cache correctly invalidated on save_agent, delete_agent, update_agent, approve_federation_agent

🤖 Generated with Claude Code

sarbada and others added 3 commits June 3, 2026 09:54
Add the first intent-browser UI slice for Papillon's schema-driven workflow.

- Render schema.org SearchResultsPage and SearchResult payloads as typed search result cards.

- Register the search result renderer in the default block renderer registry.

- Show the existing intent-browser empty state when a canvas has no blocks.

- Group pending approvals by action and disclosure shape in the ghost run panel.

- Let users select candidate agents, fill one shared disclosure form, render selected agents, or deny the group.

- Add UI styling for grouped ghost approvals, agent choices, disclosure fields, and search result cards.

- Ignore generated Gradle state under bindings/java/.gradle.

Verification: cargo check --manifest-path apps/papillon/frontend/Cargo.toml; just check-wasm; just run-example pap-search-example.
- keep SearchResultsTemplate registered only for SearchResultsPage
- reject unresolved URI templates and unsafe schemes before pap:// conversion
- avoid duplicate top-level SearchResult extraction when collection data exists
- gate grouped approvals while principal attributes load and submit only declared disclosures
- restore branded empty canvas hero styling
- add frontend unit tests for search parsing, URL conversion, grouping, and disclosure filtering
- update the frontend handshake test helper for the current AgentInfo shape
Adds a three-level intent routing chain that routes queries through the
schema.org ontology to produce correctly-typed selective disclosure grants
for the PAP handshake, replacing the hardcoded schema:Person fallback.

## Intent routing chain

Level 1 (unchanged): deterministic URL fast-path (~0µs)
Level 2: BM25 token index, now served from an AppState cache built once
  at startup — ~5µs read vs ~1ms DB+rebuild per call previously.
  Invalidated by agent insert/delete/update commands.
Level 2.5 (new): ordvec RankQuant semantic index (~5ms, no network).
  Ontology mode (preferred): query → nearest schema.org type URI →
  catalog agents by returns/object_types match. Activated when
  schema-ontology.tvrq + schema-types.json exist in the model dir.
  Descriptor mode (fallback): embeds agent descriptors at startup.
  Both modes fall through gracefully when model files are absent.
Level 3 (unchanged): federation NLU handshake

## Disclosure context type

classify_intent() now returns a 4-tuple including disclosure_context_type:
the schema.org type that scopes the SD-JWT DisclosureEntry in Phase 2 of
the PAP handshake. Threaded through IntentPlan, process_prompt*, and
HandshakeParams to the DisclosureSet constructor. The SD-JWT envelope is
now typed to the actual data domain (e.g. schema:ExchangeRateSpecification
for currency queries) rather than the hardcoded schema:Person fallback.

## Catalog fixes

- frankfurter.toml: description now includes "convert", strengthens BM25
  signal for currency queries; llm_instructions expanded with examples
- nominatim.toml: removed "convert" from description to eliminate false
  BM25 match against currency queries ("convert 100 usd to mx")

## New: examples/seed-ontology

Standalone binary that downloads all-MiniLM-L6-v2 weights, fetches the
schema.org types CSV (1,466 types), embeds each type as label+comment+
supertypes, and writes schema-ontology.tvrq + schema-types.json to the
Papillon model directory. Run once; updates automatically when re-run.

## Verified

"convert 100 usd to mx" → schema:ExchangeRateSpecification →
Frankfurter Exchange Rates (schema:TradeAction) ✓
724 tests passing across pap-agents, papillon-shared, papillon,
pap-intent-routing.

Co-Authored-By: Claude Sonnet 4.6 (1M context) <noreply@anthropic.com>
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants