You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
## Summary
Adds **voice input** to SimplePDF Copilot and reworks the composer +
demo entitlement model. All changes are under `copilot/`.
## Changes
- **Speech-to-text** in the composer: tap the mic → record → transcribe
on stop → editable transcript. Server `/api/transcribe` with a streaming
SSE relay; capped, metered, and sanitized (no raw provider text to the
client).
- **BYOK speech-to-text**: a per-function model picker with **Chat** and
**Speech-to-Text** tabs. BYOK transcription is **browser-direct**
(OpenAI or any OpenAI-compatible endpoint), with an AES-GCM credential
vault, Web-Locks-serialized mutations, linearizable revocation, a
custom-URL SSRF/loopback policy, and format-matched save-time
validation.
- **Improved composer**: one rounded box with **no layout shift** on
mic-click (the textarea ↔ a recipient-aware recording prompt swap;
mic+send ↔ the recording controls), a square-rounded blue send button,
and 12px message bubbles. A voice rate-limit surfaces the same
demo-limit + share panel as chat.
- **Config-driven demo mode** (replaces the old `?share=` invite
system): demo mode is on iff `DEMO_CHAT_API_KEY` + `DEMO_CHAT_MODEL` +
`DEMO_RATE_LIMIT_TURNS` + `DEMO_STT_OPENAI_API_KEY` are all set. Every
visitor then uses the operator's keys, rate-limited **per IP** (no
invite links). Removed the `SHARED_API_KEYS` map, `share_query.ts`, and
all client `?share=` plumbing.
- **System prompt**: refactored the form-filling prompt (collapsed
overlapping interactivity/filling/tone sections, trimmed verbose
examples) and added a **bulk-fill branch** — when the user hands over
many values at once, fill them all, confirm once, then resume the
one-at-a-time flow.
- **Security**: `ChatRequestSchema` now allowlists `user`/`assistant`
roles, so a `role:'system'` message in the request body can no longer be
promoted into the system prompt (system-authority prompt injection).
Found via an adversarial review.
## Reviews & tests
Full-branch and per-change `/code-review`, including BADobe security
passes on the transcribe route, the BYOK vault/revocation, the demo
entitlement cutover, and the chat injection vector. **168 unit tests,
`tsc` + Biome green; 23 locales complete.**
## Operator note
Demo mode is now **open to anyone who can reach the page** (no invite
gating), so the per-IP turn cap (`DEMO_RATE_LIMIT_TURNS`) is the cost
control — size it accordingly and make sure your edge sets a
non-spoofable client-IP header. `.env.example` documents all four
`DEMO_*` vars.
## Deferred follow-ups (non-blocking)
Real-mic / cross-browser visual QA, a two-page Playwright
revocation-race e2e, format-matched fixture speech clips, and ~17
pre-existing en-only `modelPicker` i18n keys.
Copy file name to clipboardExpand all lines: copilot/README.md
+28-8Lines changed: 28 additions & 8 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -69,6 +69,10 @@ Browser
69
69
- SimplePDF Copilot drives the editor through `postMessage` (focus a field, set a value, navigate, submit)
70
70
- LLM streaming runs through your server via the Vercel AI SDK; you choose the provider
71
71
- Tool calls are executed in the browser, against the iframe. Your server only proxies the chat stream.
72
+
-**Voice input is different — it is a deliberate audio egress, on one of two paths.** Dictating into the composer records a short audio clip in the browser; recording starts when you tap the mic, and when you confirm it (✓) the clip is transcribed and the editable transcript drops into the textarea. Two routes, each named in the recorder before the audio is sent:
73
+
-**Demo (server):** when the deployment is in demo mode (operator keys configured), the clip uploads to `/api/transcribe`, which forwards it to OpenAI (`gpt-4o-transcribe`) and returns the transcript. So **audio leaves the browser to SimplePDF's server and then OpenAI** (the server keeps no audio, logs no transcript).
74
+
-**BYOK (browser-direct):** configure a Speech-to-Text provider (OpenAI or a custom OpenAI-compatible endpoint) in the model picker's Speech-to-Text tab; the clip is sent **directly from the browser to that endpoint, never to SimplePDF**. The key lives only in this browser's encrypted vault — a demo/reference feature (a browser-held key is exposed to anything on the page).
75
+
- In both cases PDF bytes still stay on-device, and audio is sent only when you confirm the recording (never automatically). The recorder prompt names the actual audio recipient before you confirm: the **demo** prompt reads "Speak to SimplePDF Copilot…" (its server→OpenAI flow is the one described above, and "What is this demo?" documents it); the **BYOK** prompt reads "Speak to OpenAI…" or "Speak to <your endpoint>…" (sent directly to that provider, not to SimplePDF).
72
76
73
77
## Built with
74
78
@@ -108,16 +112,30 @@ In the running app, open the chat sidebar, click **Bring your own provider**, pa
108
112
> [!IMPORTANT]
109
113
> **Keep the dev port at 3001.** The SimplePDF demo workspace whitelists exactly one local origin, `http://localhost:3001`, and the editor will only load on a parent page served from that exact host and port. The browser enforces this on iframe load: any other port (e.g. 3000, 5173) or any other host is refused. The `dev` script in `package.json` pins port 3001; don't override it with `--port` flags. To run on your own domain or a different port, you need a SimplePDF [Pro](https://simplepdf.com/pricing) account so you can set your own `companyIdentifier` and whitelist your origin in the SimplePDF dashboard.
110
114
111
-
### Share it without asking viewers for a key
115
+
### Run the demo without asking viewers for a key
112
116
113
-
Sharing the demo with non-technical users (a teammate, a prospect, a friend) is friction-heavy if every visitor has to paste a provider key. To skip that step, set `SHARED_API_KEYS`in your `.env` and append `?share=<id>` to the URL: the server pays for the LLM under your account, the chat opens already wired up, and the Model Picker stays out of the way.
117
+
Asking every visitor to paste a provider key is friction-heavy. To skip that, put the deployment **in demo mode**: configure a single chat key/model/turn-cap plus a transcription key in your `.env`, and the server pays for chat + voice under your account for every visitor. There are no invite links — demo mode is simply on whenever both are configured. The chat opens already wired up and the Model Picker stays out of the way (visitors can still bring their own key to override).
114
118
115
-
Two providers are supported on the shared-key path:
119
+
Demo mode requires **both**:
116
120
117
-
-Anthropic Claude Haiku 4.5 (`model: "anthropic_haiku_4_5"`)
118
-
-DeepSeek V4 Flash (`model: "deepseek_v4_flash"`)
121
+
-`DEMO_CHAT_API_KEY` + `DEMO_CHAT_MODEL` + `DEMO_RATE_LIMIT_TURNS` — the chat key, the model, and the per-IP turn cap.
122
+
-`DEMO_STT_OPENAI_API_KEY` — the voice transcription key (transcription-only, never your chat key).
119
123
120
-
See [`.env.example`](./.env.example) for the JSON shape, the per-share rate-limit options, and the portable base64 one-liner for hosts that mangle embedded quotes (DigitalOcean App Platform, Render, fly.io). Then visit `http://localhost:3001/?share=<id>` and you're set.
124
+
Two chat models are supported on the demo path:
125
+
126
+
- Anthropic Claude Haiku 4.5 (`DEMO_CHAT_MODEL=anthropic_haiku_4_5`)
Leave the demo vars unset and the deployment runs **BYOK-only**: every visitor brings their own key via the Model Picker. Note: with no invite gating, demo mode is open to anyone who can reach the page, so the **per-IP turn cap (`DEMO_RATE_LIMIT_TURNS`) is the cost control** — size it accordingly. See [`.env.example`](./.env.example) for the exact vars.
130
+
131
+
### Voice input (dictation)
132
+
133
+
The chat composer shows a microphone whenever the browser can record. Clicking it needs **both** a Chat model and a Speech-to-Text provider configured (a transcript you can't send is useless), so it opens the model picker on whichever is missing, else it records. Two transcription routes:
134
+
135
+
-**Demo (server):** when the deployment is in demo mode (the demo vars above are set, including `DEMO_STT_OPENAI_API_KEY`), the clip is transcribed via `/api/transcribe` on the operator's key. Voice and chat share the same demo entitlement, so both require the keys to be configured; without the transcription key the deployment isn't in demo mode and voice + demo chat are unavailable (BYOK still works).
136
+
-**BYOK (browser-direct):** in the picker's **Speech-to-Text** tab, configure OpenAI (`gpt-4o-mini-transcribe` / `gpt-4o-transcribe`) or a custom OpenAI-compatible endpoint. The clip is transcribed directly browser→provider, never touching SimplePDF's server. No env var needed.
137
+
138
+
See the privacy notes above for the per-route audio-egress disclosure.
URL-encode the value whenever it carries its own query string (e.g. `?prefill=`), otherwise the nested params get parsed as part of the page URL and dropped.
131
149
132
-
The value must be an absolute `http(s)` URL on your configured base-domain family (`*.simplepdf.com` by default, or whatever host `VITE_SIMPLEPDF_BASE_DOMAIN` resolves to). Third-party origins are rejected on purpose: the iframe is granted clipboard access and is wired to the `postMessage` bridge, so framing an arbitrary site would hand it both. A malformed or off-domain `?url=` silently falls back to the default demo form. `?url=` combines with `?lang=` and `?share=`; when set, it wins for what the editor loads.
150
+
The value must be an absolute `http(s)` URL on your configured base-domain family (`*.simplepdf.com` by default, or whatever host `VITE_SIMPLEPDF_BASE_DOMAIN` resolves to). Third-party origins are rejected on purpose: the iframe is granted clipboard access and is wired to the `postMessage` bridge, so framing an arbitrary site would hand it both. A malformed or off-domain `?url=` silently falls back to the default demo form. `?url=` combines with `?lang=`; when set, it wins for what the editor loads.
133
151
134
152
### Ship it on your own domain
135
153
@@ -158,9 +176,10 @@ For multi-container deployments (or any deploy where you want per-IP rate-limit
158
176
The button reads [`.do/deploy.template.yaml`](https://github.com/SimplePDF/simplepdf-embed/blob/main/.do/deploy.template.yaml) at the repo root: Node 24 buildpack, single instance, builds from `/copilot`. DigitalOcean prompts you for the env vars at setup time:
159
177
160
178
-`VITE_SIMPLEPDF_COMPANY_IDENTIFIER` (required, no default): your SimplePDF company subdomain (Pro plan or higher)
161
-
-`SHARED_API_KEYS`(optional secret): paste a JSON or base64 payload to enable the `?share=<id>` flow; leave empty for BYOK-only
179
+
-`DEMO_CHAT_API_KEY` / `DEMO_CHAT_MODEL` / `DEMO_RATE_LIMIT_TURNS` + `DEMO_STT_OPENAI_API_KEY`(optional secrets): set **all** of them to put the deployment in demo mode (the server pays for chat + voice, capped per IP by the turn count); leave unset for BYOK-only
162
180
-`REDIS_URL` (optional secret): a Redis-protocol connection URL (Valkey on DO Managed Caching works as-is). Required for multi-container deployments where per-IP rate-limit counters must be shared. Leave empty for single-instance / BYOK-only.
163
181
-`IP_HASH_SALT` (required when `REDIS_URL` is set): salts the SHA-256 IP hash so persisted snapshots aren't brute-forceable. Generate with `openssl rand -hex 32`.
182
+
-`DEMO_STT_OPENAI_API_KEY` (optional secret): an OpenAI key for the **demo** voice path only (`gpt-4o-transcribe` via `/api/transcribe`). Never the chat key, never a BYOK key. It is part of demo mode (see the combined bullet above): without it the deployment is **not** in demo mode, so demo chat **and** voice are both off and every visitor falls back to BYOK (which is browser-direct and needs no server key).
164
183
165
184
Once deployed, copy the `.ondigitalocean.app` URL DigitalOcean assigns and add it to your SimplePDF dashboard's whitelist before opening the app.
166
185
@@ -221,6 +240,7 @@ The architecture is deliberate:
221
240
-**Document data stays in the browser.** SimplePDF processes PDFs client-side. The iframe never uploads document bytes to SimplePDF.
222
241
-**Chat traffic flows through your server.** You control the provider, the keys, the logs, and any RAG / internal data layered in.
223
242
-**Submission is direct to your storage.** On Premium with [Bring Your Own Storage](https://simplepdf.com/pricing) (S3, Azure Blob, or SharePoint), completed PDFs upload from the browser to your bucket, never to SimplePDF servers.
243
+
-**Voice input is the one exception, and it is opt-in.** Dictation sends the recorded audio clip out of the browser — unlike PDF bytes, dictated audio (which can contain PII/PHI) leaves the device. Two routes, each disclosed before recording: the **demo** route uploads to SimplePDF's server and on to OpenAI (the server keeps no audio and logs no transcript text — only an IP hash, byte size, and elapsed time); the **BYOK** route sends audio **directly to the user's chosen provider, never to SimplePDF** (SimplePDF makes no retention claim for user-selected providers — that's the provider's policy). Audio is sent only on an explicit Record.
0 commit comments