Skip to content

Commit cf56d22

Browse files
authored
feat: add client.parse() for the Data Extraction API (/extraction/parse) (#12)
* chore(spec): vendor Data Extraction API OpenAPI spec (2026-05-25) The Data Extraction API (`POST /extraction/parse`) ships on a separate OpenAPI document from the existing DWS Processor API. Vendor the public spec so the new typed client surface is anchored to a checked-in source of truth. The Processor API spec stays at `dws-api-spec.yml`; the Data Extraction spec lives alongside it at `dws-data-extraction-spec.yml`. * feat(types): add Data Extraction API types for /extraction/parse Introduce hand-written types mirroring the public Data Extraction OpenAPI 3.1 contract (version 2026-05-25): - ParseMode (text | structure | understand | agentic) - ParseOutputFormat (spatial | markdown), ParseOutputOptions - ParseInstructions and ParseOptions request shapes - ParseResponseSpatial / ParseResponseMarkdown discriminated by output payload - Per-element types: ParagraphElement, FormulaElement, PictureElement, TableElement (with ParseTableCell), KeyValueRegionElement (with KeyValuePair / KeyValueEntity), HandwritingElement, and shared ParseElementBase / ParseBounds / ParsePageRef / ParseWord - ParseErrorResponse with structured failingPaths - ParseMetrics, ParseUsage (carrying data_extraction_credits), ParseConfiguration The Data Extraction API bills against a separate extraction-credits bucket from the processor API; type JSDoc makes the distinction explicit so client code does not conflate the two billing buckets. Wires the new endpoint into RequestTypeMap / ResponseTypeMap so the existing HTTP layer stays type-safe end-to-end. * feat(client): support /extraction/parse with parse() and convenience wrappers Adds first-class client methods for the Data Extraction API: - parse(input, options?) — full-fidelity call against POST /extraction/parse, supporting local files, buffers, streams, and URL inputs. Handles multipart upload for binary inputs and JSON body for URL-only requests. - parseToMarkdown(input, mode?) — convenience wrapper returning the whole- document Markdown string directly. Defaults to mode='text' (cheapest). - parseElements(input, mode?, includeWords?) — convenience wrapper returning the typed spatial-elements array. Defaults to mode='structure'. Threads x-nutrient-api-version through the HTTP layer when the caller pins a specific API version. JSDoc on every new method makes the billing distinction explicit: the Data Extraction API bills against extraction credits, a separate bucket from the processor API credits used by the rest of NutrientClient. The full set of new types is re-exported from the package root. * test(parse): cover request shape, modes, output formats, and error paths Adds 19 unit tests around the new /extraction/parse surface: - Request shape: multipart vs JSON, apiVersion header forwarding, option serialisation (language, output, includeWords), default behaviour. - Mode coverage: all four modes (text, structure, understand, agentic) round-trip through the instructions payload. - Output coverage: spatial elements and whole-document Markdown variants validated end-to-end, including extraction-credit accounting on the response (data_extraction_credits, not processor credits). - Error paths: HTTP-layer ValidationError propagation, file-input preflight failures surfaced before the request leaves the process. - Convenience wrappers: parseToMarkdown and parseElements default modes and includeWords forwarding, plus defensive output-mismatch errors. Adds examples/src/parse_smoke.ts — a live operator-runnable smoke test that prints a parsed summary plus extraction-credit usage. Documents the build/pack/install/run recipe in the file header. * docs: document /extraction/parse surface and extraction-credit billing - README: new "Data Extraction (/extraction/parse)" section with mode/ credit table, request examples for spatial + Markdown outputs, URL input, convenience wrappers, and a pointer to the smoke example. - docs/METHODS.md: new entries for parse, parseToMarkdown, parseElements inserted alongside the existing extract* convenience methods. - LLM_DOC.md: inject the same three method signatures so coding agents steered by this rule file know about parse and the extraction-credits bucket. - CHANGELOG.md: Unreleased entry covering the new client surface, the newly-exported public types, the live smoke script, and an explicit call-out that /extraction/parse bills against extraction credits (separate from processor API credits). Every doc surface that mentions cost says "extraction credits" explicitly so downstream readers cannot conflate the two billing buckets. * docs: fix smoke script path and parseElements doc fragment - CHANGELOG: correct path to live smoke script - METHODS.md: fix dangling sentence on parseElements compile-time guard * refactor(types): extract ExtractionCredits to dedicated module Factor the inline extraction-credit billing shape out of ParseUsage into a standalone ExtractionCredits interface in src/types/extraction_credits.ts, mirroring the Python client's type-factoring approach. ParseUsage.data_extraction_credits now references ExtractionCredits instead of an anonymous inline type, making the billing object reusable if future endpoints surface the same shape. ExtractionCredits is re-exported from the package root alongside the other parse types. * docs(client): rewrite parse() JSDoc with use-case-first framing Lead with the "Designed for" preamble naming the three canonical workflows (RAG/search indexing, form/invoice extraction, layout-aware understanding) before describing modes and output formats. Broaden the @PARAM input description to explicitly mention non-PDF inputs (Office documents, images), matching the actual endpoint capability rather than implying PDF-only like sign(). Update the @example block to show a form/invoice extraction recipe alongside the RAG recipe, and replace the generic paragraph-walk with a keyValueRegion traversal that a form-extraction caller can copy directly. * docs: rewrite Data Extraction section with use-case-first framing Restructure the README's /extraction/parse section to lead with use cases (RAG ingestion, form/invoice extraction, layout-aware understanding) before the mode table and code, matching the Python client's documentation approach. Add: - "Choosing an output format" table (markdown vs spatial, with shape and best-for columns). - "Modes — when to use which" table with credit costs and decision guidance. - Two worked recipes: RAG ingestion (PDF → Markdown → embed) and form/invoice extraction (PDF → spatial elements → structured object), each with the convenience-wrapper alternative shown alongside. - Explicit note that the endpoint accepts PDFs, Office documents, and images — not PDFs only. - Mention of the new ExtractionCredits type in the exported-types list. Update METHODS.md parse/parseToMarkdown/parseElements entries to match: lead with use-case positioning, add a parameters table, align examples with the recipe pattern from the README. * feat(client): route parse() via DWS Extract key DWS Extract is a separate product from DWS Processor with its own API key and credit pool. Calling /extraction/parse with the Processor key returns 403. Add an optional `extractApiKey` constructor option (string or async getter) that parse() prefers over apiKey when set; every non-parse method keeps using apiKey. Falls back to apiKey when extractApiKey is omitted, so tenants with a single global DWS key still work. The routing happens via a per-call options copy that swaps apiKey to the extract key — leaves this.options untouched and covers both the multipart file-input path and the JSON url-input path. Drop the bundled parse smoke script — its dual-key dance and pack/install recipe were superseded by the unit-test coverage of the request shape, response handling, and routing. Live verification against a real account belongs to ad-hoc developer sessions, not committed scaffolding. Mirrors PR #47 on the Python sibling client. * refactor(types): derive parse types from generated OpenAPI spec Add `npm run generate:types:extract` that runs openapi-typescript against the vendored dws-data-extraction-spec.yml into src/generated/extract-types.ts, peer to the existing `generate:types` flow for the Processor spec. Rewrite src/types/parse.ts so the schema primitives derive from the generated `components['schemas']` rather than being hand-rolled: - ParseMode, ParseOutputFormat - ParseElement and the six element subtypes (ParagraphElement, FormulaElement, PictureElement, TableElement, KeyValueRegionElement, HandwritingElement) - ParseElementBase, ParseBounds, ParsePageRef, ParseWord - ParseTableCell, KeyValuePair, KeyValueEntity - ParseMetrics, ParseUsage, ParseConfiguration - ParseErrorResponse, ParseErrorDetails, ParseErrorFailingPath - ParagraphRole (now `NonNullable<ParagraphElement['role']>`) Keep four types hand-composed where they add something the spec doesn't express: - ParseOutputOptions / ParseInstructions — the spec marks `OutputOptions.includeWords` as required, but the server has a default and clients shouldn't be forced to pass it. - ParseResponseSpatial / ParseResponseMarkdown — cross-field discriminated narrowing (`elements?: undefined` / `markdown?: undefined`) the spec's ParseOutput doesn't model, letting callers write `if (output.markdown !== undefined)` without per-call `?.` access. - ParseOptions — adds the client-only `apiVersion` header concern that isn't a body field in the spec. Net: ~210 lines of hand-rolled type definitions deleted, replaced with one-line aliases that re-route through the generated schema. The public surface (every exported name) is unchanged. * refactor(types): collapse parse types into http.ts and namespace the spec re-export Most APIs in this client (sign, ocr, watermark, redact, etc.) don't have a dedicated `src/types/<api>.ts` file — they reach types via `components['schemas']['X']` from `src/generated/api-types.ts`. The `src/types/parse.ts` and `src/types/extraction_credits.ts` files added on this branch were an outlier: most of their content was thin one-line aliases over the generated extract spec. Collapse to the rest-of-codebase pattern: - Delete `src/types/parse.ts` (was 254 lines, mostly aliases). - Delete `src/types/extraction_credits.ts` (single hand-rolled interface that duplicated the generated `Usage.data_extraction_credits` shape). - Move the 5 hand-composed types into `src/types/http.ts` (it already imports `ParseInstructions` / `ParseResponse` to type the endpoint maps): `ParseOutputOptions`, `ParseInstructions`, `ParseOptions`, `ParseResponseSpatial`, `ParseResponseMarkdown`, plus the derived `ExtractionCredits` alias. Each carries the JSDoc explaining why it's hand-composed instead of derived. - Drop the 23 cosmetic spec-alias exports from the package root. Consumers who need element-subtype types reach them via the new `extractComponents['schemas']['ParagraphElement']` namespace re-export, mirroring how Processor types are exposed via the existing `components` namespace. The package's public surface still exports the 7 hand-composed types (`ParseOutputOptions`, `ParseInstructions`, `ParseOptions`, `ParseResponse`, `ParseResponseSpatial`, `ParseResponseMarkdown`, `ExtractionCredits`) by name. Internal consumers (`src/client.ts`, the parse unit tests) shift to `extractComponents['schemas']['X']` for spec-derived types. Net: -290 lines on the type-definition surface, no behaviour change. * fix: address code-review findings on the Data Extraction surface Five findings from review: 1. Empty-string `extractApiKey` bypassed constructor validation. `apiKey` uses `!options.apiKey` (falsy, catches `''`); the new `extractApiKey` validator only checked `!== undefined` plus the type guard, so `extractApiKey: ''` passed, propagated into the per-call options as `apiKey: ''`, and produced `Authorization: Bearer ` with no token — surfacing as a confusing server-side 401 instead of a constructor-time `ValidationError`. Add an explicit empty-string check. 2. `extractErrorMessage` in `src/http.ts` checked snake_case (`error_message`, `error_description`) and generic message fields but not `errorMessage` (camelCase) — the field DWS Extract returns on every 4xx/5xx. Result: the server's specific message (e.g. `"invalid mode: 'vlm'"`) was silently replaced by the generic `HTTP <status>: <statusText>` string. Add `errorMessage` to the priority list. 3. `parse()` accepted `mode='text' + output.format='spatial'` and let the server reject with 400. The Python sibling client adds a client-side `ValidationError` for this case (after reviewer feedback). The TS `parseElements()` wrapper blocked it at the type level via `Exclude`, but the low-level `parse()` did not. Add a pre-flight runtime guard. 4. `RequestTypeMap` JSDoc on `/extraction/parse` claimed `instructions` was optional for multipart upload, but the type definition marks it required and the implementation always passes it (an empty object when no options are supplied). Update the comment to match the type. 5. `parse()` `@param options.language` JSDoc described the field as "string or array of ISO 639-2 codes". The underlying spec also accepts lowercase language names (`'english'`, `'german'`) and `+`-joined multilingual strings (`'eng+spa'`). Document all four accepted forms. Adds three unit tests (empty-string `extractApiKey`, `errorMessage` extraction, text+spatial pre-flight rejection). 292 tests pass.
1 parent 121ee7d commit cf56d22

16 files changed

Lines changed: 3213 additions & 7 deletions

CHANGELOG.md

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -7,7 +7,31 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
77

88
## [Unreleased]
99

10-
_Nothing yet._
10+
### Added
11+
12+
- First-class client support for the Data Extraction API (`POST /extraction/parse`).
13+
- `NutrientClient` accepts an `extractApiKey` option (string or async getter)
14+
that `parse()` uses in place of `apiKey`. Data Extraction is a separate
15+
product with its own credit pool, so the Processor key returns 403 against
16+
`/extraction/parse`. When `extractApiKey` is omitted, `parse()` falls back
17+
to `apiKey`, which works on tenants with global DWS keys.
18+
- `NutrientClient.parse(input, options?)` — full request/response surface with
19+
typed support for all four modes (`text`, `structure`, `understand`, `agentic`)
20+
and both output formats (`spatial`, `markdown`).
21+
- `NutrientClient.parseToMarkdown(input, mode?)` — convenience wrapper returning
22+
the whole-document Markdown string directly.
23+
- `NutrientClient.parseElements(input, mode?, includeWords?)` — convenience
24+
wrapper returning the spatial elements array directly.
25+
- Public types: hand-composed `ParseOutputOptions`, `ParseInstructions`,
26+
`ParseOptions`, `ParseResponse`, `ParseResponseSpatial`, `ParseResponseMarkdown`,
27+
and `ExtractionCredits`. The spec primitives (`Mode`, `Element` and the six
28+
subtypes, `Bounds`, `PageRef`, `Word`, `Metrics`, `Usage`, `Configuration`,
29+
`ParseErrorResponse`, etc.) are accessible via the `extractComponents`
30+
namespace re-export — same pattern as `components` for the Processor spec.
31+
- Billing note: `/extraction/parse` debits the account's **extraction
32+
credits** bucket, which is separate from the **processor API credits** used
33+
by the rest of `NutrientClient`. The response surfaces this explicitly in
34+
`usage.data_extraction_credits`.
1135

1236
## [2.0.0] - 2026-01-27
1337

LLM_DOC.md

Lines changed: 58 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -461,6 +461,64 @@ if (kvps && kvps.length > 0) {
461461
}
462462
```
463463

464+
#### parse(input, options?)
465+
Extracts structured content from a document via the Data Extraction API (`POST /extraction/parse`).
466+
467+
Billed against **extraction credits** (a separate bucket from processor API credits used by every other method). Mode costs per page:
468+
- `text` — 1 extraction credit (Markdown only)
469+
- `structure` — 1.5 extraction credits (spatial elements)
470+
- `understand` — 9 extraction credits (default)
471+
- `agentic` — 18 extraction credits
472+
473+
Data Extraction is a separate product with its own API key. Pass it as `extractApiKey` on the client constructor:
474+
475+
```typescript
476+
const client = new NutrientClient({
477+
apiKey: process.env.NUTRIENT_API_KEY!,
478+
extractApiKey: process.env.NUTRIENT_EXTRACT_API_KEY!,
479+
});
480+
```
481+
482+
Falls back to `apiKey` when `extractApiKey` is omitted (only works on tenants with global DWS keys).
483+
484+
```typescript
485+
// Full call: spatial elements with bounding boxes, confidence, reading order
486+
const result = await client.parse('invoice.pdf', {
487+
mode: 'understand',
488+
output: { format: 'spatial', includeWords: true },
489+
language: ['eng', 'spa'],
490+
});
491+
492+
if (result.output.elements !== undefined) {
493+
for (const el of result.output.elements) {
494+
if (el.type === 'paragraph') console.log(el.text);
495+
}
496+
}
497+
498+
// Extraction-credit accounting (separate from processor credits):
499+
console.log(result.usage?.data_extraction_credits?.cost);
500+
501+
// URL input (server fetches the URL):
502+
const remote = await client.parse('https://example.com/doc.pdf', { mode: 'text' });
503+
```
504+
505+
#### parseToMarkdown(input, mode?)
506+
Convenience wrapper that returns just the whole-document Markdown string. Defaults to `mode='text'` (cheapest, 1 extraction credit/page).
507+
508+
```typescript
509+
const markdown = await client.parseToMarkdown('document.pdf');
510+
const richer = await client.parseToMarkdown('scan.pdf', 'understand');
511+
```
512+
513+
#### parseElements(input, mode?, includeWords?)
514+
Convenience wrapper that returns just the array of spatial elements. Defaults to `mode='structure'`. Cannot use `mode='text'`.
515+
516+
```typescript
517+
const elements = await client.parseElements('document.pdf');
518+
const tables = elements.filter(e => e.type === 'table');
519+
const withWords = await client.parseElements('scan.pdf', 'understand', true);
520+
```
521+
464522
#### flatten(file, annotationIds?)
465523
Flattens annotations in a PDF document.
466524

README.md

Lines changed: 171 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -133,6 +133,177 @@ const mergedPdf = await client.merge(['doc1.pdf', 'doc2.pdf', 'doc3.pdf']);
133133

134134
For a complete list of available methods with examples, see the [Methods Documentation](docs/METHODS.md).
135135

136+
## Data Extraction (`/extraction/parse`)
137+
138+
`client.parse()` exposes Nutrient's Data Extraction API. It's designed for
139+
**content-extraction workflows** where you need to feed document content into a
140+
downstream pipeline rather than render or transform the document itself:
141+
142+
- **RAG / search indexing / content migration** — pull a clean Markdown
143+
representation of a document for chunking, embedding, and indexing in a
144+
vector store or search engine.
145+
- **Form and invoice extraction** — pull structured fields (key/value pairs,
146+
tables, semantic regions) out of business documents with bounding boxes and
147+
confidence scores attached to every element.
148+
- **Layout-aware document understanding** — get a typed, page-anchored element
149+
list (paragraphs with semantic roles, tables with cell spans, formulas in
150+
LaTeX, pictures, handwriting) suitable for building document-comprehension
151+
tooling, including agentic workflows.
152+
153+
The endpoint accepts PDFs, Office documents (Word, Excel, PowerPoint), and
154+
images. Unlike `sign()`, it is not restricted to PDFs.
155+
156+
### Choosing an output format
157+
158+
| Format | Best for | Shape |
159+
| ------------------- | --------------------------------------------------------------------------- | --------------------------------------------------------------- |
160+
| `markdown` | RAG, search indexing, content migration — anywhere structured text beats spatial data | `response.output.markdown` — a single Markdown string |
161+
| `spatial` (default) | Form/invoice extraction, layout reconstruction, flows that need per-element confidence | `response.output.elements` — flat array of typed elements |
162+
163+
### Setup — separate Extract API key
164+
165+
Data Extraction is a separate product from the DWS Processor with its own
166+
credit pool and its own API key. Pass both keys when constructing the client:
167+
168+
```typescript
169+
const client = new NutrientClient({
170+
apiKey: process.env.NUTRIENT_API_KEY!, // Processor key
171+
extractApiKey: process.env.NUTRIENT_EXTRACT_API_KEY!, // Data Extraction key
172+
});
173+
```
174+
175+
`extractApiKey` is consulted only by `parse()`, `parseToMarkdown()`, and
176+
`parseElements()`. Every other method on the client (`convert`, `sign`, `ocr`,
177+
`merge`, …) keeps using `apiKey`. If you omit `extractApiKey`, the parse
178+
methods fall back to `apiKey` — that fallback only works on tenants whose
179+
single DWS key authorises both products.
180+
181+
### Quick start
182+
183+
```typescript
184+
import { NutrientClient } from '@nutrient-sdk/dws-client-typescript';
185+
186+
const client = new NutrientClient({
187+
apiKey: process.env.NUTRIENT_API_KEY!,
188+
extractApiKey: process.env.NUTRIENT_EXTRACT_API_KEY!,
189+
});
190+
191+
// Spatial elements (default) — paragraphs, tables, key-value regions, etc.
192+
const result = await client.parse('contract.pdf', { mode: 'understand' });
193+
if (result.output.elements !== undefined) {
194+
for (const el of result.output.elements) {
195+
if (el.type === 'table') console.log(`${el.rowCount}x${el.columnCount} table`);
196+
}
197+
}
198+
199+
// Whole-document Markdown from a born-digital PDF.
200+
const mdResult = await client.parse('report.pdf', { mode: 'text' });
201+
if (mdResult.output.markdown !== undefined) {
202+
console.log(mdResult.output.markdown);
203+
}
204+
```
205+
206+
### Modes — when to use which
207+
208+
| Mode | Credits / page | When to use |
209+
| ------------ | -------------- | ------------------------------------------------------------------------------------------------------------------------- |
210+
| `text` | 1 | Born-digital documents only. No OCR, no AI. Fastest and cheapest path to Markdown. |
211+
| `structure` | 1.5 | OCR-based segmentation with bounding boxes. Handles scanned documents, images, and any input that requires OCR. |
212+
| `understand` | 9 | Full pipeline with AI augmentation on top of OCR. Most accurate for tables, multi-column layouts, formulas, and forms. |
213+
| `agentic` | 18 | Builds on `understand` and adds a vision-language model. Best for image descriptions and complex visual layouts. |
214+
215+
### Recipes
216+
217+
**RAG ingestion** — PDF → Markdown → chunks → embeddings → vector store:
218+
219+
```typescript
220+
const result = await client.parse('whitepaper.pdf', { mode: 'text' });
221+
const markdown = result.output.markdown!;
222+
// Then: chunk on headings, embed, push to your vector store.
223+
```
224+
225+
For born-digital PDFs, `mode: 'text'` is the cheapest path (1 credit/page).
226+
For scanned PDFs or images, switch to `mode: 'structure'` so OCR runs.
227+
228+
Or use the convenience wrapper:
229+
230+
```typescript
231+
const markdown = await client.parseToMarkdown('whitepaper.pdf');
232+
```
233+
234+
**Form/invoice extraction** — PDF → spatial elements → structured object:
235+
236+
```typescript
237+
const result = await client.parse('invoice.pdf', { mode: 'understand' });
238+
const elements = result.output.elements!;
239+
240+
// Pull key/value pairs from form regions.
241+
const fields: Record<string, unknown> = {};
242+
for (const el of elements) {
243+
if (el.type === 'keyValueRegion') {
244+
for (const pair of el.pairs) {
245+
if (pair.key && pair.value) {
246+
fields[String(pair.key.value)] = pair.value.value;
247+
}
248+
}
249+
}
250+
}
251+
252+
// Walk tables — each cell carries row/col indices and span counts.
253+
for (const el of elements) {
254+
if (el.type === 'table') {
255+
console.log(`Table: ${el.rowCount}×${el.columnCount}`);
256+
for (const cell of el.cells) {
257+
console.log(` [${cell.row}][${cell.column}] ${cell.text}`);
258+
}
259+
}
260+
}
261+
```
262+
263+
For complex documents that mix dense images with text, step up to
264+
`mode: 'agentic'` so the VLM produces image descriptions and semantic
265+
classifications (18 credits/page).
266+
267+
Or use the convenience wrapper to skip output-format discrimination entirely:
268+
269+
```typescript
270+
const elements = await client.parseElements('invoice.pdf', 'understand');
271+
```
272+
273+
### Billing — extraction credits vs processor credits
274+
275+
`/extraction/parse` is billed against **extraction credits**, a separate
276+
billing bucket from the **processor API credits** consumed by `convert`,
277+
`ocr`, `sign`, `merge`, and every other endpoint on this client. The two
278+
buckets never debit each other.
279+
280+
Extraction-credit accounting is returned per request:
281+
282+
```typescript
283+
const result = await client.parse('document.pdf', { mode: 'structure' });
284+
const usage = result.usage?.data_extraction_credits;
285+
console.log(`Cost: ${usage?.cost} extraction credits`);
286+
console.log(`Remaining: ${usage?.remainingCredits} extraction credits`);
287+
```
288+
289+
The hand-composed types (`ExtractionCredits`, `ParseOptions`, `ParseInstructions`,
290+
`ParseResponse`, `ParseResponseSpatial`, `ParseResponseMarkdown`,
291+
`ParseOutputOptions`) are exported from the package root. The spec primitives —
292+
`Mode`, `Element` and the six element subtypes, `Bounds`, `PageRef`, `Word`,
293+
`TableCell`, `KeyValuePair`, `KeyValueEntity`, `Metrics`, `Usage`,
294+
`Configuration`, `ParseErrorResponse`, etc. — live under the `extractComponents`
295+
namespace:
296+
297+
```typescript
298+
import type { extractComponents } from '@nutrient-sdk/dws-client-typescript';
299+
300+
type ParagraphElement = extractComponents['schemas']['ParagraphElement'];
301+
type TableElement = extractComponents['schemas']['TableElement'];
302+
```
303+
304+
This mirrors how the Processor types are exposed via the existing `components`
305+
namespace.
306+
136307

137308
## Workflow System
138309

docs/METHODS.md

Lines changed: 106 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -455,6 +455,112 @@ if (kvps && kvps.length > 0) {
455455
}
456456
```
457457

458+
##### parse(input, options?)
459+
Calls the Data Extraction API (`POST /extraction/parse`) to extract structured
460+
content from a document. Designed for **RAG ingestion**, **search indexing**,
461+
**content migration**, and **form/invoice extraction** workflows where the goal
462+
is to feed document content into a downstream pipeline rather than render or
463+
transform the document itself.
464+
465+
Accepts PDFs, Office documents (Word, Excel, PowerPoint), and images as input.
466+
467+
Billed against **extraction credits** — a separate billing bucket from the
468+
processor API credits consumed by every other method on this client. See the
469+
[README's Data Extraction section](../README.md#data-extraction-extractionparse)
470+
for the full positioning, the per-mode comparison table, and worked recipes.
471+
472+
Requires a Data Extraction API key — pass it as `extractApiKey` on the client
473+
constructor (see [Setup — separate Extract API key](../README.md#setup--separate-extract-api-key)).
474+
Falls back to `apiKey` if `extractApiKey` is omitted.
475+
476+
**Parameters**:
477+
- `input: FileInputWithUrl` — The document to parse. Accepts local files (paths,
478+
Buffers, streams), a URL string, or a `{ type: 'url', url: '...' }` object.
479+
The endpoint accepts PDFs, Office documents, and images.
480+
- `options?: ParseOptions` — Optional configuration:
481+
- `mode``'text'` (1 cr/page, born-digital, Markdown only),
482+
`'structure'` (1.5 cr/page, OCR + spatial layout),
483+
`'understand'` (9 cr/page, AI-augmented, default),
484+
or `'agentic'` (18 cr/page, VLM-augmented).
485+
- `output.format``'spatial'` (typed elements with bounds
486+
and confidence) or `'markdown'` (whole-document Markdown string).
487+
- `output.includeWords` — include word-level OCR data inside elements.
488+
- `language` — OCR language hint (`'eng'`, `'deu'`, `['eng', 'spa']`, etc.).
489+
- `apiVersion` — optional `x-nutrient-api-version` header override.
490+
491+
**Returns**: `ParseResponse` — full response envelope with `output`, `metrics`,
492+
`configuration`, and `usage.data_extraction_credits` (cost + remaining balance).
493+
494+
```typescript
495+
// RAG ingestion — born-digital PDF → Markdown (1 extraction credit/page).
496+
const result = await client.parse('whitepaper.pdf', { mode: 'text' });
497+
if (result.output.markdown !== undefined) {
498+
console.log(result.output.markdown);
499+
}
500+
501+
// Form extraction — typed spatial elements with bounds and confidence.
502+
const invoice = await client.parse('invoice.pdf', { mode: 'understand' });
503+
if (invoice.output.elements !== undefined) {
504+
for (const el of invoice.output.elements) {
505+
if (el.type === 'keyValueRegion') {
506+
for (const pair of el.pairs) {
507+
console.log(pair.key?.value, '', pair.value?.value);
508+
}
509+
}
510+
}
511+
}
512+
513+
// OCR-backed extraction with word-level data and multilingual hint.
514+
const scan = await client.parse('scan.pdf', {
515+
mode: 'structure',
516+
output: { format: 'spatial', includeWords: true },
517+
language: ['eng', 'spa'],
518+
});
519+
520+
// URL input — the server fetches the document, no client-side download.
521+
const remote = await client.parse('https://example.com/document.pdf');
522+
523+
// Billing — extraction credits, not processor credits.
524+
const usage = remote.usage?.data_extraction_credits;
525+
console.log(`Cost: ${usage?.cost} extraction credits`);
526+
console.log(`Remaining: ${usage?.remainingCredits} extraction credits`);
527+
```
528+
529+
##### parseToMarkdown(input, mode?)
530+
Convenience wrapper that calls `parse()` with `output.format = 'markdown'` and
531+
returns the Markdown string directly. Defaults to `mode='text'` (1 extraction
532+
credit/page) — the cheapest path for born-digital PDFs. Switch to
533+
`mode='structure'` for scanned documents or images so OCR runs.
534+
535+
```typescript
536+
// Born-digital PDF → Markdown (cheapest).
537+
const markdown = await client.parseToMarkdown('document.pdf');
538+
539+
// Scanned document or image → OCR-backed Markdown.
540+
const scanned = await client.parseToMarkdown('scan.pdf', 'structure');
541+
542+
// AI-augmented Markdown for complex layouts.
543+
const rich = await client.parseToMarkdown('report.pdf', 'understand');
544+
```
545+
546+
##### parseElements(input, mode?, includeWords?)
547+
Convenience wrapper that calls `parse()` with `output.format = 'spatial'` and
548+
returns the spatial elements array directly. Defaults to `mode='structure'`
549+
(1.5 extraction credits/page). Passing `mode='text'` is rejected at compile
550+
time — `text` mode only produces Markdown, not spatial elements.
551+
552+
```typescript
553+
// OCR-backed spatial elements.
554+
const elements = await client.parseElements('document.pdf');
555+
556+
// AI-augmented extraction with word-level OCR data.
557+
const withWords = await client.parseElements('invoice.pdf', 'understand', true);
558+
559+
// Filter by element type.
560+
const tables = elements.filter(e => e.type === 'table');
561+
const kvRegions = elements.filter(e => e.type === 'keyValueRegion');
562+
```
563+
458564
##### flatten(file, annotationIds?)
459565
Flattens annotations in a PDF document.
460566

0 commit comments

Comments
 (0)