Problem
WebClaw v0.6.19 already buffers the complete response body in Response.body, then calls webclaw_pdf::extract_pdf. For scanned/image-only PDFs, PdfError::EmptyPdf is returned and the already-fetched bytes are dropped. A caller that needs an OCR or vision fallback cannot recover those exact bytes; calling fetch_raw would perform a second source request and can observe different content.
Requested narrow seam
Please expose an opt-in outcome for the final response already buffered by fetch_and_extract_with_options when PDF parsing returns EmptyPdf:
- original PDF bytes from that response;
- final URL;
- content type;
- byte length;
- SHA-256;
- stable reason such as
empty_pdf.
Existing extraction methods and ordinary CLI output should remain unchanged. A new API can return an enum such as Extracted(ExtractionResult) | Artifact(PdfArtifact). If exposed by CLI, an explicit flag plus JSON/base64 output would keep it machine-readable; the caller should supply a strict artifact-size ceiling, and bytes must not appear in logs or error formatting.
Acceptance
- one source fetch for the returned artifact; no
fetch_raw retry;
- artifact bytes are exactly the bytes passed to
extract_pdf;
- only the opt-in path can return bytes;
- over-limit artifacts fail closed;
- final URL/content type/size/SHA-256 are tested;
- existing callers and output remain backward compatible.
This does not ask WebClaw to add OCR or a model dependency. It only preserves a bounded handoff for callers that already own that policy. Is this API shape acceptable upstream?
Problem
WebClaw v0.6.19 already buffers the complete response body in
Response.body, then callswebclaw_pdf::extract_pdf. For scanned/image-only PDFs,PdfError::EmptyPdfis returned and the already-fetched bytes are dropped. A caller that needs an OCR or vision fallback cannot recover those exact bytes; callingfetch_rawwould perform a second source request and can observe different content.Requested narrow seam
Please expose an opt-in outcome for the final response already buffered by
fetch_and_extract_with_optionswhen PDF parsing returnsEmptyPdf:empty_pdf.Existing extraction methods and ordinary CLI output should remain unchanged. A new API can return an enum such as
Extracted(ExtractionResult) | Artifact(PdfArtifact). If exposed by CLI, an explicit flag plus JSON/base64 output would keep it machine-readable; the caller should supply a strict artifact-size ceiling, and bytes must not appear in logs or error formatting.Acceptance
fetch_rawretry;extract_pdf;This does not ask WebClaw to add OCR or a model dependency. It only preserves a bounded handoff for callers that already own that policy. Is this API shape acceptable upstream?