Skip to content

Expose bounded PDF bytes on EmptyPdf without refetching #104

Description

@sicko7947

Problem

WebClaw v0.6.19 already buffers the complete response body in Response.body, then calls webclaw_pdf::extract_pdf. For scanned/image-only PDFs, PdfError::EmptyPdf is returned and the already-fetched bytes are dropped. A caller that needs an OCR or vision fallback cannot recover those exact bytes; calling fetch_raw would perform a second source request and can observe different content.

Requested narrow seam

Please expose an opt-in outcome for the final response already buffered by fetch_and_extract_with_options when PDF parsing returns EmptyPdf:

  • original PDF bytes from that response;
  • final URL;
  • content type;
  • byte length;
  • SHA-256;
  • stable reason such as empty_pdf.

Existing extraction methods and ordinary CLI output should remain unchanged. A new API can return an enum such as Extracted(ExtractionResult) | Artifact(PdfArtifact). If exposed by CLI, an explicit flag plus JSON/base64 output would keep it machine-readable; the caller should supply a strict artifact-size ceiling, and bytes must not appear in logs or error formatting.

Acceptance

  • one source fetch for the returned artifact; no fetch_raw retry;
  • artifact bytes are exactly the bytes passed to extract_pdf;
  • only the opt-in path can return bytes;
  • over-limit artifacts fail closed;
  • final URL/content type/size/SHA-256 are tested;
  • existing callers and output remain backward compatible.

This does not ask WebClaw to add OCR or a model dependency. It only preserves a bounded handoff for callers that already own that policy. Is this API shape acceptable upstream?

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions