Correct failed-request billing and document auto proxy escalation - #1370
Merged
hmishra2250 merged 3 commits intoSep 8, 2026
Conversation
The "When credits are charged" section said credits are charged even when the target returns 403 or 404. Two lines above, the prompt injection row said 5 credits are billed "instead of the usual 0 for a failed scrape". Both claims cannot be true, and the pricing page FAQ says failed requests are not charged at all. Code truth on firecrawl core origin/main is a split rule, not a flat one. apps/api/src/lib/scrape-billing.ts:99 bills 0 when the document is null and apps/api/src/lib/scrape-billing.ts:140 bills 1 credit when a document came back. apps/api/src/scraper/scrapeURL/index.ts:746 makes any non-2xx status count as a returned document, so a 403 or 404 that the target actually answered bills 1 credit while a scrape that produced nothing bills 0. The carve-outs that bill with no document are also unpublished: threat protection scans (scrape-billing.ts:32), the prompt injection guard (scrape-billing.ts:122), FIRE-1 (scrape-billing.ts:103), lockdown cache miss (scrape-billing.ts:110), Monitor error pages (apps/api/src/services/monitoring/store.ts:253), and time-billed Interact and Browser sessions (apps/api/src/lib/browser-billing.ts:1). Rewrite the section as the split rule plus a table of the carve-outs, and point the prompt injection row at it instead of restating a partial rule. Finding OB-03 A-1, report DI-2026-09-04-WEEKLY. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SL5chNkWnr4Gy6uuB8PeKS
The docs say auto "retries with enhanced on failure" but never say what counts as a failure, so users cannot tell whether a given result was retried or why a 404 was not. apps/api/src/scraper/scrapeURL/index.ts:748 escalates only on 401, 403 or 429, only when proxy is auto, and only when the stealthProxy flag is not already set, so it happens at most once per request. No other status code triggers it. Add a short section stating the trigger, the once-per-request limit, and that an escalated retry is not charged separately. Enhanced proxies carry no surcharge today, per the comment at apps/api/src/lib/scrape-billing.ts:71. Finding OB-06, report DI-2026-09-04-WEEKLY, and the Sep-5 protected-target finding in DI-2026-09-05-WEEKLY. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SL5chNkWnr4Gy6uuB8PeKS
|
Preview deployment for your docs. Learn more about Mintlify Previews.
💡 Tip: Enable Automations to automatically generate PRs for you. |
This was referenced Sep 7, 2026
Chadha93
reviewed
Sep 7, 2026
The proxy parameter is deprecated and the note at the top of this page already says so. The escalation section now describes what Firecrawl does by default, and drops the sentence telling readers they can turn escalation off by setting proxy to basic or enhanced. Raised by Gaurav in team-support on the docs PR thread. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two changes.
billing.mdxgets the real failed-request rule instead of two statements that contradict each other.features/enhanced-mode.mdxgets a short section saying whenautoescalates to enhanced proxies, which was not stated on any public page.Why
Report DI-2026-09-04-WEEKLY, findings OB-03 and OB-06, with the Sep-5 protected-target finding in DI-2026-09-05-WEEKLY.
billing.mdx:50said "Credits are charged whenever Firecrawl's infrastructure processes a request, even if the target site returns an HTTP error status code such as 403 Forbidden or 404 Not Found." Nine lines earlier,billing.mdx:41said 5 credits are billed "instead of the usual 0 for a failed scrape". The pricing page FAQ says "No, we only charge for successful requests." Three surfaces, three different answers.Code says the rule is a split, not a flat one.
apps/api/src/lib/scrape-billing.ts:99bills 0 when the document is null.scrape-billing.ts:140bills 1 credit when a document came back.apps/api/src/scraper/scrapeURL/index.ts:746makes any non-2xx status count as a returned document, so a 403 or 404 the target actually answered bills 1 credit, while a scrape that produced nothing bills 0. Six cases bill with no document at all, and none of them were published: threat protection scans, the prompt injection guard, FIRE-1, alockdowncache miss, Monitor error pages, and time-billed Interact and Browser sessions.Separately,
features/enhanced-mode.mdxsaidauto"retries with enhanced on failure" without saying what counts as a failure.apps/api/src/scraper/scrapeURL/index.ts:748escalates only on 401, 403 or 429, only whenproxyisauto, and only once per request. A user seeing a 404 cannot currently tell that no retry was attempted, or why.Changes
billing.mdx: rewrite the "When credits are charged" section as the split rule plus a table of the six cases that bill without a document. Point the prompt injection row at that section instead of restating half the rule.features/enhanced-mode.mdx: add a "Whenautoescalates" section with the trigger status codes, the once-per-request limit, that an explicitproxyturns escalation off, and that the escalated retry is not charged separately. The existing no-surcharge fact is kept and reworded into that section.English source only. Locale copies are generated, per the repo guideline.
Verification
mintlify broken-links: 208 broken links, byte-identical to the same run onorigin/main. None in the two changed files.mintlify validate: 17 warnings, all missing snippet imports, byte-identical to the same run onorigin/main. None in the two changed files.curl -sSI https://docs.firecrawl.dev/features/stealth-modereturns 308 to/features/enhanced-mode, so the old path still lands on the updated page.Not in this PR
FAILED_REQUEST_POLICYin the open firecrawl-web PR #3494 carries the unqualified claim intollms.txt. Left untouched. The one-line wording change is written up separately for whoever owns that PR.🤖 Generated with Claude Code
https://claude.ai/code/session_01SL5chNkWnr4Gy6uuB8PeKS