Implement maxAge fast scraping parameter

[devin-ai-integration]

Implement maxAge fast scraping parameter

Summary

This PR implements the maxAge parameter for the firecrawl-mcp-server to enable faster scraping through caching, addressing GitHub issue #69. The implementation exposes the existing Firecrawl API maxAge parameter through the MCP server's tool schema.

Key changes:

• Added maxAge parameter to SCRAPE_TOOL inputSchema as optional number field
• Updated tool description and usage examples to highlight caching benefits
• Added test coverage to verify parameter is passed through to Firecrawl API
• Merged latest main branch changes (version bump to 1.11.0)

The maxAge parameter allows users to specify a cache duration in milliseconds. When set, the system will use cached content if available and younger than the specified age, otherwise scrape fresh content.

Review & Testing Checklist for Human

• Test maxAge with real Firecrawl API calls - Verify that setting maxAge actually enables caching behavior (most critical)
• Validate performance claims - Test whether maxAge actually provides significant speed improvements as claimed
• Test edge cases - Try invalid maxAge values (negative, non-numeric) to ensure proper error handling
• Verify backwards compatibility - Ensure existing scrape calls without maxAge parameter continue working

Recommended test plan: Create a test script that scrapes the same URL twice with maxAge set, verify the second call is faster and returns cached content.

---

Diagram

%%{ init : { "theme" : "default" }}%%
graph TD
    subgraph "MCP Server Implementation"
        IndexTS["src/index.ts"]:::major-edit
        IndexTestTS["src/index.test.ts"]:::major-edit
    end
    
    subgraph "Tool Schema"
        SCRAPE_TOOL["SCRAPE_TOOL definition"]:::major-edit
        InputSchema["inputSchema.properties"]:::major-edit
    end
    
    subgraph "External Dependencies"
        FirecrawlSDK["@mendable/firecrawl-js"]:::context
        ScrapeParams["ScrapeParams type"]:::context
    end
    
    IndexTS --> SCRAPE_TOOL
    SCRAPE_TOOL --> InputSchema
    InputSchema --> |"maxAge: number"| FirecrawlSDK
    IndexTestTS --> |"tests maxAge passing"| FirecrawlSDK
    FirecrawlSDK --> ScrapeParams
    
    subgraph Legend
        L1["Major Edit"]:::major-edit
        L2["Minor Edit"]:::minor-edit  
        L3["Context/No Edit"]:::context
    end

    classDef major-edit fill:#90EE90
    classDef minor-edit fill:#87CEEB
    classDef context fill:#FFFFFF

Loading

Notes

• Implementation relies on existing Firecrawl SDK ScrapeParams type to handle maxAge validation
• The parameter is optional and should default to 0 (always scrape fresh) per Firecrawl API behavior
• Performance improvement claims (500% faster) are based on issue description but not independently verified
• Session URL: https://app.devin.ai/sessions/49a52e8dbd37423ca390018a20461749
• Requested by: @nickscamara

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[nickscamara]

closes #69