Skip to content

feat: Add Claude Code/Agent SDK as LLM provider #1700

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
chansearrington wants to merge 2 commits into
base: main
Choose a base branch
from
Open

feat: Add Claude Code/Agent SDK as LLM provider #1700

chansearrington wants to merge 2 commits into from
+1,457 −4

Conversation

@chansearrington
Copy link

@chansearrington chansearrington commented Jan 9, 2026 •
edited
Loading

Summary

Adds Claude Code/Agent SDK as an optional LLM provider via custom LiteLLM handler, enabling users with Claude Code subscriptions to leverage their existing CLI authentication for extraction tasks instead of needing separate API keys.

Changes

Core Implementation

  • New Provider: crawl4ai/providers/claude_code_provider.py - ClaudeCodeProvider wrapping Claude Agent SDK
  • Registration: crawl4ai/providers/__init__.py - Provider registration with LiteLLM
  • Config: Added claude-code to PROVIDER_MODELS_PREFIXES in config.py
  • Init: Auto-register providers on import in __init__.py
  • Dependency: Added claude-code optional extra to pyproject.toml

Documentation

  • README.md: Added Claude Code Provider section with installation and usage
  • CHANGELOG.md: Added feature entry under [Unreleased]
  • New Docs Page: docs/md_v2/extraction/claude-code-provider.md - Comprehensive documentation
  • mkdocs.yml: Added navigation entry for new docs page
  • llm-strategies.md: Added cross-reference to Claude Code Provider
  • CONTRIBUTING.md: New contributor guide for adding LLM providers

Code Quality

  • Exception Hierarchy: Added ClaudeCodeSDKError, ClaudeCodeAuthenticationError, ClaudeCodeConnectionError
  • Enhanced Error Messages: Recovery suggestions in error messages
  • Verbose Logging: New LLM provider/model logging when verbose=True
  • Enhanced Docstrings: Comprehensive module and class documentation

Tests & Examples

  • Unit Tests: 21 tests in tests/unit/test_claude_code_provider.py
  • Integration Tests: 8 tests in tests/integration/test_claude_code_integration.py
  • Examples: examples/claude_code_extraction.py, examples/claude_code_error_handling.py

Usage

from crawl4ai import AsyncWebCrawler
from crawl4ai.async_configs import LLMConfig, CrawlerRunConfig
from crawl4ai.extraction_strategy import LLMExtractionStrategy

# No API key needed - uses local Claude Code CLI auth
llm_config = LLMConfig(provider="claude-code/claude-sonnet-4-20250514")

strategy = LLMExtractionStrategy(
    llm_config=llm_config,
    instruction="Extract all product names and prices as JSON",
    verbose=True  # Shows: [LOG] LLM Provider: claude-code | Model: claude-sonnet-4-20250514
)

run_config = CrawlerRunConfig(extraction_strategy=strategy)

async with AsyncWebCrawler() as crawler:
    result = await crawler.arun(url="https://example.com", config=run_config)

Supported Models

Model Provider String Use Case
Sonnet 4 claude-code/claude-sonnet-4-20250514 Balanced (recommended)
Opus 4 claude-code/claude-opus-4-20250514 Most capable
Haiku 3.5 claude-code/claude-haiku-3-5-latest Fastest

Installation

pip install crawl4ai[claude-code]

Requires Claude Code CLI to be installed and authenticated (claude command available).

Test Plan

  • 21 unit tests pass (mocked, no SDK required)
  • 8 integration tests (4 pass, 4 skip when SDK not installed)
  • Backward compatibility verified (existing providers still work)
  • Verbose logging verified working in Docker deployment
  • Documentation builds correctly

🤖 Generated with Claude Code

chansearrington and others added 2 commits January 8, 2026 19:35
@chansearrington @claude
feat: Add Claude Code/Agent SDK as LLM provider
1460f52 
Add Claude Code/Agent SDK as an optional LLM provider via custom LiteLLM
handler, enabling users with Claude Code subscriptions to leverage their
existing CLI authentication for extraction tasks.

Features:
- New ClaudeCodeProvider wrapping Claude Agent SDK
- Support for Opus, Sonnet, and Haiku model selection
- Automatic provider registration on import
- No API keys needed - uses local Claude Code CLI auth

Usage:
  LLMConfig(provider="claude-code/claude-sonnet-4-20250514")

Install:
  pip install crawl4ai[claude-code]

Includes:
- 21 unit tests (mocked, run without SDK)
- 8 integration tests (4 require SDK)
- Example script in examples/claude_code_extraction.py

Co-Authored-By: Claude Opus 4.5 <[email protected]>
@chansearrington @claude
feat: Add comprehensive documentation and code quality improvements
db60a87 
Documentation:
- Add Claude Code Provider section to README.md
- Add CHANGELOG.md entry for Claude Code Provider feature
- Create dedicated docs page: docs/md_v2/extraction/claude-code-provider.md
- Update mkdocs.yml with navigation entry
- Add cross-reference in llm-strategies.md
- Create CONTRIBUTING.md for contributor guidance

Code Quality:
- Add exception hierarchy: ClaudeCodeSDKError, ClaudeCodeAuthenticationError, ClaudeCodeConnectionError
- Enhance module docstrings with requirements and exceptions
- Add LLM provider/model logging to verbose output (benefits all providers)
- Create error handling example: examples/claude_code_error_handling.py
- Update tests for new exception types

Co-Authored-By: Claude Opus 4.5 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@chansearrington