Skip to content

Commit 002ff19

Browse files
agammagamm
committed
Add llms.txt
1 parent 5360925 commit 002ff19

File tree

3 files changed

+194
-5
lines changed

3 files changed

+194
-5
lines changed

‎README.md‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -396,6 +396,10 @@ job.stats(print_stats=True)
396396

397397
MIT
398398

399+
## AI Documentation
400+
401+
📋 **For AI systems**: See [llms.txt](llms.txt) for comprehensive documentation optimized for AI consumption.
402+
399403
## Todos
400404

401405
- [x] ~~Add pricing metadata and max_spend controls~~ (Cost tracking implemented)

‎llms.txt‎

Lines changed: 164 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,164 @@
1+
# Batchata
2+
3+
> Python SDK for AI batch processing with structured output and citation mapping. 50% cost savings via Anthropic's batch API with automatic cost tracking, structured output using Pydantic models, and field-level citations.
4+
5+
Batchata is a Python library that provides a simple interface for batch processing with AI models (currently supports Anthropic Claude, OpenAI support coming soon). **The preferred way to use Batchata is through `BatchManager`** as it abstracts most of the work of the lower-level `batch()` function and provides advanced features like parallel processing, state persistence, and cost management.
6+
7+
## Recommended Usage Pattern
8+
9+
**Use `BatchManager` for production workloads** - it handles job splitting, parallel execution, state persistence, cost limits, and retry logic automatically. Only use the lower-level `batch()` function for simple one-off tasks or when you need direct control over the batch processing.
10+
11+
## Supported Models
12+
13+
**Claude 4 Models (Latest & Best Performance):**
14+
- `claude-opus-4-20250514` ⭐ **Best overall performance**
15+
- `claude-sonnet-4-20250514` ⭐ **Best performance for most tasks**
16+
17+
**Claude 3.7 Models:**
18+
- `claude-3-7-sonnet-20250219` (also available as `claude-3-7-sonnet-latest`)
19+
20+
**Claude 3.5 Models:**
21+
- `claude-3-5-sonnet-20241022` (also available as `claude-3-5-sonnet-latest`)
22+
- `claude-3-5-sonnet-20240620`
23+
- `claude-3-5-haiku-20241022` (also available as `claude-3-5-haiku-latest`) - Fast, cost-effective
24+
25+
**Claude 3 Models:**
26+
- `claude-3-haiku-20240307` - Most cost-effective option
27+
28+
**Legacy Models (Deprecated):**
29+
- `claude-3-opus-20240229`
30+
- `claude-3-sonnet-20240229`
31+
- `claude-3-5-haiku-20240307`
32+
33+
**⭐ For best performance, use Claude Sonnet 4 or Opus 4 models for complex tasks, PDF processing, and structured output. These models offer the highest accuracy and capability.**
34+
35+
All models support batch processing with 50% cost savings. PDF/file processing requires file-capable models (all models except claude-3-haiku-20240307 support file input).
36+
37+
## Core API Documentation
38+
39+
- [Main README](README.md): Complete documentation with installation, usage examples, and API reference
40+
- [Core Implementation](src/core.py): Lower-level batch() function implementation and PDF processing utilities
41+
- [Batch Manager](src/batch_manager.py): **Recommended approach** - Large-scale batch processing with parallel execution, state persistence, and cost management
42+
- [Batch Job](src/batch_job.py): Individual batch job handling and status management
43+
- [Citations](src/citations.py): Citation data structures and field-level citation mapping
44+
- [Types](src/types.py): Type definitions and data structures used throughout the library
45+
- [Utilities](src/utils.py): Helper functions for batch processing operations
46+
47+
## Examples and Usage Patterns
48+
49+
- [Batch Manager Example](examples/batch_manager_example.py): **Recommended** - Large-scale processing with parallel execution and state management
50+
- [Spam Detection Example](examples/spam_detection.py): Email classification using structured output with confidence scores
51+
- [PDF Extraction Example](examples/pdf_extraction.py): Extract structured data from PDF invoices with citations
52+
- [Citation Example](examples/citation_example.py): Basic citation usage for text analysis
53+
- [Citation with Pydantic](examples/citation_with_pydantic.py): Field-level citations with structured output models
54+
- [Raw Text Example](examples/raw_text_example.py): Simple text processing without structured output
55+
56+
## Provider Architecture
57+
58+
- [Base Provider](src/providers/base.py): Abstract base class for AI providers with batch processing interface
59+
- [Anthropic Provider](src/providers/anthropic.py): Anthropic Claude implementation with batch API support and model definitions
60+
- [Provider Registry](src/providers/__init__.py): Provider selection and initialization utilities
61+
62+
## Key Features and Implementation Details
63+
64+
### BatchManager (Recommended Approach)
65+
- **Automatic Job Splitting**: Breaks large batches into configurable chunks (items_per_job)
66+
- **Parallel Processing**: Concurrent job execution with ThreadPoolExecutor (max_parallel_jobs)
67+
- **State Persistence**: JSON-based state files for resume capability after interruptions
68+
- **Cost Management**: Stop processing when budget limits are reached (max_cost parameter)
69+
- **Progress Monitoring**: Real-time progress updates with statistics and cost tracking
70+
- **Retry Mechanism**: Built-in retry for failed items with exponential backoff
71+
- **Result Management**: Organized directory structure for saving and loading results
72+
73+
### Batch Processing Features
74+
- **Cost Optimization**: 50% cost savings through Anthropic's batch API pricing
75+
- **Structured Output**: Full Pydantic model support with automatic validation
76+
- **Citation Mapping**: Field-level citations that map results to source documents
77+
- **Cost Tracking**: Automatic token usage and cost calculation using tokencost library
78+
- **Type Safety**: Full TypeScript-style type annotations and validation
79+
80+
### Citation System
81+
- **Text + Citations Mode**: Flat list of citations for unstructured text responses
82+
- **Structured + Field Citations**: Citations mapped to specific Pydantic model fields
83+
- **Robust JSON Parsing**: Handles complex JSON structures with escaped quotes, nested objects, and special characters
84+
- **Page-Level Citations**: Precise document location tracking with page numbers and text spans
85+
86+
### Response Formats
87+
- **Unified Format**: Consistent `{"result": ..., "citations": ...}` structure across all modes
88+
- **BatchManager Summary**: Processing summary with `total_items`, `completed_items`, `failed_items`, `total_cost`, `jobs_completed`, `cost_limit_reached`
89+
- **Results Loading**: `get_results_from_disk()` for retrieving individual results from saved files
90+
91+
## Installation and Setup
92+
93+
**Installation**: `pip install batchata`
94+
95+
**Environment Setup**: Requires `ANTHROPIC_API_KEY` environment variable
96+
97+
**Python Version**: Requires Python 3.12+
98+
99+
**Dependencies**:
100+
- `anthropic>=0.57.1` for Claude API access
101+
- `python-dotenv>=1.1.1` for environment management
102+
- `tokencost>=0.1.24` for cost tracking
103+
104+
## Testing and Development
105+
106+
- [Test Suite](tests/): Comprehensive test coverage including unit, integration, and e2e tests
107+
- [Test Fixtures](tests/fixtures.py): Reusable test utilities and mock data
108+
- [PDF Test Utils](tests/utils/pdf_utils.py): PDF generation utilities for testing
109+
- [E2E Tests](tests/e2e/): End-to-end integration tests with real API calls
110+
111+
## Configuration and Customization
112+
113+
### BatchManager Parameters (Recommended)
114+
- `items_per_job`: Number of items to process per batch job (default: 50)
115+
- `max_parallel_jobs`: Maximum concurrent jobs (default: 10)
116+
- `max_cost`: Budget limit to stop processing (default: None)
117+
- `max_wait_time`: Maximum wait time for job completion (default: 3600 seconds)
118+
- `state_path`: Path to JSON state file for persistence
119+
- `save_results_dir`: Directory to save processed results
120+
121+
### Batch Function Parameters (Lower-level)
122+
- `messages`: List of message conversations for chat-based processing
123+
- `files`: List of PDF file paths or bytes for document processing
124+
- `prompt`: Processing instruction (required for file processing)
125+
- `model`: AI model identifier (recommend: "claude-sonnet-4-20250514")
126+
- `response_model`: Optional Pydantic model for structured output
127+
- `enable_citations`: Boolean to enable citation extraction
128+
- `raw_results_dir`: Directory to save raw API responses
129+
130+
## Error Handling and Limitations
131+
132+
- **Citation Limitations**: Only works with flat Pydantic models (no nested models)
133+
- **Model Requirements**: PDFs require file-capable models (use Sonnet 4/Opus 4 for best results)
134+
- **Batch Timing**: Jobs can take up to 24 hours to process
135+
- **Cost Limits**: Best effort enforcement - final costs may slightly exceed max_cost
136+
- **Provider Support**: Currently Anthropic only, OpenAI support planned
137+
138+
## CLI Commands
139+
140+
- `batchata-example`: Run spam detection example
141+
- `batchata-pdf-example`: Run PDF extraction example
142+
143+
## Project Structure
144+
145+
```
146+
batchata/
147+
├── src/ # Source code
148+
│ ├── core.py # Lower-level batch() function
149+
│ ├── batch_manager.py # Recommended BatchManager class
150+
│ ├── batch_job.py # Individual job handling
151+
│ ├── citations.py # Citation data structures
152+
│ └── providers/ # AI provider implementations
153+
├── examples/ # Usage examples
154+
├── tests/ # Test suite
155+
└── specs/ # Feature specifications
156+
```
157+
158+
## Development Status
159+
160+
- **Version**: 0.2.2 (Alpha)
161+
- **License**: MIT
162+
- **Repository**: https://github.com/agamm/batchata
163+
- **PyPI**: https://pypi.org/project/batchata/
164+
- **Status**: Active development with regular updates

‎src/providers/anthropic.py‎

Lines changed: 26 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -21,22 +21,43 @@ class AnthropicBatchProvider(BaseBatchProvider):
2121

2222
# Supported models for this provider
2323
SUPPORTED_MODELS = {
24+
# Claude 4 models
25+
"claude-opus-4-20250514",
26+
"claude-sonnet-4-20250514",
27+
# Claude 3.7 models
28+
"claude-3-7-sonnet-20250219",
29+
"claude-3-7-sonnet-latest",
30+
# Claude 3.5 models
2431
"claude-3-5-sonnet-20241022",
25-
"claude-3-5-haiku-20241022",
32+
"claude-3-5-sonnet-latest",
33+
"claude-3-5-sonnet-20240620",
34+
"claude-3-5-haiku-20241022",
35+
"claude-3-5-haiku-latest",
36+
# Claude 3 models
37+
"claude-3-haiku-20240307",
38+
# Legacy models (deprecated)
2639
"claude-3-opus-20240229",
2740
"claude-3-sonnet-20240229",
28-
"claude-3-haiku-20240307",
29-
"claude-3-5-sonnet-20240620",
3041
"claude-3-5-haiku-20240307",
3142
}
3243

3344
# Models that support file/document input (PDFs, images, etc.)
3445
FILE_CAPABLE_MODELS = {
46+
# Claude 4 models
47+
"claude-opus-4-20250514",
48+
"claude-sonnet-4-20250514",
49+
# Claude 3.7 models
50+
"claude-3-7-sonnet-20250219",
51+
"claude-3-7-sonnet-latest",
52+
# Claude 3.5 models
3553
"claude-3-5-sonnet-20241022",
36-
"claude-3-5-haiku-20241022",
54+
"claude-3-5-sonnet-latest",
55+
"claude-3-5-sonnet-20240620",
56+
"claude-3-5-haiku-20241022",
57+
"claude-3-5-haiku-latest",
58+
# Legacy models (deprecated)
3759
"claude-3-opus-20240229",
3860
"claude-3-sonnet-20240229",
39-
"claude-3-5-sonnet-20240620",
4061
"claude-3-5-haiku-20240307",
4162
}
4263

0 commit comments

Comments
 (0)