Merge pull request #13 from vstorm-co/feature/issue#9

issue #9 resolved

Author: pawelkiszczak

.gitignore

@@ -43,3 +43,5 @@ build/
 # Custom
 *.db
 .ruff_cache
+ocr_parsing/files/results
+ocr_parsing/files/temp_files

README.md

@@ -1,5 +1,41 @@
 # PydanticAI Examples
 
+A comprehensive collection of examples demonstrating PydanticAI framework capabilities, from basic model requests to advanced document processing with schema validation.
+
+## Prerequisites
+
+### System Requirements
+
+- Python 3.10+
+- `uv` package manager
+
+### Environment Setup
+
+```bash
+# Install dependencies
+uv sync
+
+# Create .env file with your API key
+echo "OPENAI_API_KEY=your-key-here" > .env
+```
+
+**Note**: Most examples use OpenAI's GPT-5.1. Ensure your API key has appropriate permissions and sufficient quota.
+
+## Learning Path
+
+**Recommended order for learning PydanticAI**:
+
+1. **[Direct Model Requests](direct_model_request/)** - Understand basic LLM API calls
+2. **[Temperature](temperature/)** - Understand model parameters
+3. **[Reasoning Effort](reasoning_effort/)** - Uncover how the reasoning effort may change the model's output
+4. **[Basic Sentiment](basic_sentiment/)** - Learn structured outputs with Pydantic
+5. **[Dynamic Classification](dynamic_classification/)** - Runtime schema generation
+6. **[Bielik](bielik_example/)** - Local models and tools
+7. **[History Processor](history_processor/)** - Multi-turn conversations
+8. **[OCR Parsing](ocr_parsing_demo/)** - Complex real-world document processing
+
+## Examples Overview
+
 ### 1. Direct Model Requests
 
 **Location**: `direct_model_request/`
@@ -32,6 +68,7 @@ Demonstrates reasoning_effort parameter for gpt-5.2.
 
 - Control depth of internal reasoning
 - Complex problem-solving examples
+- Trade-off between accuracy and latency
 
 [View Example →](reasoning_effort/)
 
@@ -71,6 +108,36 @@ Learn to use **Bielik**, a Polish language LLM, with PydanticAI running locally
 
 [View Example →](bielik_example/)
 
+### 7. Conversation History Management
+
+**Location**: `history_processor/`
+
+Learn how to manage conversation history in AI agents.
+
+- Basic history handling and inspection
+- Multi-turn conversations with context awareness
+- History persistence (JSON and Database)
+- Advanced filtering and transformation
+- Context window management strategies (fixed, dynamic, and tool-aware)
+- Production-ready database archival
+
+[View Example →](history_processor/)
+
+### 8. OCR Parsing with Data Validation
+
+**Location**: `ocr_parsing/`
+
+Learn how to work with documents using PydanticAI for OCR (Optical Character Recognition).
+
+- **Basic OCR**: Unstructured text extraction from PDFs
+- **Structured Output**: Type-safe document analysis with schema validation
+- **Validation Errors**: Error handling when LLM output doesn't match schema
+- PDF to image conversion pipeline
+- Parallel async processing with concurrency control
+- Production-ready document processing patterns
+
+[View Example →](ocr_parsing/)
+
 ## Quick Start
 
 ### Setup
@@ -107,82 +174,100 @@ cd dynamic_classification
 uv run dynamic_classifier.py
 
 # Bielik local model examples
+# Note: Requires Ollama setup (see bielik_example/README.md)
 cd bielik_example
-uv run python bielik_basic_inference.py
-uv run python bielik_basic_tools.py
+uv run bielik_basic_inference.py
+uv run bielik_basic_tools.py
+
+# History processor - Run individual examples
+cd history_processor
+uv run 1_basic_history_handling.py
+uv run 2_continuous_history.py
+uv run 3_history_usage.py
+uv run 4_history_filtering.py
+uv run 5a_history_length_fixed.py
+uv run 5b_history_length_dynamic.py
+uv run 5c_history_with_tools.py
+uv run 6_persistent_history.py
+
+# OCR Parsing - Run examples in order
+cd ocr_parsing
+uv run 1_basic_ocr_demo.py
+uv run 2_ocr_with_structured_output.py
+uv run 3_ocr_validation.py  # Uncomment validation line in code first
 ```
 
-### 6. History Processor
+## Key Concepts Demonstrated
 
-**Location**: `history_processor/`
+### Agents
 
-Learn how to manage conversation history in AI agents.
+Most examples use PydanticAI's `Agent` class, which wraps an LLM with:
 
-- Basic history handling and inspection
-- Multi-turn conversations with context
-- History persistence (JSON and Database)
-- Advanced filtering and transformation
-- Context window management strategies (fixed, dynamic, and tool-aware)
-- Production-ready database archival
+- System prompts to guide behavior
+- Output type schemas for structured responses
+- Async/await support for concurrent requests
 
-[View Example →](history_processor/)
+### Tools
 
-## Quick Start
+It's worth noticing that since those are examples, most of them are pretty basic. However, it's easy to add an a tool for given agent. Let's look at **[OCR Parsing](ocr_parsing/) code.
 
-### Setup - General
+Currently the Agent does all the work itself - classifies document, parses the output, does the OCR and so on for every document in the same way. But what if we'd like to have a different behavior based on the document type?
 
-```bash
-# Install dependencies
-uv sync
+```python
+from pydantic_ai import Agent, RunContext
+from my_schemas import OCRInvoiceOutput, ReportOcrOutput
 
-# Set API key
-echo "OPENAI_API_KEY=your-key" > .env
+# The Agent acts as a router, deciding which tool to call
+# based on the document's visual or textual cues.
+agent = Agent(
+    'openai:gpt-5.1',
+    system_prompt="Analyze the document and use the appropriate tool for parsing."
+)
+
+@agent.tool
+async def parse_invoice(ctx: RunContext[MyDeps], data: bytes) -> OCRInvoiceOutput:
+    """Use this tool when the document is identified as an Invoice."""
+    # Your specialized OCR & validation logic here
+    return await ctx.deps.ocr_service.process(data, schema=OCRInvoiceOutput)
+
+@agent.tool
+async def parse_report(ctx: RunContext[MyDeps], data: bytes) -> ReportOcrOutput:
+    """Use this tool when the document is a multi-page Annual Report."""
+    # Custom logic for complex reports
+    return await ctx.deps.ocr_service.process(data, schema=ReportOcrOutput)
 ```
 
-### Run Examples 1-5
+### Structured Outputs
 
-```bash
-# Direct model requests
-cd direct_model_request
-uv run direct_request_demo.py
+Examples show how to enforce type safety using Pydantic `BaseModel`:
 
-# Model parameters
-cd temperature
-uv run temperature_demo.py
+- Basic classification: `Literal` types
+- Dynamic classification: `create_model()` for runtime schemas
+- OCR parsing: Complex nested schemas with validation
 
-# Reasoning effort
-cd reasoning_effort
-uv run reasoning_demo.py
+### Async Concurrency
 
-# Basic sentiment classifier
-cd basic_sentiment
-uv run sentiment_classifier.py
+Several examples demonstrate async patterns:
 
-# Dynamic classifier
-cd dynamic_classification
-uv run dynamic_classifier.py
-```
+- Parallel processing with `asyncio.gather()`
+- Semaphore-based rate limiting
+- Efficient handling of multiple documents
 
-### Run Example 6 - History Processor
+### Context & History
 
-```bash
-cd history_processor
+Learn how to manage conversational context:
 
-# Configure environment
-cp .env.example .env
-# Edit .env and add your OpenAI API key (i.e., with `nano`)
-nano .env
-
-# Run individual examples
-uv run python 1_basic_history_handling.py
-uv run python 2_continuous_history.py
-uv run python 3_history_usage.py
-uv run python 4_history_filtering.py
-uv run python 5a_history_length_fixed.py
-uv run python 5b_history_length_dynamic.py
-uv run python 5c_history_with_tools.py
-uv run python 6_persistent_history.py
-```
+- Persistent history storage
+- Token-aware context windowing
+- History filtering and transformation
+
+### Local Models
+
+Bielik example shows alternative to cloud APIs:
+
+- Local model serving with Ollama
+- Custom tool integration
+- Same agent patterns as OpenAI models
 
 ## Project Structure
 
@@ -218,13 +303,54 @@ uv run python 6_persistent_history.py
 │   ├── 5c_history_with_tools.py
 │   ├── 6_persistent_history.py
 │   ├── README.md
-│   ├── .env.example
-│   └── pyproject.toml
+│   ├── output_3.json
+├── ocr_parsing/
+│   ├── 1_basic_ocr_demo.py
+│   ├── 2_ocr_with_structured_output.py
+│   ├── 3_ocr_validation.py
+│   ├── README.md
+│   ├── files/
+│   │   ├── samples/        # Sample PDF documents
+│   │   ├── temp_files/      # Temporary image files during processing
+│   │   ├── results/        # Output JSON files
 ├── pyproject.toml
 └── README.md
 ```
 
+## Common Issues & Troubleshooting
+
+### API Key Issues
+
+- Ensure `OPENAI_API_KEY` is set in `.env`
+- Verify key has appropriate permissions
+- Check for rate limiting (503 errors)
+
+### Import Errors
+
+- Run `uv sync` to install all dependencies
+- Verify you're using Python 3.10+
+
+### Async Issues
+
+- Some examples require async-compatible event loops
+- On Windows, you may need to set event loop policy
+
+### OCR-Specific Issues
+
+- **poppler not found**: Install via your package manager (brew/apt/choco)
+- **PDF conversion fails**: Ensure PDF is valid and readable
+- **Rate limiting**: Reduce semaphore value in `ocr_parsing/shared_fns.py`
+
+See individual example READMEs for specific setup requirements.
+
 ## Resources
 
-- [Pydantic AI Documentation](https://ai.pydantic.dev/)
-- [Python Documentation](https://docs.python.org/)
+- [Python Documentation](https://docs.python.org/3/)
+- [PydanticAI Documentation](https://ai.pydantic.dev/)
+- [Pydantic Documentation](https://docs.pydantic.dev/)
+- [OpenAI API Reference](https://platform.openai.com/docs/api-reference)
+- [Python asyncio Guide](https://docs.python.org/3/library/asyncio.html)
+
+## Contributing
+
+Found an issue or have an improvement? Feel free to contribute to this example repository.

bielik_example/README.md

@@ -83,9 +83,9 @@ For the tools example, you'll need a free API key:
 2. Sign up for a free account
 3. Create a `.env` file in this directory:
 
-   ```
-   WEATHER_API_KEY=your_key_here
-   ```
+```bash
+WEATHER_API_KEY=your_key_here
+```
 
 ## Running the Examples
 

history_processor/1_basic_history_handling.py

@@ -19,7 +19,7 @@
 def main() -> None:
     """Run basic history inspection example."""
     # Create a basic agent
-    agent = Agent(model="openai:gpt-4o", system_prompt="Be a helpful assistant")
+    agent = Agent(model="openai:gpt-5.1", system_prompt="Be a helpful assistant")
 
     # Run a single inference
     prompt = "Tell me a funny joke. Respond in plain text."

history_processor/2_continuous_history.py

@@ -21,7 +21,7 @@
 def main() -> None:
     """Run multi-turn conversation example."""
     # Create agent
-    agent = Agent(model="openai:gpt-4o", system_prompt="Be a helpful assistant")
+    agent = Agent(model="openai:gpt-5.1", system_prompt="Be a helpful assistant")
 
     # First turn: Agent generates a joke
     prompt_1 = "Provide a really, really funny joke. Respond in plain text."

history_processor/3_history_usage.py

@@ -21,7 +21,7 @@
 def main() -> None:
     """Run multi-turn conversation with persistence example."""
     # Create agent
-    agent = Agent(model="openai:gpt-4o", system_prompt="Be a helpful assistant")
+    agent = Agent(model="openai:gpt-5.1", system_prompt="Be a helpful assistant")
 
     # Turn 1: Get initial motto
     log.info("\n=== Turn 1 ===")

history_processor/4_history_filtering.py

@@ -59,13 +59,13 @@ def main() -> None:
 
     # Example 1: Summarize only user messages
     log.info("\n=== Filtering: User Messages Only ===")
-    agent_user = Agent("openai:gpt-4o", history_processors=[user_message_filter])
+    agent_user = Agent("openai:gpt-5.1", history_processors=[user_message_filter])
     result_1 = agent_user.run_sync("Please summarize the whole chat history until now.", message_history=history)
     log.info(f"Summary (user messages only):\n{result_1.output}")
 
     # Example 2: Attempt to filter only model messages (will fail)
     log.info("\n=== Filtering: Model Messages Only ===")
-    agent_model = Agent("openai:gpt-4o", history_processors=[model_message_filter])
+    agent_model = Agent("openai:gpt-5.1", history_processors=[model_message_filter])
     try:
         result_2 = agent_model.run_sync("Please summarize the whole chat history until now.", message_history=history)
         log.info(f"Summary (model messages only):\n{result_2.output}")

history_processor/5a_history_length_fixed.py

@@ -51,7 +51,7 @@ def main() -> None:
 
     # Create agent with message count limiter
     log.info("\n=== Agent with Fixed Message Limit (last 3) ===")
-    agent_1 = Agent("openai:gpt-4o", history_processors=[keep_last_messages])
+    agent_1 = Agent("openai:gpt-5.1", history_processors=[keep_last_messages])
     result_1 = agent_1.run_sync("What were we talking about?", message_history=history)
     log.info(f"Answer (with truncated history):\n{result_1.output}")
 

history_processor/5b_history_length_dynamic.py

@@ -23,7 +23,7 @@
 # `tiktoken` is used for OpenAI models, therefore if you're going to
 # use different model provided, this bit will need to be changed
 # to different tokenizer that corresponding to model used
-tokenizer = tiktoken.encoding_for_model("gpt-4o")
+tokenizer = tiktoken.encoding_for_model("gpt-5.1")
 
 
 @dataclass
@@ -58,7 +58,7 @@ def estimate_tokens(messages: list[ModelMessage]) -> int:
 # of this example, threshold is set low for the logic to trigger. Usually,
 # this value is much bigger and corresponds to used model's context
 # window size. To fully utilize model processing capabilities it is best to
-# set this value close to context size. For `gpt-4o` model this value is
+# set this value close to context size. For `gpt-5.1` model this value is
 # equal to 128_000 tokens
 
 
@@ -100,7 +100,7 @@ def main() -> None:
 
     log.info("\n=== Agent with Dynamic Token-Based Context Guard ===")
     agent_2 = Agent(
-        "openai:gpt-4o",
+        "openai:gpt-5.1",
         deps_type=MemoryState,
         history_processors=[context_guard],
         system_prompt="You are a helpful and concise assistant.",