Initial release: AI-powered jq filter synthesis

Author: nulone

.gitignore

@@ -10,3 +10,4 @@ dist/
 build/
 .pytest_cache/
 .claude/
+*.zip

Archive.zip

@@ -0,0 +1,21 @@
+MIT License
+
+Copyright (c) 2026 nulone
+
+Permission is hereby granted, free of charge, to any person obtaining a copy
+of this software and associated documentation files (the "Software"), to deal
+in the Software without restriction, including without limitation the rights
+to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
+copies of the Software, and to permit persons to whom the Software is
+furnished to do so, subject to the following conditions:
+
+The above copyright notice and this permission notice shall be included in all
+copies or substantial portions of the Software.
+
+THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
+AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
+OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
+SOFTWARE.

LICENSE

@@ -8,6 +8,8 @@ JQ-Synth automatically generates [jq](https://stedolan.github.io/jq/) filter exp
 [![Python 3.10+](https://img.shields.io/badge/python-3.10+-blue.svg)](https://www.python.org/downloads/)
 [![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)](https://opensource.org/licenses/MIT)
 
+![Demo](demo.gif)
+
 ## Overview
 
 JQ-Synth solves a common developer problem: you know what JSON transformation you want, but writing the correct jq filter is tricky. Simply provide example input/output pairs, and JQ-Synth will synthesize the filter for you.
@@ -48,64 +50,6 @@ source .venv/bin/activate  # On Windows: .venv\Scripts\activate
 pip install -e .
 ```
 
-## Supported Providers
-
-| Provider | Status | Note |
-|----------|--------|------|
-| OpenAI | Stable ✅ | Default, tested |
-| Anthropic | Beta ⚠️ | May have edge cases |
-| OpenRouter | Beta ⚠️ | Via OpenAI-compatible endpoint |
-| Ollama | Alpha 🧪 | Local only, requires setup |
-
-> Note: OpenAI is default and most tested. Others should work but report issues if found.
-
-### Provider Setup
-
-**OpenAI (Default)**
-
-```bash
-export OPENAI_API_KEY='sk-...'
-# Optional: specify model (default: gpt-4o)
-export LLM_MODEL='gpt-4o'
-```
-
-**Anthropic**
-
-```bash
-export LLM_PROVIDER='anthropic'
-export ANTHROPIC_API_KEY='sk-ant-...'
-# Optional: specify model (default: claude-sonnet-4-20250514)
-export LLM_MODEL='claude-sonnet-4-20250514'
-```
-
-**OpenRouter**
-
-```bash
-export LLM_BASE_URL='https://openrouter.ai/api/v1'
-export OPENAI_API_KEY='sk-or-...'
-export LLM_MODEL='anthropic/claude-3.5-sonnet'
-```
-
-**Local (Ollama)**
-
-```bash
-export LLM_BASE_URL='http://localhost:11434/v1'
-export LLM_MODEL='llama3'
-export OPENAI_API_KEY='dummy'  # Ollama doesn't require a real key
-```
-
-**Together AI / Groq**
-
-```bash
-# Together AI
-export LLM_BASE_URL='https://api.together.xyz/v1'
-export OPENAI_API_KEY='...'
-
-# Groq
-export LLM_BASE_URL='https://api.groq.com/openai/v1'
-export OPENAI_API_KEY='gsk_...'
-```
-
 ## Quick Start
 
 ### Interactive Mode
@@ -245,6 +189,19 @@ jq-synth --base-url https://openrouter.ai/api/v1 --model anthropic/claude-3.5-so
 jq-synth --base-url http://localhost:11434/v1 --model llama3 --task nested-field
 ```
 
+## How It Works
+
+JQ-Synth uses a **deterministic oracle** approach:
+
+1. **Generation**: An LLM (GPT-4, Claude, or compatible model) generates candidate jq filters based on your examples and description
+2. **Verification**: Each filter is executed against the real jq binary with your input examples
+3. **Scoring**: A deterministic algorithm compares actual vs expected outputs, computing similarity scores (0.0 to 1.0)
+4. **Feedback**: The algorithm classifies errors (syntax, shape, missing/extra elements, order) and generates actionable feedback
+5. **Refinement**: The LLM receives the feedback and generates an improved filter
+6. **Iteration**: Steps 2-5 repeat until a perfect match is found or limits are reached
+
+This hybrid approach combines LLM creativity with deterministic verification, ensuring correctness while leveraging AI for filter synthesis.
+
 ## Architecture
 
 JQ-Synth follows a modular architecture with clear separation of concerns:
@@ -349,6 +306,104 @@ The reviewer classifies errors by priority (highest to lowest):
 - **Scalars**: Binary (1.0 for exact match, 0.0 for mismatch)
 - **Multiple examples**: Arithmetic mean of scores
 
+## Supported jq Patterns
+
+JQ-Synth works well with these common jq operations:
+
+- **Field extraction**: `.foo`, `.user.name`, `.data.items[0]`
+- **Array operations**: `.[]`, `.[0]`, `.[1:3]`, `.[-1]`
+- **Filtering**: `select(.active == true)`, `select(.age > 18)`
+- **Mapping**: `map(.name)`, `[.[] | .id]`
+- **Array construction**: `[.items[].name]`
+- **Object construction**: `{name: .user.name, email: .user.email}`
+- **Conditionals**: `if .status == "active" then .name else null end`
+- **Null handling**: `select(. != null)`, `.field // "default"`
+- **String operations**: String interpolation, concatenation
+- **Arithmetic**: Addition, subtraction, comparison operators
+- **Type checking**: `type`, `length`
+
+## Known Limitations
+
+JQ-Synth may struggle with these advanced jq features:
+
+- **Aggregations**: `group_by()`, `reduce`, `min_by()`, `max_by()`
+- **Complex recursion**: `recurse()`, `walk()`
+- **Variable bindings**: Complex `as $var` patterns
+- **Custom functions**: `def` statements (blocked for security)
+- **Advanced array operations**: `combinations()`, `transpose()`
+- **Path manipulation**: `getpath()`, `setpath()`, `delpaths()`
+- **Format strings**: `@csv`, `@json`, `@base64`
+
+For these cases, you may need to write the filter manually or break down the task into simpler steps.
+
+## Model recommendations
+
+| Task complexity | Recommended model | Speed |
+|-----------------|-------------------|-------|
+| Simple filters (extract, select) | GPT-4o-mini, Claude Haiku | Fast |
+| Medium (grouping, aggregation, recursion) | Claude Sonnet, GPT-4o | Fast |
+| Complex algorithms (graph traversal, sorting) | DeepSeek R1 | Slow (minutes) |
+
+> Note: DeepSeek R1 solved topological sort and Dijkstra's shortest path in jq. Most users won't need this — standard models handle 95%+ of real-world tasks.
+
+## Supported Providers
+
+| Provider | Status | Note |
+|----------|--------|------|
+| OpenAI | Stable ✅ | Default provider |
+| Anthropic | Beta ⚠️ | Different API format |
+| OpenRouter | Tested ✅ | OpenAI-compatible |
+| Ollama | Alpha 🧪 | Local only, requires setup |
+
+> Note: OpenAI is default and most tested. Others should work but report issues if found.
+
+### Provider Setup
+
+**OpenAI (Default)**
+
+```bash
+export OPENAI_API_KEY='sk-...'
+# Optional: specify model (default: gpt-4o)
+export LLM_MODEL='gpt-4o'
+```
+
+**Anthropic**
+
+```bash
+export LLM_PROVIDER='anthropic'
+export ANTHROPIC_API_KEY='sk-ant-...'
+# Optional: specify model (default: claude-sonnet-4-20250514)
+export LLM_MODEL='claude-sonnet-4-20250514'
+```
+
+**OpenRouter**
+
+```bash
+export LLM_BASE_URL='https://openrouter.ai/api/v1'
+export OPENAI_API_KEY='sk-or-...'
+export LLM_MODEL='anthropic/claude-3.5-sonnet'
+```
+
+**Local (Ollama)**
+
+```bash
+export LLM_BASE_URL='http://localhost:11434/v1'
+export LLM_MODEL='llama3'
+export OPENAI_API_KEY='dummy'  # Ollama doesn't require a real key
+```
+
+**Together AI / Groq**
+
+```bash
+# Together AI
+export LLM_BASE_URL='https://api.together.xyz/v1'
+export OPENAI_API_KEY='...'
+
+# Groq
+export LLM_BASE_URL='https://api.groq.com/openai/v1'
+export OPENAI_API_KEY='gsk_...'
+```
+
 ## Task File Format
 
 Tasks are defined in JSON format:
@@ -604,7 +659,7 @@ mypy src && \
 pytest -m "not e2e"
 ```
 
-### Project Structure
+## Project Structure
 
 ```
 jq-synth/
@@ -650,7 +705,7 @@ Contributions are welcome! Please follow these steps:
 6. **Push** to your fork: `git push origin feature/my-feature`
 7. **Open** a Pull Request
 
-### Code Style
+## Code Style
 
 - Type hints required for all public functions
 - Docstrings required for all public functions and classes (Google style)
@@ -669,49 +724,6 @@ MIT License - see [LICENSE](LICENSE) for details.
 - [OpenAI](https://openai.com) - GPT models and API
 - [Anthropic](https://anthropic.com) - Claude models and API
 
-## Supported jq Patterns
-
-JQ-Synth works well with these common jq operations:
-
-- **Field extraction**: `.foo`, `.user.name`, `.data.items[0]`
-- **Array operations**: `.[]`, `.[0]`, `.[1:3]`, `.[-1]`
-- **Filtering**: `select(.active == true)`, `select(.age > 18)`
-- **Mapping**: `map(.name)`, `[.[] | .id]`
-- **Array construction**: `[.items[].name]`
-- **Object construction**: `{name: .user.name, email: .user.email}`
-- **Conditionals**: `if .status == "active" then .name else null end`
-- **Null handling**: `select(. != null)`, `.field // "default"`
-- **String operations**: String interpolation, concatenation
-- **Arithmetic**: Addition, subtraction, comparison operators
-- **Type checking**: `type`, `length`
-
-## Known Limitations
-
-JQ-Synth may struggle with these advanced jq features:
-
-- **Aggregations**: `group_by()`, `reduce`, `min_by()`, `max_by()`
-- **Complex recursion**: `recurse()`, `walk()`
-- **Variable bindings**: Complex `as $var` patterns
-- **Custom functions**: `def` statements (blocked for security)
-- **Advanced array operations**: `combinations()`, `transpose()`
-- **Path manipulation**: `getpath()`, `setpath()`, `delpaths()`
-- **Format strings**: `@csv`, `@json`, `@base64`
-
-For these cases, you may need to write the filter manually or break down the task into simpler steps.
-
-## How It Works
-
-JQ-Synth uses a **deterministic oracle** approach:
-
-1. **Generation**: An LLM (GPT-4, Claude, or compatible model) generates candidate jq filters based on your examples and description
-2. **Verification**: Each filter is executed against the real jq binary with your input examples
-3. **Scoring**: A deterministic algorithm compares actual vs expected outputs, computing similarity scores (0.0 to 1.0)
-4. **Feedback**: The algorithm classifies errors (syntax, shape, missing/extra elements, order) and generates actionable feedback
-5. **Refinement**: The LLM receives the feedback and generates an improved filter
-6. **Iteration**: Steps 2-5 repeat until a perfect match is found or limits are reached
-
-This hybrid approach combines LLM creativity with deterministic verification, ensuring correctness while leveraging AI for filter synthesis.
-
 ---
 
 **JQ-Synth** - Because life's too short to debug jq filters manually.