Skip to content

AGENTS.md

Latest commit

 

History

History
183 lines (129 loc) · 7.45 KB

AGENTS.md

File metadata and controls

183 lines (129 loc) · 7.45 KB
Raw

SEAD Authority Service - AI Coding Agent Instructions

Project Overview

FastAPI-based reconciliation service for SEAD (Strategic Environmental Archaeology Database) implementing the OpenRefine Reconciliation API. Core function: match fuzzy text queries to canonical entity identifiers from archaeological/environmental database.

Key architectural insight: This is a registry-based plugin system where reconciliation strategies auto-register via decorators and are looked up at runtime by entity type.

Critical Patterns

1. Strategy Registry Pattern

All entity reconciliation strategies must be decorated with @Strategies.register():

from src.strategies.strategy import ReconciliationStrategy, Strategies

@Strategies.register(key="site", repository_cls=SiteRepository)
class SiteReconciliationStrategy(ReconciliationStrategy):
    # Implementation

Why: Strategies auto-register on import. main.py imports src.strategies which triggers recursive module loading in src/strategies/__init__.py that instantiates all decorated classes.

2. Configuration Resolution with ConfigValue

Use ConfigValue for lazy config resolution instead of direct lookups:

from src.configuration import ConfigValue

# Lazy resolution with fallback
threshold = ConfigValue("options:auto_accept_threshold", default=0.90).resolve()

# Multi-path fallback (checks each path in order)
model = ConfigValue("llm.ollama.model,llm.model", default="llama3").resolve()

Location: src/configuration/resolve.py. Supports colon-separated paths (options:database:dbname) and comma-separated fallbacks.

3. Database Connection Singleton

Never create database connections directly. Use the config provider pattern:

from src.configuration import get_connection

async with await get_connection() as conn:
    async with conn.cursor() as cur:
        await cur.execute(query, params)

Connection is created once at startup via src/configuration/setup.py and stored in runtime config.

4. Schema Generation from Templates

Entity schemas are generated, not hand-written. To add a new entity:

  1. Add entity config to config/entities.yml
  2. Run make generate-schema (calls src/scripts/generate_entity_schema.py)
  3. Generated SQL appears in schema/generated/

Do not edit generated files directly. Edit schema/templates/ or config/entities.yml instead.

Development Workflows

Running the Service

make serve           # Development with auto-reload
make dev-serve       # Start uvicorn + OpenRefine together
make dev-stop        # Stop both services

Port: 8000 (uvicorn), 3333 (OpenRefine). PIDs written to uvicorn.pid and refine.pid.

Testing

make test            # Run pytest suite
uv run pytest -m integration  # Integration tests only (require DB/Ollama)
uv run pytest -k test_name     # Run specific test

Test markers: @pytest.mark.integration, @pytest.mark.manual, @pytest.mark.debug (see pyproject.toml).

Code Quality

make lint            # Run tidy + pylint + check-imports
make tidy            # Run black + isort
make check-imports   # Verify no relative imports beyond current package

Import rules: Ruff enforces ban-relative-imports = "parents" (see pyproject.toml). Use absolute imports from src.*.

Architecture Deep Dive

Request Flow

OpenRefine → POST /reconcile → router.py:reconcile() →
  reconcile.py:reconcile_queries() →
    Strategies.get("site") → SiteReconciliationStrategy →
      SiteRepository.search() → PostgreSQL

RAG Hybrid Strategy (New Pattern)

Phase 1 implementation uses embedded MCP server for small-prompt reconciliation:

from src.strategies.rag_hybrid import RAGHybridReconciliationStrategy

class MethodReconciliationStrategy(RAGHybridReconciliationStrategy):
    # Uses MCP search_lookup → 5-10 candidates → LLM validation

Feature flag: features.use_mcp_server in config/config.yml. When disabled, falls back to standard fuzzy search.

Multi-Environment Config

Override config file: export CONFIG_FILE=./tests/config/config.yml

Common Gotchas

  1. Strategy not found: Ensure strategy file is in src/strategies/ and decorated with @Strategies.register(). Import errors are printed to console during startup but don't fail startup.

  2. Config not available: Call await setup_config_store() before accessing config. FastAPI does this in main.py startup event.

  3. Schema changes ignored: Run make generate-schema after editing config/entities.yml. The --force flag overwrites existing files.

  4. Test isolation: Use fixtures from tests/conftest.py. MockConfigProvider prevents tests from hitting real database.

  5. LLM provider setup: Ollama/OpenAI providers lazy-load config. See src/llm/providers/ for provider-specific settings.

Git Commit Conventions

This project uses Conventional Commits with semantic-release for automated versioning. AI agents making commits must follow this format:

<type>[optional scope]: <description>

Release-Triggering Types

  • feat: New feature → MINOR release (1.2.0)
  • fix: Bug fix → PATCH release (1.2.1)
  • refactor/perf/style: Code improvements → PATCH release
  • docs: Documentation → PATCH if scope is README
  • test/build/ci/chore: No release

Breaking Changes

Add ! after type/scope or BREAKING CHANGE: in footer → MAJOR release (2.0.0):

feat(api)!: change response format for validation errors

Common Scopes

core, config, api, cache, loaders, tests, deps

Examples

feat(cache): implement hash-based cache invalidation
fix(validation): prevent null pointer in entity resolution
refactor(core): simplify dependency resolution logic
docs(README): update installation instructions
test(loaders): add comprehensive UCanAccessSqlLoader tests

Rules: Use imperative mood, lowercase description, no trailing period, keep under 72 chars.

Key Files Reference

Docker Deployment

Multi-environment support:

  • Development: cd docker && docker-compose up --build
  • Production: Uses GHCR images from CI/CD (see docker/README.md)

GitHub Actions builds on every push to main/dev and pushes to ghcr.io/humlab-sead/sead_authority_service.