feat: worker abstractions; fix: various improvements and bug fixes

[tazlin]

This PR introduces new features (mostly around generation/worker abstractions), refactors core SDK structures, expands endpoint support, improves testing and documentation, and fixes a range of issues across API models, naming, logging, and compatibility.

Features

• Add support for remaining endpoints
  • Add shared key support
    • Introduce shared key models and API mappings
    • Add MessageSpecifiesSharedKeyMixin
    • Warn on empty shared keys
  • Add support for user deletion endpoint
• Add time_constructed property to HordeResponse
• Introduce generation_parameters namespace for backend-agnostic parameter models
  • Add initial HordeSingleGeneration support
  • Add generation, dispatch handling, and related infrastructure
• Add backend parsing modules and example worker generation scripts
• Add opt-in telemetry
• Add _async_client_exceptions standardization in SDK namespace
• Add image utilities:
  • first_pass_resolution
  • calc_upscale_sampler_steps
  • base64_str_to_pil_image
• Add bridge agent string helper with default usage
• Add safety-related classes and feature flags
• Add support for load_large_models argument in model_meta (for worker model choices)

Refactoring

• Rename JobID → GenerationID and related variables (job_id → gen_id)
• Rename generation_ids → result_ids
• Refactor states, result types, and thread safety
• Refactor image-specific constants naming
• Move constants into more appropriate namespaces
• Move generation constants into gen._params/*/consts.py
• Refactor namespace ai_horde_worker → worker
• Refactor bridge data and dispatch source constants
• Refactor dynamic request handling using requests.request(...)
• Introduce dynamic parameters via additional_params
• Remove cyclic dependencies between namespaces
• Remove unused and superseded code
• Remove pydantic pin

Fixes

• UUID handling:
  • Warn on incompatible UUID comparisons
  • Handle additional comparison cases
• Naming and consistency:
  • Improve naming for user ID mixin and known type aliases
  • Fix inconsistent naming for job pop types
• API/model correctness:
  • Fix API model name mismatches
  • Fix malformed models and missing fields/imports
  • Fix request/response model mappings
  • Fix python→API field rewriting on submit
  • Fix protected namespace usage (model_)
  • Fix incorrect API methods (e.g., PUT vs POST)
• Logging:
  • Align logging with other projects
  • Ensure safe dump on timeout
  • Fix logger handler behavior
• Async/client:
  • Fix ssl_context not passed in async client
  • Expose NoValidRequestFound
• Job/generation behavior:
  • Fix sorting behavior for IDs and uploads in responses
  • Fix hashing behavior for job pop responses
  • Fix default factories (e.g., skipped, _time_constructed)
• Image and processing:
  • Fix concurrent download of source images
• Compatibility:
  • Fix Python <3.12 and <3.13 issues
  • Use typing_extensions where required
• Misc:
  • Fix recursive submodule checkout
  • Fix missing __init__.py files
  • Fix enum and type handling
  • Fix message namespace issues

Tests

• Expand and refactor pytest fixtures:
  • Add reusable fixtures (id_factory, image fixtures, job fixtures)
  • Add request and response validation in dynamic_json_load
• Add tests for:
  • Stats and model state handling
  • Shared keys
  • Job and generation flows
  • HTTP verb coverage
  • API surface validation
• Improve test reliability:
  • Use fixed UUIDs
  • Add compatibility handling for Python versions
• Refactor test structure:
  • Move shared fixtures to conftest
  • Rename namespaces to match refactors
• Improve debugging:
  • Add detailed output on validation failures
• Ignore certain API models intentionally in tests

Documentation

• Update terminology (JobID → GenerationID)
• Add and update shared key documentation
• Add SDK model map entries
• Standardize docstrings for requests, responses, and API models
• Fix dangling docstring references
• Rebuild mkdocs and autogenerated docs
• Improve MkDocs configuration:
  • Layout adjustments
  • Mermaid support
  • Asset integration
• Add and restructure documentation:
  • Move AI Horde–specific docs to subdirectory
  • Add onboarding, contribution guide, and style guide
  • Add naming conventions and metadata clarifications
• Fix markdown linting and formatting issues
• Add docstring coverage in tests

CI / Build / Tooling

• Add CI enforcement for inter-package dependencies (tach)
• Update and apply pre-commit fixes
• Update submodules and include assets in docs builds
• Add temporary requirements file prior to tooling transition
• Ignore additional virtual environment directories and config files

Chores / Style

• Apply lint and formatting fixes across codebase
• Remove unused imports and references
• Add additional trace logging for UUID handling
• Clean up documentation and formatting
• Remove outdated references (e.g., date black config)

Notes

• Includes large-scale refactors to job/generation systems, API mappings, and package structure
• Expands SDK coverage of API endpoints and behaviors
• Aligns naming, typing, and logging conventions across the codebase