Caching, Testing, and Documentation Updates (#19)

* Caching, Testing, and Documentation Updates

* Fix CLI tests with certain options

Author: tedivm

.github/workflows/test.yaml

@@ -10,6 +10,7 @@ jobs:
   test-template:
     runs-on: ubuntu-latest
     strategy:
+      fail-fast: false
       matrix:
         config-file:
           - full.yaml
@@ -25,9 +26,36 @@ jobs:
         with:
           python-version: "3.x"
 
+      - name: Install yq
+        run: |
+          sudo wget -qO /usr/local/bin/yq https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64
+          sudo chmod +x /usr/local/bin/yq
+
       - name: Install cookiecutter
         run: pip install cookiecutter
 
       - name: Run cookiecutter template
         run: |
           cookiecutter . --no-input --config-file tests/${{ matrix.config-file }} --output-dir /tmp
+
+      - name: Get project name
+        id: project
+        run: |
+          PROJECT_NAME=$(yq -r '.default_context.package_name' tests/${{ matrix.config-file }})
+          echo "name=$PROJECT_NAME" >> $GITHUB_OUTPUT
+
+      - name: Run pytest
+        working-directory: /tmp/${{ steps.project.outputs.name }}
+        run: make pytest
+
+      - name: Run ruff check
+        working-directory: /tmp/${{ steps.project.outputs.name }}
+        run: make ruff_check
+
+      - name: Run black format check
+        working-directory: /tmp/${{ steps.project.outputs.name }}
+        run: make black_check
+
+      - name: Run mypy
+        working-directory: /tmp/${{ steps.project.outputs.name }}
+        run: make mypy_check

.gitignore

@@ -0,0 +1 @@
+workspaces/

AGENTS.md

@@ -0,0 +1,9 @@
+# Agent Instructions
+
+This project is a CookieCutter template used to generate new python projects. It is extremely dynamic, with many optional settings. Make sure to review the README.md to get more context, as well as the AGENTS.md file int he project template itself (`{{cookiecutter.__package_slug}}/AGENTS.md`).
+
+Since this is a Cookiecutter template you should expect to encounter Jinja2 template blocks in various files.
+
+When being asked to test functionality that requires you to create a new project from the template create them in the `workspaces` directory.
+
+When creating new files for optional services make sure you include them in the post_gen_project.py configuration so that unneeded files are removed. For example, if the caching functionality is not enabled the system should remove all of the caching related files.

README.md

@@ -86,6 +86,17 @@ Pick and choose the features you need. Unused components are completely removed
 - Type-safe configuration using Pydantic for automatic validation of input data, with configurable queue sizes, worker counts, and graceful shutdown handling
 - Perfect for CPU-intensive workloads like data processing, image manipulation, scientific computing, and batch operations that need to scale beyond single-threaded execution
 
+### Caching
+
+**[aiocache](https://aiocache.readthedocs.io/) Integration**
+
+- High-performance async caching library with support for multiple backends including Redis, Memcached, and in-memory storage, providing millisecond-level response times for frequently accessed data
+- Automatic cache configuration and connection management with separate cache instances for different TTL requirements: default (5 minutes), persistent (1 hour), and custom durations for specific use cases
+- Decorator-based caching with `@cached` for effortless function result memoization, automatically serializing complex Python objects including Pydantic models, dataclasses, and custom types
+- Built-in cache warming on application startup for Celery workers and web servers, pre-populating critical data to eliminate cold-start latency and ensure consistent performance from the first request
+- Type-safe settings configuration for cache behavior including host, port, TTL values, and enable/disable flags, with automatic validation and clear error messages for misconfigurations
+- Production-ready Redis integration with connection pooling, automatic reconnection handling, and graceful degradation when cache is unavailable, preventing cascading failures
+
 ### Database & ORM
 
 **[SQLAlchemy](https://www.sqlalchemy.org/) + [Alembic](https://alembic.sqlalchemy.org/en/latest/)**
@@ -156,6 +167,7 @@ Every generated project includes documentation tailored to your selected feature
 - **Developer Guide Hub**: Organized documentation index in `docs/dev/` with dedicated guides for each enabled feature
 - **FastAPI Documentation**: Integration guide covering static file serving, Docker configuration, and FastAPI dependency system usage
 - **Database Documentation**: SQLAlchemy and Alembic guide covering model organization, migration creation using Make commands, FastAPI integration, and automatic schema diagram generation with Paracelsus
+- **Caching Documentation**: aiocache integration guide covering cache configuration, decorator usage, multiple TTL strategies, and cache warming for optimal performance
 - **Task Processing Guides**: Documentation for Celery (worker and beat configuration, Docker setup) and QuasiQueue (configuration file location, Docker images)
 - **CLI Documentation**: Guide showing how to use the generated CLI and where to add new commands
 - **Docker Documentation**: Container setup documentation covering image sources, development environment, and registry publishing
@@ -167,7 +179,8 @@ Every generated project includes documentation tailored to your selected feature
 The template intelligently configures itself based on your choices through sophisticated post-generation hooks:
 
 - **Surgical Dependency Management**: Only includes packages you actually need in `pyproject.toml`, with proper optional dependency groups for dev tools, testing, and feature-specific requirements, avoiding bloated dependency trees
-- **Conditional Docker Services**: Automatically generates docker-compose.yaml with only the services your project requires: PostgreSQL for SQLAlchemy, Redis for Celery/caching, with properly configured health checks, volumes, and networking
+- **Conditional Docker Services**: Automatically generates docker-compose.yaml with only the services your project requires: PostgreSQL for SQLAlchemy, Redis for Celery/aiocache caching, with properly configured health checks, volumes, and networking
+- **Cache-Aware Configuration**: When aiocache is enabled, automatically configures Redis connection settings, multiple cache instances with different TTL strategies, and cache warming hooks for FastAPI and Celery startup events
 - **Database-Aware Configuration**: Sets up appropriate connection strings, pool sizes, and dialect-specific settings for PostgreSQL or SQLite, with Alembic migrations configured for cross-database compatibility
 - **Feature-Driven CI/CD Workflows**: GitHub Actions workflows are conditionally installed based on your feature selection: container building and publishing only when Docker is enabled, PyPI publishing workflow only when configured, eliminating unused automation files from your repository
 - **Framework Integration**: Automatically wires together selected components (FastAPI with SQLAlchemy database dependencies, Celery with Redis broker, CLI with async command support) providing working examples of how pieces fit together

cookiecutter.json

@@ -3,6 +3,7 @@
   "author_name": "",
   "short_description": "",
   "python_version": "3.14",
+  "github_org": "EXAMPLE",
   "license": [
     "All Rights Reserved",
     "MIT license",
@@ -15,7 +16,7 @@
   "include_sqlalchemy": "y/N",
   "include_quasiqueue": "y/N",
   "include_jinja2": "y/N",
-  "include_dogpile": "y/N",
+  "include_aiocache": "y/N",
   "include_celery": "y/N",
   "include_docker": "y/N",
   "include_github_actions": "y/N",

hooks/post_gen_project.py

@@ -10,7 +10,7 @@
 INCLUDE_DOCKER={% if cookiecutter.include_docker == "y" %}True{% else %}False{% endif %}
 INCLUDE_QUASIQUEUE={% if cookiecutter.include_quasiqueue == "y" %}True{% else %}False{% endif %}
 INCLUDE_JINJA2={% if cookiecutter.include_jinja2 == "y" %}True{% else %}False{% endif %}
-INCLUDE_DOGPILE={% if cookiecutter.include_dogpile == "y" %}True{% else %}False{% endif %}
+INCLUDE_AIOCACHE={% if cookiecutter.include_aiocache == "y" %}True{% else %}False{% endif %}
 INCLUDE_SQLALCHEMY={% if cookiecutter.include_sqlalchemy == "y" %}True{% else %}False{% endif %}
 INCLUDE_GITHUB_ACTIONS={% if cookiecutter.include_github_actions == "y" %}True{% else %}False{% endif %}
 INCLUDE_REQUIREMENTS_FILES={% if cookiecutter.include_requirements_files == "y" %}True{% else %}False{% endif %}
@@ -30,6 +30,7 @@
     remove_paths.add(f'dockerfile.www')
     remove_paths.add(f'docker/www')
     remove_paths.add(f'docs/dev/api.md')
+    remove_paths.add(f'tests/test_www.py')
 
 if INCLUDE_CELERY:
     docker_containers.add('celery')
@@ -38,13 +39,15 @@
     remove_paths.add(f'dockerfile.celery')
     remove_paths.add(f'docker/celery')
     remove_paths.add(f'docs/dev/celery.md')
+    remove_paths.add(f'tests/test_celery.py')
 
 if INCLUDE_QUASIQUEUE:
     docker_containers.add('qq')
 else:
     remove_paths.add(f'{PACKAGE_SLUG}/qq.py')
     remove_paths.add(f'dockerfile.qq')
     remove_paths.add(f'docs/dev/quasiqueue.md')
+    remove_paths.add(f'tests/test_qq.py')
 
 if not INCLUDE_SQLALCHEMY:
     remove_paths.add(f'{PACKAGE_SLUG}/models')
@@ -59,16 +62,23 @@
 if not INCLUDE_CLI:
     remove_paths.add(f'{PACKAGE_SLUG}/cli.py')
     remove_paths.add(f'docs/dev/cli.md')
+    remove_paths.add(f'tests/test_cli.py')
 
 if not INCLUDE_JINJA2:
     remove_paths.add(f'{PACKAGE_SLUG}/templates')
     remove_paths.add(f'{PACKAGE_SLUG}/services/jinja.py')
     remove_paths.add(f'docs/dev/templates.md')
+    remove_paths.add(f'tests/services/test_jinja.py')
 
-if not INCLUDE_DOGPILE:
+if not INCLUDE_AIOCACHE:
+    remove_paths.add(f'{PACKAGE_SLUG}/conf/cache.py')
     remove_paths.add(f'{PACKAGE_SLUG}/services/cache.py')
+    remove_paths.add(f'tests/services/test_cache.py')
     remove_paths.add(f'docs/dev/cache.md')
 
+# Always include test_settings.py as it tests core settings functionality
+# that exists regardless of optional features
+
 if not INCLUDE_DOCKER:
     remove_paths.add('.dockerignore')
     remove_paths.add('compose.yaml')

tests/bare.yaml

@@ -8,7 +8,7 @@ default_context:
   include_fastapi: "n"
   include_sqlalchemy: "n"
   include_jinja2: "n"
-  include_dogpile: "n"
+  include_aiocache: "n"
   include_celery: "n"
   include_docker: "n"
   include_github_actions: "n"

tests/full.yaml

@@ -9,7 +9,7 @@ default_context:
   include_sqlalchemy: "y"
   include_quasiqueue: "y"
   include_jinja2: "y"
-  include_dogpile: "y"
+  include_aiocache: "y"
   include_celery: "y"
   include_docker: "y"
   include_github_actions: "y"

tests/library.yaml

@@ -8,7 +8,7 @@ default_context:
   include_fastapi: "n"
   include_sqlalchemy: "n"
   include_jinja2: "n"
-  include_dogpile: "n"
+  include_aiocache: "n"
   include_celery: "n"
   include_docker: "n"
   include_github_actions: "y"

{{cookiecutter.__package_slug}}/README.md

@@ -22,3 +22,27 @@ pip install {{ cookiecutter.package_name }}
 ```
 
 {%- endif %}
+
+## Developer Documentation
+
+Comprehensive developer documentation is available in [`docs/dev/`](./docs/dev/) covering testing, configuration, deployment, and all project features.
+
+### Quick Start for Developers
+
+```bash
+# Install development environment
+make install
+{%- if cookiecutter.include_docker == "y" %}
+
+# Start services with Docker
+docker compose up -d
+{%- endif %}
+
+# Run tests
+make tests
+
+# Auto-fix formatting
+make chores
+```
+
+See the [developer documentation](./docs/dev/README.md) for complete guides and reference.