Merge pull request #127 from Promptly-Technologies-LLC/126-move-template-routers-into-a-core-folder-with-a-separate-one-for-non-template-app-logic

Refactor for better template-app separation, make clear where to put app logic

Author: chriscarrollsmith

.cursor/rules/routers.mdc

@@ -1,12 +1,12 @@
 ---
 description: Testing FastAPI routes
-globs: routers/*.py
+globs: routers/**/*.py
 alwaysApply: false
 ---
 Here are the five most critical patterns to maintain consistency when adding a new router:
 
 1. **Authentication & Dependency Injection**
-   - Import `get_authenticated_user` from `utils.dependencies` and include `user: User = Depends(get_authenticated_user)` in the arguments of routes requiring authentication
+   - Import `get_authenticated_user` from `utils.core.dependencies` and include `user: User = Depends(get_authenticated_user)` in the arguments of routes requiring authentication
    - Similarly, use the `get_optional_user` dependency for public routes with potential auth status
 
 2. **Validation Patterns**
@@ -21,7 +21,7 @@ Here are the five most critical patterns to maintain consistency when adding a n
    - Check permissions at both route and template levels via `user_permissions`
 
 4. **Database & Transaction Patterns**
-   - Inject session via `Depends(get_session)`
+   - Inject session via `Depends(get_session)` from `utils/core/dependencies.py`
    - Commit after writes and refresh objects where needed
    - Use `selectinload` for eager loading relationships
    - Follow PRG pattern with RedirectResponse after mutations

.cursor/rules/routers_tests.mdc

@@ -1,6 +1,6 @@
 ---
 description: 
-globs: tests/routers/test_*.py
+globs: tests/routers/**/*.py
 alwaysApply: false
 ---
 # Setting test expectations regarding HTTP status codes

.cursor/rules/tests.mdc

@@ -1,6 +1,6 @@
 ---
 description: Building, running, and debugging tests
-globs: tests/*.py
+globs: tests/**.py
 alwaysApply: false
 ---
 This project uses `uv` for dependency management, so tests must be run with `uv run pytest` to ensure they are run in the project's virtual environment.
@@ -9,4 +9,4 @@ The project uses test-driven development, so failing tests are often what we wan
 
 Session-wide test setup is performed in `tests/conftest.py`. In that file, you will find fixtures that can and should be reused across the test suite, including fixtures for database setup and teardown. We have intentionally used PostgreSQL, not SQLite, in the test suite to keep the test environment as production-like as possible, and you should never change the database engine unless explicitly told to do so.
 
-If you find that the test database is not available, you may need to start Docker Desktop with `systemctl --user start docker-desktop` or the database with `docker compose up`. You may `grep` the `DB_PORT=` line from `.env` if you need to know what port the database is available on. (This environment variable is used for port mapping in `docker-compose.yml` as well as in the `get_connection_url` function defined in `utils/db.py`.) If dropping tables fails during test setup due to changes to the database schema, `docker compose down -v && docker compose up` may resolve the issue.
+If you find that the test database is not available, you may need to start Docker Desktop with `systemctl --user start docker-desktop` or the database with `docker compose up`. You may `grep` the `DB_PORT=` line from `.env` if you need to know what port the database is available on. (This environment variable is used for port mapping in `docker-compose.yml` as well as in the `get_connection_url` function defined in `utils/core/db.py`.) If dropping tables fails during test setup due to changes to the database schema, `docker compose down -v && docker compose up` may resolve the issue.

docs/customization.qmd

@@ -22,6 +22,14 @@ If you are using VSCode or Cursor as your IDE, you will need to select the `uv`-
 
 If your IDE does not automatically detect and display this option, you can manually select the interpreter by selecting "Enter interpreter path" and then navigating to the `.venv/bin/python` subfolder in your project directory.
 
+### Extending the template
+
+The `routers/core/` and `utils/core/` directories contain the core backend logic for the template. 
+
+Your custom Python backend code should go primarily in the `routers/app/` and `utils/app/` directories.
+
+For the frontend, you will also need to develop custom Jinja2 templates in the `templates/` folder and add custom static assets in `static/`.
+
 ### Testing
 
 The project uses Pytest for unit testing. It's highly recommended to write and run tests before committing code to ensure nothing is broken!
@@ -57,11 +65,7 @@ We find that mypy is an enormous time-saver, catching many errors early and grea
 
 ### Developing with LLMs
 
-In line with the [llms.txt standard](https://llmstxt.org/), we have provided a Markdown-formatted prompt—designed to help LLM agents understand how to work with this template—as a text file: [llms.txt](static/llms.txt).
-
-One use case for this file, if using the Cursor IDE, is to rename it to `.cursorrules` and place it in your project directory (see the [Cursor docs](https://docs.cursor.com/context/rules-for-ai) on this for more information). Alternatively, you could use it as a custom system prompt in the web interface for ChatGPT, Claude, or the LLM of your choice.
-
-We have also exposed the full Markdown-formatted project documentation as a [single text file](static/documentation.txt) for easy downloading and embedding for RAG workflows.
+The `.cursor/rules` folder contains a set of AI rules for working on this codebase in the Cursor IDE. We have also provided an [llms.txt](static/llms.txt) system prompt file for use with other agentic LLM workflows and exposed the full Markdown-formatted project documentation as a [single text file](docs/static/documentation.txt) for easy downloading and embedding for RAG.
 
 ## Application architecture
 
@@ -71,45 +75,48 @@ In this template, we use FastAPI to define the "API endpoints" of our applicatio
 
 We also create POST endpoints, which accept form submissions so the user can create, update, and delete data in the database. This template follows the Post-Redirect-Get (PRG) pattern to handle POST requests. When a form is submitted, the server processes the data and then returns a "redirect" response, which sends the user to a GET endpoint to re-render the page with the updated data. (See [Architecture](https://promptlytechnologies.com/fastapi-jinja2-postgres-webapp/docs/architecture.html) for more details.)
 
-#### Customizable folders and files
+#### File structure
 
 - FastAPI application entry point and homepage GET route: `main.py`
-- FastAPI routes: `routers/`
+- Template FastAPI routes: `routers/core/`
     - Account and authentication endpoints: `account.py`
     - User profile management endpoints: `user.py`
     - Organization management endpoints: `organization.py`
     - Role management endpoints: `role.py`
     - Dashboard page: `dashboard.py`
     - Static pages (e.g., about, privacy policy, terms of service): `static_pages.py`
+- Custom FastAPI routes for your app: `routers/app/`
 - Jinja2 templates: `templates/`
 - Static assets: `static/`
 - Unit tests: `tests/`
 - Test database configuration: `docker-compose.yml`
-- Helper functions: `utils/`
+- Template helper functions: `utils/core/`
     - Auth helpers: `auth.py`
     - Database helpers: `db.py`
     - FastAPI dependencies: `dependencies.py`
     - Enums: `enums.py`
     - Image helpers: `images.py`
     - Database models: `models.py`
+- Custom template helper functions for your app: `utils/app/`
 - Exceptions: `exceptions/`
     - HTTP exceptions: `http_exceptions.py`
     - Other custom exceptions: `exceptions.py`
-- Environment variables: `.env.example`
+- Environment variables: `.env.example`, `.env`
 - CI/CD configuration: `.github/`
 - Project configuration: `pyproject.toml`
 - Quarto documentation: 
     - README source: `index.qmd`
     - Website source: `index.qmd` + `docs/`
-    - Configuration: `_quarto.yml`
+    - Configuration: `_quarto.yml` + `_environment`
+- Rules for developing with LLMs in Cursor IDE: `.cursor/rules/`
 
 Most everything else is auto-generated and should not be manually modified.
 
 ## Backend
 
 ### Code conventions
 
-The GET route for the homepage is defined in the main entry point for the application, `main.py`. The entrypoint imports router modules from the `routers/` directory, which contain the other GET and POST routes for the application. In CRUD style, the router modules are named after the resource they manage, e.g., `account.py` for account management.
+The GET route for the homepage is defined in the main entry point for the application, `main.py`. The entrypoint imports router modules from the `routers/core/` directory (for core/template logic) and `routers/app/` directory (for app-specific logic). In CRUD style, the core router modules are named after the resource they manage, e.g., `account.py` for account management. You should place your own endpoints in `routers/app/`.
 
 We name our GET routes using the convention `read_<name>`, where `<name>` is the name of the resource, to indicate that they are read-only endpoints that do not modify the database. In POST routes that modify the database, you can use the `get_session` dependency as an argument to get a database session.
 
@@ -177,7 +184,7 @@ SQLModel is an Object-Relational Mapping (ORM) library that allows us to interac
 
 ### Models and relationships
 
-Our database models are defined in `utils/models.py`. Each model is a Python class that inherits from `SQLModel` and represents a database table. The key models are:
+Core database models are defined in `utils/core/models.py`. Each model is a Python class that inherits from `SQLModel` and represents a database table. The key core models are:
 
 - `Account`: Represents a user account with email and password hash
 - `User`: Represents a user profile with details like name and avatar; the email and password hash are stored in the related `Account` model
@@ -192,15 +199,15 @@ Two additional models are used by SQLModel to manage many-to-many relationships;
 - `UserRoleLink`: Maps users to their roles (many-to-many relationship)
 - `RolePermissionLink`: Maps roles to their permissions (many-to-many relationship)
 
-Here's an entity-relationship diagram (ERD) of the current database schema, automatically generated from our SQLModel definitions:
+Here's an entity-relationship diagram (ERD) of the current core database schema, automatically generated from our SQLModel definitions:
 
 ```{python}
 #| echo: false
 #| warning: false
 import sys
 sys.path.append("..")
-from utils.models import *
-from utils.db import engine
+from utils.core.models import *
+from utils.core.db import engine
 from sqlalchemy import MetaData
 from sqlalchemy_schemadisplay import create_schema_graph
 
@@ -220,16 +227,17 @@ graph.write_png('static/schema.png')
 
 ![Database Schema](static/schema.png)
 
+To extend the database schema, define your own models in `utils/app/models.py` and import them in `utils/core/db.py` to make sure they are included in the `metadata` object in the `create_all` function.
 
 ### Database helpers
 
-Database operations are facilitated by helper functions in `utils/db.py`. Key functions include:
+Database operations are facilitated by helper functions in `utils/core/db.py` (for core logic) and `utils/app/` (for app-specific helpers). Key functions in the core utils include:
 
 - `set_up_db()`: Initializes the database schema and default data (which we do on every application start in `main.py`)
 - `get_connection_url()`: Creates a database connection URL from environment variables in `.env`
 - `get_session()`: Provides a database session for performing operations
 
-To perform database operations in route handlers, inject the database session as a dependency:
+To perform database operations in route handlers, inject the database session as a dependency (from `utils/core/db.py`):
 
 ```python
 @app.get("/users")
@@ -240,7 +248,7 @@ async def get_users(session: Session = Depends(get_session)):
 
 The session automatically handles transaction management, ensuring that database operations are atomic and consistent.
 
-There is also a helper method on the `User` model that checks if a user has a specific permission for a given organization. Its first argument must be a `ValidPermissions` enum value (from `utils/models.py`), and its second argument must be an `Organization` object or an `int` representing an organization ID:
+There is also a helper method on the `User` model that checks if a user has a specific permission for a given organization. Its first argument must be a `ValidPermissions` enum value (from `utils/core/models.py`), and its second argument must be an `Organization` object or an `int` representing an organization ID:
 
 ```python
 permission = ValidPermissions.CREATE_ROLE
@@ -249,7 +257,7 @@ organization = session.exec(select(Organization).where(Organization.name == "Acm
 user.has_permission(permission, organization)
 ```
 
-You should create custom `ValidPermissions` enum values for your application and validate that users have the necessary permissions before allowing them to modify organization data resources.
+You should create custom `AppPermissions` enum values for your application in `utils/app/` (if needed) and validate that users have the necessary permissions before allowing them to modify organization data resources.
 
 ### Cascade deletes
 

docs/static/documentation.txt

@@ -785,8 +785,8 @@ Here's an entity-relationship diagram (ERD) of the current database schema, auto
 #| warning: false
 import sys
 sys.path.append("..")
-from utils.models import *
-from utils.db import engine
+from utils.core.models import *
+from utils.core.db import engine
 from sqlalchemy import MetaData
 from sqlalchemy_schemadisplay import create_schema_graph
 

docs/static/llms.txt

@@ -1,75 +1,39 @@
 # Project Architecture
-- Keep GET routes in main.py and POST routes in routers/ directory
-- Name GET routes using read_<name> convention
+- Name GET routes using `read_<name>` convention
 - Follow Post-Redirect-Get (PRG) pattern for all form submissions
 - Use Jinja2 HTML templates for server-side rendering and minimize client-side JavaScript
 - Use forms for all POST routes
 - Validate form data comprehensively on the client side as first line of defense, with server-side Pydantic validation as fallback
-
-# File Structure
-- main.py: Application entry point and GET routes
-- routers/: POST route modules
-- templates/: Jinja2 templates
-- static/: Static assets
-- tests/: Unit tests
-- utils/: Helper functions and models
-- docker-compose.yml: Test database configuration
-- .env: Environment variables
-
-# Python/FastAPI Guidelines
-- For all POST routes, define request models in a separate section at the top of the router file
-- Implement as_form() classmethod for all form-handling request models
-- Use Pydantic for request/response models with @field_validator and custom exceptions for custom form validation
 - Use middleware defined in main.py for centralized exception handling
-- Add type hints to all function signatures and variables
-- Follow mypy type checking standards rigorously
+- Add type hints to all function signatures and variables for static type checking
 
-# Form Validation Strategy
-- Implement thorough client-side validation via HTML pattern attributes where possible and Javascript otherwise
-- Use Pydantic models with custom validators as server-side fallback
-- Handle validation errors through middleware exception handlers
-- Render validation_error.html template for failed server-side validation
+# File Structure
+- `main.py`: Application entry point and GET routes
+- `routers/core`: Base webapp template API routes
+- `routers/app`: Application API routes that extend the template
+- `utils/core`, `utils/app`: Helper functions, FastAPI dependencies, database models
+- `templates/`: Jinja2 templates
+- `static/`: Static assets
+- `tests/`: Unit tests
+- `docker-compose.yml`: Test database configuration
+- `.env`: Environment variables
+- `docs/`: Quarto documentation website source files
 
 # Database Operations
 - Use SQLModel for all database interactions
-- Use get_session() from utils/db.py for database connections
-- Define database relational models explicitly in utils/models.py
-- Inject database session as dependency in route handlers
+- Use `get_session()` FastAPI dependency from `utils/core/dependencies.py` for database connections
 
 # Authentication System
-- JWT-based token authentication with separate access/refresh tokens and bcrypt for password hashing are defined in utils/auth.py
-- Password and email reset tokens with expiration and password reset email flow powered by Resend are defined in utils/auth.py
-- HTTP-only cookies are implemented with secure flag and SameSite=strict
-- Inject common_authenticated_parameters as a dependency in all authenticated GET routes
-- Inject common_unauthenticated_parameters as a dependency in all unauthenticated GET routes
-- Inject get_session as a dependency in all POST routes
-- Handle security-related errors without leaking information
-
-# Testing
-- Run mypy type checking before committing code
-- Write comprehensive unit tests using pytest
-- Test both success and error cases
-- Use test fixtures from tests/conftest.py: engine, session, client, test_user
-- set_up_database and clean_db fixtures are autoused by pytest to ensure clean database state
-
-# Error Handling
-- Use middleware for centralized exception handling
-- Define custom exception classes for specific error cases
-- Return appropriate HTTP status codes and error messages
-- Render error templates with context data
-- Log errors with "uvicorn.error" logger
-
-# Template Structure
-- Extend base.html for consistent layout
-- Use block tags for content sections
-- Include reusable components
-- Pass request object and context data to all templates
-- Keep form validation logic in corresponding templates
-- Use Bootstrap for styling
+- JWT-based token authentication with separate access/refresh tokens and bcrypt for password hashing are defined in `utils/core/auth.py`
+- Password and email reset tokens with expiration and password reset email flow powered by Resend are defined in `utils/core/auth.py`
+- HTTP-only cookies are implemented with secure flag and `SameSite=strict`
+- Inject `common_authenticated_parameters` as a dependency in all authenticated GET routes
+- Inject `common_unauthenticated_parameters` as a dependency in all unauthenticated GET routes
 
 # Contributing Guidelines
 - Follow existing code style and patterns
 - Preserve existing comments and docstrings
 - Ensure all tests pass before submitting PR
 - Update .qmd documentation files for significant changes
-- Use uv for dependency management
+- Use uv for dependency management
+- Run `uv run mypy .` to ensure code passes a static type check

exceptions/exceptions.py

@@ -1,4 +1,4 @@
-from utils.models import User
+from utils.core.models import User
 
 
 class NeedsNewTokens(Exception):

exceptions/http_exceptions.py

@@ -1,5 +1,5 @@
 from fastapi import HTTPException, status
-from utils.enums import ValidPermissions
+from utils.core.enums import ValidPermissions
 
 class EmailAlreadyRegisteredError(HTTPException):
     def __init__(self):

index.qmd

@@ -207,11 +207,7 @@ with open(output_path, 'w', encoding='utf-8') as f:
     f.write(final_content)
 ```
 
-In line with the [llms.txt standard](https://llmstxt.org/), we have provided a Markdown-formatted prompt—designed to help LLM agents understand how to work with this template—as a text file: [llms.txt](docs/static/llms.txt).
-
-One use case for this file, if using the Cursor IDE, is to rename it to `.cursorrules` and place it in your project directory (see the [Cursor docs](https://docs.cursor.com/context/rules-for-ai) on this for more information). Alternatively, you could use it as a custom system prompt in the web interface for ChatGPT, Claude, or the LLM of your choice.
-
-We have also exposed the full Markdown-formatted project documentation as a [single text file](docs/static/documentation.txt) for easy downloading and embedding for RAG workflows.
+The `.cursor/rules` folder contains a set of AI rules for working on this codebase in the Cursor IDE. We have also provided an [llms.txt](static/llms.txt) system prompt file for use with other agentic LLM workflows and exposed the full Markdown-formatted project documentation as a [single text file](docs/static/documentation.txt) for easy downloading and embedding for RAG.
 
 ## Contributing