Refactor database schema and plugin system for improved flexibility and type safety

- Updated schema definitions in `schema.ts` to use column builder functions for better compatibility with different database dialects.
- Modified `PluginManager` to utilize a unified `AnyDatabase` type, allowing for both SQLite and PostgreSQL databases.
- Enhanced plugin initialization to handle cases where the database may not be set, allowing plugins to manage their own database interactions.
- Introduced new database configuration management in `config.ts`, enabling dynamic setup for SQLite and PostgreSQL.
- Created new routes for database setup and status, utilizing Zod for request validation and response schemas.
- Added migration SQL files for example plugin tables to ensure proper database structure.
- Implemented type guards and casting in example plugin to handle database-specific operations.
- Updated Fastify instance to include new database and plugin manager decorations for easier access in routes.

Author: Lasim

.gitignore

@@ -53,4 +53,5 @@ fastly-events.log
 .turbo
 .cache
 
-deploystack.db
+deploystack.db
+services/backend/persistent_data/*

package-lock.json

@@ -4,6 +4,55 @@
 
 DeployStack uses SQLite with Drizzle ORM for database operations. This combination provides excellent performance, type safety, and a modern, developer-friendly experience without the need for external database dependencies.
 
+## Database Setup and Configuration
+
+The backend server provides API endpoints for managing the initial database setup and checking its status.
+
+### Database Status
+
+You can check the current status of the database (whether it's configured and initialized) using the following endpoint:
+
+- **Endpoint:** `GET /api/db/status`
+- **Method:** `GET`
+- **Response:** A JSON object indicating the database `configured` status (boolean), `initialized` status (boolean), and current `dialect` (e.g., "sqlite" or "postgres", or null if not configured).
+
+### Initial Database Setup
+
+To perform the initial setup of the database, use the following endpoint:
+
+- **Endpoint:** `POST /api/db/setup`
+- **Method:** `POST`
+- **Request Body:** A JSON object specifying the database type and configuration.
+
+**For SQLite:**
+The server will automatically manage the database file location. The request body should be:
+```json
+{
+  "type": "sqlite"
+}
+```
+The SQLite database file will be created and stored at: `services/backend/persistent_data/database/deploystack.db`.
+
+**For PostgreSQL:**
+The request body should be:
+```json
+{
+  "type": "postgres",
+  "connectionString": "postgresql://username:password@host:port/mydatabase"
+}
+```
+Replace the `connectionString` with your actual PostgreSQL connection URI.
+
+**Important:** After the initial database setup via this API, you **must restart the backend server** for the changes to take full effect and for the application to connect to the newly configured database.
+
+### Database Configuration File
+
+The choice of database (SQLite or PostgreSQL) and its specific configuration (like the connection string for PostgreSQL) is stored in a JSON file located at:
+
+- `services/backend/persistent_data/db.selection.json`
+
+This file is automatically managed by the setup API. You typically do not need to edit it manually.
+
 ## Key Components
 
 - **SQLite**: Embedded SQL database engine
@@ -117,13 +166,14 @@ You can inspect the SQLite database directly using various tools:
 - **SQLite CLI**:
 
   ```bash
-  sqlite3 ./data/deploystack.db
+  sqlite3 services/backend/persistent_data/database/deploystack.db
   ```
+  (Assuming the command is run from the project root directory)
 
 - **Visual Tools**: [DB Browser for SQLite](https://sqlitebrowser.org/) or VSCode extensions like SQLite Viewer
 
 ## Troubleshooting
 
 - If you get a "table already exists" error, check if you've already applied the migration
 - For complex schema changes, you may need to create multiple migrations
-- To reset the database, delete the `./data/deploystack.db` file and restart the server
+- To reset the database, delete the `services/backend/persistent_data/database/deploystack.db` file and restart the server

services/backend/DB.md

@@ -79,6 +79,20 @@ services/backend/
 └── tsconfig.json       # TypeScript configuration
 ```
 
+## 💾 Persistent Data
+
+The `services/backend/persistent_data/` directory is designated for storing all data that needs to persist across application restarts or deployments.
+
+**Purpose:**
+- To provide a single, consistent location for all persistent backend data.
+- When developing backend features that require data persistence (e.g., database files, configuration files that should not be in version control but are generated/modified at runtime), use this directory exclusively.
+
+**Examples of data stored here:**
+- SQLite database file (e.g., `persistent_data/database/deploystack.db`)
+- Database selection configuration (e.g., `persistent_data/db.selection.json`)
+
+This ensures that persistent data is managed in a predictable way and is not scattered across the project.
+
 ## 🌍 Environment Variables
 
 Create a `.env` file in the `services/backend` directory with the following variables: