Skip to content

Commit 6a7a3f4

Browse files
ZoneMixZoneMix
committed
Add PostgreSQL support and improve database initialization
- Add PostgreSQL as default database option alongside SQLite - Create separate production docker-compose files for PostgreSQL and SQLite - Fix PostgreSQL schema initialization race condition - Add input validation for client IDs, usernames, UUIDs, and auth codes - Update Docker Compose to use bind mounts instead of named volumes - Add .env.postgres file support for Docker Compose PostgreSQL service - Refactor database layer to support both SQLite and PostgreSQL with unified async API - Update all database operations to be async - Add proper schema initialization waiting mechanism - Update monitoring configuration paths - Improve environment variable documentation
1 parent edbcd08 commit 6a7a3f4

25 files changed

+1130
-176
lines changed

‎.env.example‎

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -172,6 +172,39 @@ ALLOWED_ORIGINS=http://localhost:3000,http://localhost:5678
172172
# Default: info
173173
LOG_LEVEL=info
174174

175+
# =============================================================================
176+
# DATABASE CONFIGURATION (Authentication DB)
177+
# =============================================================================
178+
# PostgreSQL is the default database (provides password authentication).
179+
# SQLite is available as an alternative for simple deployments.
180+
# =============================================================================
181+
182+
# Database type: 'postgres' (default) or 'sqlite'
183+
# Default: postgres
184+
# DB_TYPE=postgres
185+
186+
# PostgreSQL connection - use EITHER POSTGRES_URL OR individual components below
187+
# Option 1: Connection URL (all in one)
188+
# Format: postgresql://user:password@host:port/database
189+
# Optional: Only set if you prefer using a connection URL
190+
# POSTGRES_URL=postgresql://actual_api:password@localhost:5432/actual_api_auth
191+
192+
# Option 2: Individual connection components (recommended - easier to manage)
193+
# These will be automatically combined into a connection URL if POSTGRES_URL is not set
194+
# IMPORTANT: Use 'localhost' when running locally, 'postgres' when running in Docker Compose
195+
POSTGRES_HOST=localhost
196+
POSTGRES_PORT=5432
197+
POSTGRES_DB=actual_api_auth
198+
POSTGRES_USER=actual_api
199+
POSTGRES_PASSWORD=your_postgres_password
200+
201+
# Note: When using Docker Compose, POSTGRES_HOST will be overridden to 'postgres' automatically
202+
# For local development (npm start), use 'localhost' in your .env file
203+
204+
# To use SQLite instead, set:
205+
# DB_TYPE=sqlite
206+
# (No additional configuration needed for SQLite)
207+
175208
# =============================================================================
176209
# REDIS CONFIGURATION (Optional)
177210
# =============================================================================
@@ -188,6 +221,7 @@ LOG_LEVEL=info
188221

189222
# Redis connection details (alternative to REDIS_URL)
190223
# Optional: Only set if using Redis for rate limiting
224+
# IMPORTANT: Use 'localhost' when running locally, 'redis' when running in Docker Compose
191225
# REDIS_HOST=localhost
192226
# REDIS_PORT=6379
193227
# REDIS_PASSWORD=your_redis_password
@@ -215,6 +249,10 @@ LOG_LEVEL=info
215249
# - N8N_CLIENT_SECRET (32+ chars)
216250
# - N8N_OAUTH2_CALLBACK_URL
217251
#
252+
# OPTIONAL FOR DATABASE (PostgreSQL):
253+
# - DB_TYPE='postgres' (if not set, defaults to 'sqlite')
254+
# - POSTGRES_URL or all of: POSTGRES_HOST, POSTGRES_DB, POSTGRES_USER, POSTGRES_PASSWORD
255+
#
218256
# SECURITY CHECKLIST:
219257
# □ All secrets are unique and randomly generated
220258
# □ Secrets are at least 32 characters long

‎README.md‎

Lines changed: 10 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -153,10 +153,18 @@ This project uses **dotenvx** to encrypt environment files in production.
153153

154154
### Production Deployment
155155

156-
**Docker Compose** (recommended):
156+
**Docker Compose with PostgreSQL** (recommended for production):
157157
```bash
158158
export DOTENV_PRIVATE_KEY="$(grep DOTENV_PRIVATE_KEY .env.keys | cut -d '=' -f2 | tail -n1)"
159-
docker compose up -d --build
159+
# Create .env.postgres file with: POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB
160+
docker compose -f docker-compose.prod.postgres.yml up -d --build
161+
```
162+
163+
**Docker Compose with SQLite** (simpler, single-container):
164+
```bash
165+
export DOTENV_PRIVATE_KEY="$(grep DOTENV_PRIVATE_KEY .env.keys | cut -d '=' -f2 | tail -n1)"
166+
# Set DB_TYPE=sqlite in your .env file
167+
docker compose -f docker-compose.prod.sqlite.yml up -d --build
160168
```
161169

162170
**Docker Image**:

‎docker-compose.dev.yml‎

Lines changed: 49 additions & 35 deletions
Original file line numberDiff line numberDiff line change
@@ -1,31 +1,30 @@
11
services:
2-
actual-api-wrapper:
2+
actual-rest-api-dev:
33
build:
44
context: .
55
dockerfile: Dockerfile.dev
6-
image: actual-api-wrapper-dev
7-
container_name: actual-api-wrapper-dev
6+
image: actual-rest-api-dev
7+
container_name: actual-rest-api-dev
88
ports:
99
- "3000:3000"
1010
volumes:
11-
- ./data/dev/actual-api-wrapper:/app/.actual-cache
11+
- ./data/dev/actual-rest-api:/app/.actual-cache
1212
- ./.env.local:/app/.env:ro
1313
working_dir: /app
1414
command: >
1515
sh -c "dotenvx run -- npm start"
1616
environment:
17-
# Redis connection (optional - will use memory store if not set)
18-
REDIS_HOST: redis
19-
REDIS_PORT: 6379
20-
REDIS_PASSWORD: ${REDIS_PASSWORD:-} # Optional: Set if Redis password is configured
21-
# Or use REDIS_URL instead: redis://redis:6379 or redis://password@redis:6379
22-
# Log level: Set to 'warn' or 'error' to reduce log verbosity
23-
# LOG_LEVEL: ${LOG_LEVEL:-info}
17+
NODE_ENV: development
18+
DB_TYPE: postgres
19+
REDIS_HOST: redis-dev
20+
POSTGRES_HOST: postgres-dev
2421
depends_on:
25-
actual-server:
22+
actual-server-dev:
2623
condition: service_healthy
2724
restart: true
28-
redis:
25+
redis-dev:
26+
condition: service_healthy
27+
postgres-dev:
2928
condition: service_healthy
3029
healthcheck:
3130
test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider http://localhost:3000/health || exit 1"]
@@ -40,13 +39,13 @@ services:
4039
max-file: "3"
4140
compress: "true"
4241

43-
redis:
42+
redis-dev:
4443
image: redis:7-alpine
45-
container_name: actual-api-redis-dev
44+
container_name: actual-rest-api-redis-dev
4645
ports:
4746
- "6379:6379"
4847
volumes:
49-
- redis-data-dev:/data
48+
- ./data/dev/redis:/data
5049
# Redis runs without password by default in dev (can be set via REDIS_PASSWORD env var)
5150
# For production, always set a password
5251
command: redis-server --appendonly yes
@@ -66,7 +65,7 @@ services:
6665
max-file: "2"
6766
compress: "true"
6867

69-
n8n:
68+
n8n-dev:
7069
image: n8nio/n8n:latest
7170
container_name: n8n-dev
7271
ports:
@@ -80,15 +79,15 @@ services:
8079
# For local development of the Actual Budget REST API n8n node
8180
- ./n8n-nodes-actual-budget-rest-api:/home/node/.n8n/custom/node_modules/n8n-nodes-actual-budget-rest-api
8281
depends_on:
83-
- actual-api-wrapper
82+
- actual-rest-api-dev
8483
logging:
8584
driver: "json-file"
8685
options:
8786
max-size: "10m"
8887
max-file: "3"
8988
compress: "true"
9089

91-
actual-server:
90+
actual-server-dev:
9291
container_name: actual-server-dev
9392
image: docker.io/actualbudget/actual-server:latest
9493
ports:
@@ -111,23 +110,23 @@ services:
111110
max-file: "2"
112111
compress: "true"
113112

114-
prometheus:
113+
prometheus-dev:
115114
image: prom/prometheus:latest
116115
container_name: prometheus-dev
117116
ports:
118117
- "9090:9090"
119118
volumes:
120-
- ./data/dev/prometheus:/etc/prometheus
121-
- prometheus-data-dev:/prometheus
119+
- ./data/dev/prometheus:/prometheus
120+
- ./monitoring/prometheus.yml:/prometheus/prometheus.yml
122121
command:
123-
- '--config.file=/etc/prometheus/prometheus.yml'
122+
- '--config.file=/prometheus/prometheus.yml'
124123
- '--storage.tsdb.path=/prometheus'
125124
- '--web.console.libraries=/usr/share/prometheus/console_libraries'
126125
- '--web.console.templates=/usr/share/prometheus/consoles'
127126
- '--storage.tsdb.retention.time=30d'
128127
- '--web.enable-lifecycle'
129128
depends_on:
130-
- actual-api-wrapper
129+
- actual-rest-api-dev
131130
restart: unless-stopped
132131
logging:
133132
driver: "json-file"
@@ -136,7 +135,7 @@ services:
136135
max-file: "2"
137136
compress: "true"
138137

139-
grafana:
138+
grafana-dev:
140139
image: grafana/grafana:latest
141140
container_name: grafana-dev
142141
ports:
@@ -147,20 +146,35 @@ services:
147146
- GF_USERS_ALLOW_SIGN_UP=false
148147
- GF_SERVER_ROOT_URL=http://localhost:3001
149148
volumes:
150-
- grafana-data-dev:/var/lib/grafana
151-
- ./data/dev/grafana/provisioning:/etc/grafana/provisioning
152-
- ./data/dev/grafana/dashboards:/var/lib/grafana/dashboards
149+
- ./data/dev/grafana/data:/var/lib/grafana
150+
- ./monitoring/grafana/provisioning:/etc/grafana/provisioning
151+
- ./monitoring/grafana/dashboards:/var/lib/grafana/dashboards
153152
depends_on:
154-
- prometheus
153+
- prometheus-dev
154+
restart: unless-stopped
155+
156+
postgres-dev:
157+
image: postgres:16-alpine
158+
container_name: postgres-dev
159+
ports:
160+
- "5432:5432"
161+
environment:
162+
# Fallback defaults if .env.postgres doesn't exist
163+
POSTGRES_USER: ${POSTGRES_USER:-postgres}
164+
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD:-dev_password}
165+
POSTGRES_DB: ${POSTGRES_DB:-postgres}
166+
volumes:
167+
- ./data/dev/postgres:/var/lib/postgresql/data
168+
healthcheck:
169+
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-postgres}"]
170+
interval: 10s
171+
timeout: 3s
172+
retries: 5
173+
start_period: 10s
155174
restart: unless-stopped
156175
logging:
157176
driver: "json-file"
158177
options:
159-
max-size: "10m"
178+
max-size: "5m"
160179
max-file: "2"
161180
compress: "true"
162-
163-
volumes:
164-
redis-data-dev:
165-
prometheus-data-dev:
166-
grafana-data-dev:

‎docker-compose.prod.postgres.yml‎

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
services:
2+
actual-api-rest-api:
3+
build: . # Builds Dockerfile (copies from ./src)
4+
image: actual-api-rest-api # Tagged for reuse
5+
container_name: actual-api-rest-api
6+
ports:
7+
- "3000:3000"
8+
volumes:
9+
- ./data/prod/actual-api:/app/.actual-cache # Persistent encrypted DB
10+
working_dir: /app
11+
command: >
12+
sh -c "dotenvx run -- npm start"
13+
environment:
14+
DOTENV_PRIVATE_KEY: ${DOTENV_PRIVATE_KEY} # Master key from host .env or secrets
15+
# Redis connection (optional - will use memory store if not set)
16+
REDIS_HOST: redis
17+
REDIS_PORT: 6379
18+
REDIS_PASSWORD: ${REDIS_PASSWORD:-} # Optional: Set if Redis password is configured
19+
# Or use REDIS_URL instead: redis://redis:6379 or redis://password@redis:6379
20+
# Database connection (PostgreSQL)
21+
DB_TYPE: postgres
22+
# Override POSTGRES_HOST for Docker Compose (use service name 'postgres' instead of 'localhost')
23+
POSTGRES_HOST: postgres
24+
depends_on:
25+
redis:
26+
condition: service_healthy
27+
postgres:
28+
condition: service_healthy
29+
healthcheck:
30+
test: ["CMD-SHELL", "wget --no-verbose --tries=1 --spider http://localhost:3000/health || exit 1"]
31+
interval: 30s
32+
timeout: 10s
33+
retries: 3
34+
start_period: 40s
35+
36+
redis:
37+
image: redis:7-alpine
38+
container_name: actual-rest-api-redis
39+
ports:
40+
- "6379:6379"
41+
volumes:
42+
- ./data/prod/redis:/data
43+
# Redis password (set REDIS_PASSWORD env var for production)
44+
# In production, always use a strong password
45+
command: >
46+
sh -c "if [ -n \"$$REDIS_PASSWORD\" ]; then
47+
redis-server --appendonly yes --requirepass \"$$REDIS_PASSWORD\"
48+
else
49+
redis-server --appendonly yes
50+
fi"
51+
environment:
52+
- REDIS_PASSWORD=${REDIS_PASSWORD:-}
53+
healthcheck:
54+
test: ["CMD", "redis-cli", "ping"]
55+
interval: 10s
56+
timeout: 3s
57+
retries: 5
58+
start_period: 10s
59+
restart: unless-stopped
60+
61+
postgres:
62+
image: postgres:16-alpine
63+
container_name: actual-rest-api-postgres
64+
ports:
65+
- "5432:5432"
66+
env_file:
67+
# Use separate .env.postgres file since Docker Compose can't read encrypted dotenvx files
68+
# Create .env.postgres with: POSTGRES_USER, POSTGRES_PASSWORD, POSTGRES_DB
69+
- .env.postgres
70+
- .env.postgres.local # Optional: local overrides (gitignored)
71+
environment:
72+
# Fallback defaults if .env.postgres doesn't exist
73+
POSTGRES_USER: ${POSTGRES_USER:-actual_api}
74+
POSTGRES_PASSWORD: ${POSTGRES_PASSWORD}
75+
POSTGRES_DB: ${POSTGRES_DB:-actual_api_auth}
76+
volumes:
77+
- ./data/prod/postgres:/var/lib/postgresql/data
78+
healthcheck:
79+
test: ["CMD-SHELL", "pg_isready -U ${POSTGRES_USER:-actual_api}"]
80+
interval: 10s
81+
timeout: 3s
82+
retries: 5
83+
start_period: 10s
84+
restart: unless-stopped
85+

docker-compose.yml renamed to docker-compose.prod.sqlite.yml

Lines changed: 9 additions & 9 deletions
Original file line numberDiff line numberDiff line change
@@ -1,12 +1,12 @@
11
services:
2-
actual-api-wrapper:
2+
actual-rest-api:
33
build: . # Builds Dockerfile (copies from ./src)
4-
image: actual-api-wrapper # Tagged for reuse
5-
container_name: actual-api-wrapper
4+
image: actual-rest-api # Tagged for reuse
5+
container_name: actual-rest-api
66
ports:
77
- "3000:3000"
88
volumes:
9-
- ./data/actual-api:/app/.actual-cache # Persistent encrypted DB
9+
- ./data/prod/actual-api:/app/.actual-cache # Persistent encrypted DB and SQLite auth.db
1010
working_dir: /app
1111
command: >
1212
sh -c "dotenvx run -- npm start"
@@ -17,6 +17,9 @@ services:
1717
REDIS_PORT: 6379
1818
REDIS_PASSWORD: ${REDIS_PASSWORD:-} # Optional: Set if Redis password is configured
1919
# Or use REDIS_URL instead: redis://redis:6379 or redis://password@redis:6379
20+
# Database connection (SQLite)
21+
DB_TYPE: sqlite
22+
# SQLite database will be stored in DATA_DIR/auth.db (default: /app/.actual-cache/auth.db)
2023
depends_on:
2124
redis:
2225
condition: service_healthy
@@ -29,11 +32,11 @@ services:
2932

3033
redis:
3134
image: redis:7-alpine
32-
container_name: actual-api-redis
35+
container_name: actual-rest-api-redis
3336
ports:
3437
- "6379:6379"
3538
volumes:
36-
- redis-data:/data
39+
- ./data/prod/redis:/data
3740
# Redis password (set REDIS_PASSWORD env var for production)
3841
# In production, always use a strong password
3942
command: >
@@ -52,6 +55,3 @@ services:
5255
start_period: 10s
5356
restart: unless-stopped
5457

55-
volumes:
56-
auth-data:
57-
redis-data:

‎monitoring/README.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -173,3 +173,4 @@ For production deployments:
173173
- [Grafana Documentation](https://grafana.com/docs/grafana/latest/)
174174
- [PromQL Query Language](https://prometheus.io/docs/prometheus/latest/querying/basics/)
175175

176+

‎monitoring/grafana/dashboards/actual-budget-api-dashboard.json‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -884,3 +884,4 @@
884884
"version": 1
885885
}
886886

887+

0 commit comments

Comments
 (0)