Skip to content

Commit dcccf07

Browse files
joeldicksonjoel
joeldickson
and
joel
authored
Docs: Docker + PostgreSQL setup, client routing options, and deployment diagrams (#10)
* Docs: add Docker/PostgreSQL setup and deployment scenarios * Fix Mermaid diagram syntax for GitHub rendering * Docs: add compose example, k8s+helm scenario, and DNS guidance * Docs: add simple SQLite docker-compose deployment scenario * Docs: simplify SQLite compose example by removing connection string override * Docs: mark SQLite scenario as dev/testing only, not production * Docs: add concrete corporate DNS default-host example --------- Co-authored-by: Joel Dickson <joel@users.noreply.github.com>
1 parent 64eed6d commit dcccf07

File tree

2 files changed

+250
-16
lines changed

2 files changed

+250
-16
lines changed

‎README.md‎

Lines changed: 51 additions & 16 deletions
Original file line numberDiff line numberDiff line change
@@ -2,17 +2,17 @@
22

33
A full-stack developer experience telemetry dashboard that collects, stores, and visualizes build and test metrics from development tools.
44

5-
This is an internal tool designed to be deployed within your organization's infrastructure. It receives telemetry from build plugins and test runners across your engineering teams, providing visibility into compile times, test pass rates, hot reload performance, and other developer productivity signals. It is not intended to be exposed to the public internet.
5+
This is an internal tool designed to be deployed within your organization's infrastructure. It receives telemetry from build plugins and test runners across your engineering teams, providing visibility into compile times, test pass rates, hot reload performance, and other developer productivity signals. It is **not** intended to be exposed to the public internet.
66

77
## Architecture
88

9-
- **Backend:** .NET 10, ASP.NET Core (Kestrel), EF Core with SQLite
9+
- **Backend:** .NET 10, ASP.NET Core (Kestrel), EF Core with SQLite or PostgreSQL
1010
- **Frontend:** React 19, TypeScript, Vite 8, Tailwind CSS 3, Tremor, Recharts
11-
- **CI/CD:** GitHub Actions, Azure App Service (Linux)
11+
- **Container:** Docker image available at `agoda/devex-telemetry`
1212

1313
## Project Structure
1414

15-
```
15+
```text
1616
src/
1717
├── Agoda.DevExTelemetry.WebApi/ # ASP.NET Core API (controllers, Program.cs)
1818
├── Agoda.DevExTelemetry.Core/ # Domain layer (entities, DTOs, services, DbContext)
@@ -62,25 +62,62 @@ These are the client libraries that instrument developer tooling and send teleme
6262
- **API Build Performance** — compile, startup, and first response times
6363
- **Clientside Build Performance** — hot reload vs full build metrics
6464

65-
## Client Configuration
65+
## PostgreSQL Support
66+
67+
The API supports PostgreSQL when `POSTGRES_CONNECTION_STRING` is provided.
68+
69+
```bash
70+
export POSTGRES_CONNECTION_STRING='Host=localhost;Port=5432;Database=devex_telemetry;Username=devex;Password=devex'
71+
```
72+
73+
Notes:
74+
- If `POSTGRES_CONNECTION_STRING` is not set, the app uses SQLite.
75+
- Current PostgreSQL initialization uses `EnsureCreated()` as an MVP path.
76+
- For long-term schema evolution, move to dedicated PostgreSQL migrations and `db.Database.Migrate()`.
6677

67-
The telemetry clients (devfeedback NuGet packages, webpack/vite plugins, test reporter plugins) need to know where to send data. There are two ways to point them at your deployment:
78+
## Docker Usage
6879

69-
### Option 1: Environment Variable Override
80+
### Run from Docker Hub image
7081

71-
Set the `DEVFEEDBACK_URL` environment variable on developer machines to your server's URL:
82+
```bash
83+
docker run --rm -p 8080:8080 \
84+
-e POSTGRES_CONNECTION_STRING='Host=host.docker.internal;Port=5432;Database=devex_telemetry;Username=devex;Password=devex' \
85+
agoda/devex-telemetry:latest
86+
```
87+
88+
### Run with included docker-compose (API + PostgreSQL)
7289

7390
```bash
74-
export DEVFEEDBACK_URL=https://your-devex-telemetry.azurewebsites.net
91+
docker compose up -d
7592
```
7693

77-
This overrides the default URL in all compilation metrics clients. You can set this in shell profiles, dotfiles, or machine-level environment variables.
94+
This starts:
95+
- API on `http://localhost:8080`
96+
- PostgreSQL on `localhost:5432`
97+
98+
## Pointing Clients at Your Deployment
7899

79-
### Option 2: Internal DNS
100+
Two options, depending on how much per-machine configuration you want.
80101

81-
If you control your organization's DNS, you can create a DNS record that resolves the default hostname used by the compilation metrics clients to your deployment's IP. This way clients work without any local configuration — traffic is routed transparently via your internal DNS server.
102+
### Option 1 (preferred for larger teams): Internal DNS
82103

83-
This approach is preferable for large teams since it requires zero setup on individual developer machines.
104+
The compilation clients default to `http://compilation-metrics`.
105+
106+
Create an internal DNS record so `compilation-metrics` resolves to wherever you host this API. This gives zero per-machine setup: engineers don’t configure anything, telemetry just flows.
107+
108+
### Option 2: Environment variable override per machine
109+
110+
Set `DEVFEEDBACK_URL` on workstations:
111+
112+
```bash
113+
export DEVFEEDBACK_URL=https://your-devex-telemetry.example.com
114+
```
115+
116+
This is useful when DNS changes aren’t available yet. Your IT support team can roll this out centrally via endpoint management.
117+
118+
## Deployment Scenarios and Diagrams
119+
120+
See [docs/deployment-scenarios.md](docs/deployment-scenarios.md) for concrete deployment topologies, client routing examples, and markdown diagrams.
84121

85122
## Development
86123

@@ -117,9 +154,7 @@ cd src
117154
dotnet test
118155
```
119156

120-
83 tests total (21 unit + 62 integration).
121-
122-
## Deployment
157+
## CI/CD
123158

124159
Pushes to `main` trigger automatic deployment to Azure App Service via GitHub Actions.
125160

‎docs/deployment-scenarios.md‎

Lines changed: 199 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,199 @@
1+
# Deployment Scenarios
2+
3+
This document shows common deployment patterns for Agoda.DevExTelemetry and how telemetry clients connect in each case.
4+
5+
## Scenario A: Single-host Docker Compose (small team / PoC)
6+
7+
- API and PostgreSQL run on one host.
8+
- Good for quick validation and small-team internal usage.
9+
10+
```mermaid
11+
flowchart LR
12+
DevMachines["Developer machines<br/>(build/test clients)"] -->|HTTP metrics| API["DevExTelemetry API<br/>Docker container"]
13+
API --> DB[("PostgreSQL<br/>Docker volume")]
14+
```
15+
16+
### Example `docker-compose.yml`
17+
18+
```yaml
19+
services:
20+
app:
21+
image: agoda/devex-telemetry:latest
22+
ports:
23+
- "8080:8080"
24+
environment:
25+
- POSTGRES_CONNECTION_STRING=Host=db;Port=5432;Database=devex_telemetry;Username=devex;Password=devex
26+
depends_on:
27+
- db
28+
29+
db:
30+
image: postgres:17-alpine
31+
environment:
32+
POSTGRES_DB: devex_telemetry
33+
POSTGRES_USER: devex
34+
POSTGRES_PASSWORD: devex
35+
volumes:
36+
- pgdata:/var/lib/postgresql/data
37+
ports:
38+
- "5432:5432"
39+
40+
volumes:
41+
pgdata:
42+
```
43+
44+
### Pros
45+
- Fastest setup
46+
- Minimal infra dependencies
47+
48+
### Cons
49+
- Single host is a SPOF
50+
- Limited scaling
51+
52+
---
53+
54+
## Scenario B: Kubernetes + Helm (app) with externally managed PostgreSQL
55+
56+
- App is deployed to Kubernetes via Helm.
57+
- PostgreSQL is managed externally (Azure Database for PostgreSQL / RDS / Cloud SQL / internal managed DB).
58+
- App receives DB connectivity via `POSTGRES_CONNECTION_STRING` in Helm values.
59+
60+
```mermaid
61+
flowchart LR
62+
Clients["Developer machines"] --> DNS["Internal DNS<br/>compilation-metrics"]
63+
DNS --> Ingress["K8s Ingress / Internal LB"]
64+
Ingress --> App["DevExTelemetry API<br/>Kubernetes pods"]
65+
App --> DB[("Managed PostgreSQL<br/>outside cluster")]
66+
```
67+
68+
### Minimal Helm chart (app layer)
69+
70+
#### `Chart.yaml`
71+
72+
```yaml
73+
apiVersion: v2
74+
name: devex-telemetry
75+
version: 0.1.0
76+
appVersion: "latest"
77+
```
78+
79+
#### `values.yaml`
80+
81+
```yaml
82+
image:
83+
repository: agoda/devex-telemetry
84+
tag: latest
85+
pullPolicy: IfNotPresent
86+
87+
replicaCount: 2
88+
89+
service:
90+
type: ClusterIP
91+
port: 8080
92+
93+
ingress:
94+
enabled: true
95+
className: nginx
96+
host: devex-telemetry.internal.example.com
97+
98+
env:
99+
POSTGRES_CONNECTION_STRING: "Host=managed-pg.internal;Port=5432;Database=devex_telemetry;Username=devex;Password=***"
100+
```
101+
102+
#### `templates/deployment.yaml` (env excerpt)
103+
104+
```yaml
105+
env:
106+
- name: POSTGRES_CONNECTION_STRING
107+
value: {{ .Values.env.POSTGRES_CONNECTION_STRING | quote }}
108+
```
109+
110+
### Pros
111+
- Better scalability and operability
112+
- Clean separation between app runtime and managed database
113+
114+
### Cons
115+
- Requires Kubernetes + Helm + ingress setup
116+
- Requires platform/DNS coordination
117+
118+
---
119+
120+
## Scenario C: SQLite-only Docker Compose (ultra-simple local setup)
121+
122+
- Runs only the app container.
123+
- Uses SQLite file storage inside a mounted Docker volume.
124+
- Best for solo usage, demos, and very small temporary setups.
125+
- **For development and testing only — not for production use.**
126+
127+
```mermaid
128+
flowchart LR
129+
DevMachines["Developer machines"] -->|HTTP metrics| API["DevExTelemetry API<br/>Docker container"]
130+
API --> SQLITE[("SQLite DB file<br/>mounted volume")]
131+
```
132+
133+
### Example `docker-compose.yml`
134+
135+
```yaml
136+
services:
137+
app:
138+
image: agoda/devex-telemetry:latest
139+
ports:
140+
- "8080:8080"
141+
volumes:
142+
- telemetry-data:/app/data
143+
144+
volumes:
145+
telemetry-data:
146+
```
147+
148+
### Pros
149+
- Simplest possible deployment
150+
- No separate database service required
151+
152+
### Cons
153+
- Not suitable for multi-instance scaling
154+
- Lower durability/performance characteristics vs managed PostgreSQL
155+
156+
157+
## Pointing Clients at Your Deployment
158+
159+
Compilation/test clients route as follows:
160+
161+
1. If `DEVFEEDBACK_URL` is set, use it.
162+
2. Otherwise, use default `http://compilation-metrics`.
163+
164+
### Preferred enterprise pattern: DNS default host
165+
166+
Most corporate networks already use internal DNS for service discovery (for internal APIs, proxies, package mirrors, etc.).
167+
168+
Because clients default to `http://compilation-metrics`, your network/workstation team can create an internal DNS record for `compilation-metrics` pointing to your telemetry API endpoint (ingress/internal LB/service host).
169+
170+
Example (corporate internal DNS):
171+
- You may already have internal domains like `yourcompany.local`.
172+
- Teams might already access internal systems such as `sql01.yourcompany.local`.
173+
- Add a DNS record like `compilation-metrics.yourcompany.local` (or a short host alias `compilation-metrics`) and map it to your telemetry endpoint.
174+
- Once that DNS path exists, the default client URL can resolve and telemetry will flow with zero local setup.
175+
176+
Once this is in place:
177+
178+
- no per-developer setup is required,
179+
- telemetry works out-of-the-box,
180+
- rollout and changes stay centralized with infra/network teams.
181+
182+
If you don’t control DNS yet, use `DEVFEEDBACK_URL` as a temporary bridge.
183+
184+
### Workstation override (optional)
185+
186+
```bash
187+
export DEVFEEDBACK_URL='https://your-devex-telemetry.example.com'
188+
```
189+
190+
Your IT support team can also push this centrally with endpoint/device management tooling if needed.
191+
192+
---
193+
194+
## Security notes
195+
196+
- Keep the API internal (VPN/private network/internal ingress).
197+
- Do not expose PostgreSQL directly to the public internet.
198+
- Use TLS for cross-network telemetry traffic.
199+
- Prefer secrets management (K8s secrets / vault / parameter store) over plaintext credentials in Helm values.

0 commit comments

Comments
 (0)