|
| 1 | +# SymfonyCon 2025 - Scaling PHP Systems |
| 2 | + |
| 3 | +This project demonstrates a scalable Symfony CQRS application with Redis and DB projections, Docker Compose, and k6 load |
| 4 | +testing. All common tasks are managed via the Makefile. |
| 5 | + |
| 6 | +## Setup |
| 7 | + |
| 8 | +### Required Dependencies |
| 9 | + |
| 10 | +1. **Docker & Docker Compose** |
| 11 | + - [Docker Desktop](https://www.docker.com/products/docker-desktop/) (includes Docker Compose) |
| 12 | + - Or install separately: [Docker](https://docs.docker.com/get-docker/) |
| 13 | + and [Docker Compose](https://docs.docker.com/compose/install/) |
| 14 | + |
| 15 | +2. **Make** |
| 16 | + - **macOS**: Usually pre-installed, or install via Homebrew: `brew install make` |
| 17 | + - **Linux**: Install via package manager: |
| 18 | + - Ubuntu/Debian: `sudo apt-get install make` |
| 19 | + - CentOS/RHEL: `sudo yum install make` |
| 20 | + - Fedora: `sudo dnf install make` |
| 21 | + |
| 22 | +3. **K6 (Load Testing Tool)** |
| 23 | + - **macOS**: `brew install k6` |
| 24 | + - **Linux**: |
| 25 | + - Ubuntu/Debian: `sudo gpg -k` |
| 26 | + ```bash |
| 27 | + sudo gpg --no-default-keyring --keyring /usr/share/keyrings/k6-archive-keyring.gpg --keyserver hkp://keyserver.ubuntu.com:80 --recv-keys C5AD17C747E3415A3642D57D77C6C491D6AC1D69 |
| 28 | + echo "deb [signed-by=/usr/share/keyrings/k6-archive-keyring.gpg] https://dl.k6.io/deb stable main" | sudo tee /etc/apt/sources.list.d/k6.list |
| 29 | + sudo apt-get update |
| 30 | + sudo apt-get install k6 |
| 31 | + ``` |
| 32 | + - CentOS/RHEL/Fedora: `sudo yum install k6` or `sudo dnf install k6` |
| 33 | + |
| 34 | +### Verify Installation |
| 35 | + |
| 36 | +Run the test script to verify all dependencies are installed: |
| 37 | + |
| 38 | +```bash |
| 39 | +./test-system.sh |
| 40 | +``` |
| 41 | + |
| 42 | +This will check for Docker, Docker Compose, Make, and K6 and report their status. |
| 43 | + |
| 44 | +### Docker Images Setup |
| 45 | + |
| 46 | +Before starting the application, you need to pull the required Docker images. You can do this in two ways: |
| 47 | + |
| 48 | +**Pull all images at once** |
| 49 | + |
| 50 | +```bash |
| 51 | +make pull-docker |
| 52 | +``` |
| 53 | + |
| 54 | +This ensures all required Docker images are available locally before starting the services. |
| 55 | + |
| 56 | +## Quick Start |
| 57 | + |
| 58 | +1. **Build and start all services:** |
| 59 | + |
| 60 | +```bash |
| 61 | +make up |
| 62 | +``` |
| 63 | + |
| 64 | +2. **Set up the database and seed data:** |
| 65 | + |
| 66 | +```bash |
| 67 | +make migrate |
| 68 | +make seed |
| 69 | +``` |
| 70 | + |
| 71 | +Go to http://localhost:8088 |
| 72 | + |
| 73 | +### opcache introduction |
| 74 | + |
| 75 | +``` |
| 76 | +make up-opcache-dashboard |
| 77 | +``` |
| 78 | +
|
| 79 | +open - http://localhost:42042/opcache/status |
| 80 | +
|
| 81 | + |
| 82 | +
|
| 83 | +https://www.php.net/manual/en/opcache.configuration.php#:~:text=on%20all%20architectures.-,opcache.max_accelerated_files,-int |
| 84 | +
|
| 85 | +``` |
| 86 | +find . -type f -name "*.php" | wc -l |
| 87 | +``` |
| 88 | +
|
| 89 | +``` |
| 90 | +opcache.max_accelerated_files=16087 |
| 91 | +``` |
| 92 | +
|
| 93 | +Prime Number: 10000 -> 10007 (nearest prime) |
| 94 | +
|
| 95 | +### show fpm and opcache dashboard GUI |
| 96 | +
|
| 97 | +show fpm status page - http://localhost:8088/fpm-status |
| 98 | +
|
| 99 | + |
| 100 | +
|
| 101 | +``` |
| 102 | +fpm.conf - pm.status_path = /fpm-status |
| 103 | +aa-nginx.conf - location ~ ^/fpm-status$ { |
| 104 | + |
| 105 | +``` |
| 106 | +
|
| 107 | +**fpm-exporter** |
| 108 | +
|
| 109 | +``` |
| 110 | +make up-exporter |
| 111 | +make ps | grep exporter |
| 112 | +``` |
| 113 | +
|
| 114 | +Go to http://localhost:9253/metrics |
| 115 | +
|
| 116 | + |
| 117 | +
|
| 118 | +``` |
| 119 | +make up-prometheus |
| 120 | +make ps | grep prom |
| 121 | +``` |
| 122 | +
|
| 123 | +Go to http://localhost:9090/targets?search= |
| 124 | +
|
| 125 | + |
| 126 | +
|
| 127 | +``` |
| 128 | +make up-grafana |
| 129 | +make ps | grep grafana |
| 130 | +``` |
| 131 | +
|
| 132 | +Go to http://localhost:3000/, choose Dashboard in the sidebar and click `PHP-FPM Performance Dashboard` |
| 133 | +
|
| 134 | + |
| 135 | +
|
| 136 | +### Show target prom sources |
| 137 | +
|
| 138 | +http://localhost:9090/targets?search= |
| 139 | +
|
| 140 | +### view grafana dashboard |
| 141 | +
|
| 142 | +open http://localhost:3000 |
| 143 | +username: symfony |
| 144 | +password: symfony |
| 145 | +
|
| 146 | +### show grafana fpm/opcache dashboard |
| 147 | +
|
| 148 | +```bash |
| 149 | +make benchmark-product-random-fpm |
| 150 | +``` |
| 151 | + |
| 152 | +Output: |
| 153 | + |
| 154 | + |
| 155 | +see file inside k6/report-UTC-xxxxxxx.html |
| 156 | +> i.e: k6/report-product-by-id-random-8088-2025-11-25T10-45-06.548Z.html |
| 157 | +
|
| 158 | +Check grafana output - http://localhost:3000 |
| 159 | + |
| 160 | +See fpm active processes. change to 5m (on left side) |
| 161 | +http://localhost:3000/d/phpfpm-performance/php-fpm-performance-dashboard?orgId=1&from=now-5m&to=now&timezone=browser&var-datasource=PBFA97CFB590B2093&var-pool=www&refresh=5s |
| 162 | + |
| 163 | +## PHP-FPM Performance Monitoring & Optimization |
| 164 | + |
| 165 | +### Accessing FPM Status & Metrics |
| 166 | + |
| 167 | +**FPM Status Page:** |
| 168 | +- URL: http://localhost:8088/fpm-status |
| 169 | +- Shows real-time process states, active/idle workers, queue length |
| 170 | +- Configured in `fpm.conf` with `pm.status_path = /fpm-status` |
| 171 | + |
| 172 | +**Prometheus Metrics:** |
| 173 | +- URL: http://localhost:9253/metrics (via php-fpm-exporter) |
| 174 | +- Scraped by Prometheus for historical data and alerting |
| 175 | +- Visualized in Grafana dashboards |
| 176 | + |
| 177 | +### Right-Sizing PHP-FPM Pool Configuration |
| 178 | + |
| 179 | +PHP-FPM pool sizing is critical for optimal performance. You need to balance: |
| 180 | +- Available RAM |
| 181 | +- Expected concurrent requests |
| 182 | +- Per-request memory usage |
| 183 | +- Response time requirements |
| 184 | + |
| 185 | +**Configuration Logic:** |
| 186 | + |
| 187 | + |
| 188 | + |
| 189 | +**Formula for `pm.max_children`:** |
| 190 | +``` |
| 191 | +pm.max_children = (Total Available RAM - RAM for other services) / Average PHP Process Memory |
| 192 | +``` |
| 193 | + |
| 194 | +**Example Calculation:** |
| 195 | +``` |
| 196 | +Server RAM: 4GB (4096 MB) |
| 197 | +System + MySQL: 1GB (1024 MB) |
| 198 | +Average PHP process: 50 MB |
| 199 | +
|
| 200 | +pm.max_children = (4096 - 1024) / 50 = 61 processes |
| 201 | +``` |
| 202 | + |
| 203 | +**FPM Calculator Tool:** |
| 204 | + |
| 205 | +Use the interactive calculator at https://spot13.com/pmcalculator/ to determine optimal settings: |
| 206 | + |
| 207 | + |
| 208 | + |
| 209 | +### Key Configuration Settings |
| 210 | + |
| 211 | +PHP-FPM uses a process manager to handle incoming requests efficiently. The configuration directly affects what you see in system monitoring tools like `htop`. |
| 212 | + |
| 213 | +```ini |
| 214 | +pm = dynamic # Process manager type (static, dynamic, ondemand) |
| 215 | +pm.max_children = 50 # Maximum number of child processes |
| 216 | +pm.start_servers = 5 # Number of children created on startup |
| 217 | +pm.min_spare_servers = 5 # Minimum idle processes |
| 218 | +pm.max_spare_servers = 35 # Maximum idle processes |
| 219 | +pm.max_requests = 500 # Requests before process restart (helps with memory leaks) |
| 220 | +``` |
| 221 | + |
| 222 | +**Process Manager Types:** |
| 223 | +- `static` - Fixed number of processes (best for consistent load) |
| 224 | +- `dynamic` - Scales between min/max spare servers (good for variable load) |
| 225 | +- `ondemand` - Creates processes on demand (best for low/intermittent traffic) |
| 226 | + |
| 227 | +**Important:** `pm.start_servers = 5` means you will see **5 child processes** in `htop` when the container starts, plus 1 master process (total 6 PHP-FPM processes). |
| 228 | + |
| 229 | +### Process Hierarchy and Behavior |
| 230 | + |
| 231 | +When you run `htop` or `ps aux | grep php-fpm`, you'll see: |
| 232 | + |
| 233 | +``` |
| 234 | +1 × php-fpm: master process (manages child processes) |
| 235 | +5 × php-fpm: pool www (child processes handling requests) |
| 236 | +``` |
| 237 | + |
| 238 | +**Process Roles:** |
| 239 | +- **Master Process** - Manages child processes, handles signals, doesn't serve requests |
| 240 | +- **Child Processes** - Handle actual HTTP requests from nginx/web server |
| 241 | +- **Dynamic Scaling** - Processes spawn/die based on load (between `min_spare_servers` and `max_spare_servers`) |
| 242 | + |
| 243 | +**Process States:** |
| 244 | +- `Idle` - Waiting for requests |
| 245 | +- `Running` - Actively processing a request |
| 246 | +- `Finishing` - Completing request cleanup |
| 247 | +- `Reading headers` - Parsing request headers |
| 248 | +- `Ending` - Process is shutting down |
| 249 | + |
| 250 | +## Monitoring Commands (Run Inside Container During k6 Tests) |
| 251 | + |
| 252 | +### Check Real-Time Process Usage |
| 253 | + |
| 254 | +**List processes sorted by memory (RSS):** |
| 255 | +```bash |
| 256 | +ps -ylC php-fpm --sort:rss |
| 257 | +``` |
| 258 | + |
| 259 | +**Watch process states in real-time:** |
| 260 | +```bash |
| 261 | +watch -n 1 'ps aux | grep php-fpm' |
| 262 | +``` |
| 263 | + |
| 264 | +**Count active vs idle processes:** |
| 265 | +```bash |
| 266 | +curl -s http://localhost:8088/fpm-status | grep -E "active|idle" |
| 267 | +``` |
| 268 | + |
| 269 | +### Calculate Process Memory |
| 270 | + |
| 271 | +**Average memory per process (in MB):** |
| 272 | +```bash |
| 273 | +ps --no-headers -o "rss,cmd" -C php-fpm | awk '{ sum+=$1} END { print sum/NR/1024 }' |
| 274 | +``` |
| 275 | + |
| 276 | +**Total memory usage by all PHP-FPM processes:** |
| 277 | +```bash |
| 278 | +ps --no-headers -o "rss,cmd" -C php-fpm | awk '{ sum+=$1 } END { print sum/1024 " MB" }' |
| 279 | +``` |
| 280 | + |
| 281 | +**Memory usage per individual process:** |
| 282 | +```bash |
| 283 | +ps -o pid,rss,cmd -C php-fpm | awk 'NR>1 {print $1, $2/1024 " MB", $3}' |
| 284 | +``` |
| 285 | + |
| 286 | +### Advanced Monitoring |
| 287 | + |
| 288 | +**Get CPU usage by PHP-FPM:** |
| 289 | +```bash |
| 290 | +ps -C php-fpm -o %cpu,pid,cmd --no-headers |
| 291 | +``` |
| 292 | + |
| 293 | +**Find the most memory-intensive PHP-FPM process:** |
| 294 | +```bash |
| 295 | +ps --no-headers -o "rss,pid,cmd" -C php-fpm | sort -rn | head -5 |
| 296 | +``` |
| 297 | + |
| 298 | +### Understanding the Output |
| 299 | + |
| 300 | +**RSS (Resident Set Size):** |
| 301 | +- Physical memory used by the process |
| 302 | +- Shown in KB by default |
| 303 | +- Divide by 1024 for MB |
| 304 | + |
| 305 | +**VSZ (Virtual Memory Size):** |
| 306 | +- Total virtual memory allocated |
| 307 | +- Usually much larger than RSS |
| 308 | +- Includes shared libraries |
| 309 | + |
| 310 | +**Example Output Interpretation:** |
| 311 | +```bash |
| 312 | +$ ps -ylC php-fpm --sort:rss |
| 313 | + RSS PID CMD |
| 314 | +52340 12345 php-fpm: master process |
| 315 | +48512 12346 php-fpm: pool www |
| 316 | +50240 12347 php-fpm: pool www |
| 317 | +``` |
| 318 | + |
| 319 | +This shows: |
| 320 | +- Master process using ~51 MB |
| 321 | +- Child processes using ~47-49 MB each |
| 322 | +- Total usage: ~150 MB for 3 processes |
| 323 | + |
| 324 | +### Composer Autoload Optimization |
| 325 | + |
| 326 | +For optimal performance, this project uses Composer autoload optimizations configured in `composer.json`: |
| 327 | + |
| 328 | +```json |
| 329 | +{ |
| 330 | + "config": { |
| 331 | + "optimize-autoloader": true, |
| 332 | + "classmap-authoritative": true |
| 333 | + } |
| 334 | +} |
| 335 | +``` |
| 336 | + |
| 337 | +**Performance Impact:** |
| 338 | + |
| 339 | +- `optimize-autoloader`: ~10-15% faster autoloading (converts PSR-0/PSR-4 to classmap) |
| 340 | +- `apcu-autoloader`: ~50-70% faster (requires APCu extension) |
| 341 | +- `classmap-authoritative`: Set to `false` for development, `true` for production only |
| 342 | + |
| 343 | +**Reference:** See the |
| 344 | +official [Symfony Performance Documentation](https://symfony.com/doc/current/performance.html#optimize-composer-autoloader) |
| 345 | +for detailed autoloader optimization guidelines and best practices. |
| 346 | + |
| 347 | +## Web Interfaces & Dashboards |
| 348 | + |
| 349 | +| Service | URL | Description | |
| 350 | +|-----------------------|-------------------------------|---------------------------------------| |
| 351 | +| FPM App | http://localhost:8088 | Main Symfony app (FPM) | |
| 352 | +| Franken | http://localhost:8080 | FrankenPHP (HTTP, regular mode) | |
| 353 | +| Franken Worker | http://localhost:8081 | FrankenPHP Worker (HTTP, optimized) | |
| 354 | +| Grafana | http://localhost:3000 | Metrics dashboard (admin/admin) | |
| 355 | +| Prometheus | http://localhost:9090 | Prometheus metrics | |
| 356 | +| Opcache Dashboard | http://localhost:42042 | PHP Opcache dashboard | |
| 357 | +| Opcache Metrics (FPM) | http://localhost:8088/metrics | PHP Opcache metrics via FPM app | |
| 358 | +| Franken Metrics | http://localhost:2019/metrics | Caddy/FrankenPHP metrics (non-worker) | |
| 359 | +| Worker Metrics | http://localhost:2020/metrics | Caddy/FrankenPHP metrics (worker) | |
| 360 | + |
| 361 | + |
| 362 | +## FrankenPHP Configuration |
| 363 | + |
| 364 | +This project uses FrankenPHP (a modern PHP runtime built on Caddy) with two different configurations for performance |
| 365 | +comparison and monitoring. |
| 366 | + |
| 367 | +**See [`frankenphp.md`](frankenphp.md) for complete FrankenPHP documentation including:** |
| 368 | + |
| 369 | +- Service configuration and differences |
| 370 | +- Auto-reload (file watching) setup |
| 371 | +- Caddy configuration and environment variables |
| 372 | +- Performance testing and monitoring |
| 373 | +- Troubleshooting guide |
| 374 | +- Resource optimization guidelines |
| 375 | + |
| 376 | +## Grafana Dashboard |
| 377 | + |
| 378 | +A detailed PHP-FPM and OPcache monitoring dashboard is available in Grafana. It includes: |
| 379 | + |
| 380 | +- PHP-FPM health, queue, and process metrics |
| 381 | +- Request rate, duration, and memory usage |
| 382 | +- OPcache hit ratio, memory, and script cache stats |
| 383 | +- JIT and interned strings monitoring |
| 384 | +- Alerts and color-coded panels for quick health checks |
| 385 | + |
| 386 | +**See [`grafana-dashboard.md`](grafana-dashboard.md) for a full description of all panels and dashboard features.** |
| 387 | + |
| 388 | + |
| 389 | +## Show symfony.prod.ini file |
0 commit comments