-
Notifications
You must be signed in to change notification settings - Fork 3.2k
fix(db): reduce overall number of db max conncetions to incr performance #1575
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Greptile Overview
Summary
This PR optimizes database connection pooling across multiple services to improve application performance by reducing overall database connection usage. The changes standardize connection pool configurations while decreasing maximum connection limits and adjusting timeout settings.The PR modifies three key database connection points: the main database pool in packages/db/index.ts (max connections reduced from 30 to 20), the socket server database operations in apps/sim/socket-server/database/operations.ts (max connections reduced from 15 to 5), and the socket server room manager in apps/sim/socket-server/rooms/manager.ts (timeout values standardized). The changes also remove verbose logging that was outputting pool configuration details, reducing log noise.
This optimization strategy addresses potential connection pool exhaustion and database server overload by forcing better connection reuse patterns. The timeout adjustments (shorter connect_timeout from 30s/20s to 10s, consistent idle_timeout of 20s) implement a "fail fast" approach for connection establishment while maintaining connections longer once established. These changes integrate well with the existing Drizzle ORM setup and PostgreSQL client configuration used throughout the application.
PR Description Notes:
- Title contains a typo: "conncetions" should be "connections"
- Title contains a typo: "incr" should likely be "increase" for clarity
Important Files Changed
Changed Files
| Filename | Score | Overview |
|---|---|---|
| packages/db/index.ts | 3/5 | Reduced max connections from 30 to 20 and connect_timeout from 30s to 10s, removed verbose logging |
| apps/sim/socket-server/database/operations.ts | 3/5 | Reduced max connections from 15 to 5, adjusted timeout values for consistency |
| apps/sim/socket-server/rooms/manager.ts | 4/5 | Standardized timeout settings to match global config while maintaining conservative 3 connection limit |
Confidence score: 3/5
- This PR introduces potential performance risks despite optimization goals due to aggressive connection limit reductions
- Score reflects concerns about connection bottlenecks under high load, particularly with socket server limited to 5 connections and shortened connect timeouts
- Pay close attention to all three database configuration files for potential connection failures in production environments
Sequence Diagram
sequenceDiagram
participant User
participant SocketServer as Socket Server
participant DatabaseOps as Database Operations
participant RoomManager as Room Manager
participant DB as PostgreSQL Database
User->>SocketServer: "Connect to workflow room"
SocketServer->>RoomManager: "Create/join workflow room"
RoomManager->>DB: "Validate workflow consistency (max: 3 connections)"
DB-->>RoomManager: "Return validation results"
RoomManager-->>SocketServer: "Room joined successfully"
User->>SocketServer: "Perform workflow operation"
SocketServer->>DatabaseOps: "Persist workflow operation"
DatabaseOps->>DB: "Begin transaction (max: 5 connections)"
DatabaseOps->>DB: "Update workflow timestamp"
DatabaseOps->>DB: "Handle block/edge/subflow operation"
DatabaseOps->>DB: "Update subflow node lists if needed"
DatabaseOps->>DB: "Commit transaction"
DB-->>DatabaseOps: "Operation completed"
DatabaseOps-->>SocketServer: "Operation persisted"
SocketServer->>RoomManager: "Broadcast operation to room"
RoomManager->>SocketServer: "Emit to all room participants"
SocketServer-->>User: "Operation synchronized"
Note over DatabaseOps: Connection pool reduced from 20 to 5 max connections
Note over RoomManager: Connection pool reduced to 3 max connections
Note over DB: Overall reduction from 20 to 8 total max connections
Additional Comments (1)
-
apps/sim/socket-server/rooms/manager.ts, line 11-20 (link)style: Consider consolidating database connections by importing the shared
dbinstance from@sim/dbinstead of creating a separate connection pool. This would reduce total connection count and simplify connection management.
3 files reviewed, 1 comment
Summary
reduce overall number of db max conncetions to incr performance
Type of Change
Testing
Tested manually.
Checklist