Skip to content

Commit 92e9623

Browse files
geroplgeropl
committed
[memory-bank] Document proxy, server and ws-proxy
Tool: gitpod/catfood.gitpod.cloud
1 parent 5f6123e commit 92e9623

File tree

5 files changed

+412
-1
lines changed

5 files changed

+412
-1
lines changed

‎memory-bank/activeContext.md‎

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -25,6 +25,9 @@ Key areas of focus include:
2525
- ide-service: Manages IDE configurations and resolves workspace IDE requirements
2626
- registry-facade: Modifies container images by adding layers for workspaces
2727
- image-builder-mk3: Builds custom workspace images from user-defined Dockerfiles
28+
- server: Main backend service handling API requests and user management
29+
- proxy: Main entry point for all HTTP and WebSocket traffic
30+
- ws-proxy: Handles routing and proxying of traffic to workspaces
2831

2932
As work progresses, this section will continue to be updated to reflect:
3033
- Additional component documentation

‎memory-bank/components/proxy.md‎

Lines changed: 123 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,123 @@
1+
# Proxy Component
2+
3+
## Overview
4+
5+
The Proxy is a critical component in Gitpod that serves as the main entry point for all HTTP and WebSocket traffic to the platform. It routes requests to the appropriate backend services, handles TLS termination, enforces security policies, and provides various routing and transformation capabilities for the Gitpod platform.
6+
7+
## Purpose
8+
9+
The primary purposes of the Proxy component are:
10+
- Act as the main ingress point for all Gitpod traffic
11+
- Route requests to appropriate backend services
12+
- Terminate TLS connections
13+
- Enforce security headers and policies
14+
- Handle workspace-specific routing
15+
- Provide WebSocket support
16+
- Implement cross-origin resource sharing (CORS) policies
17+
- Support custom domain routing
18+
- Provide health checks and metrics endpoints
19+
20+
## Architecture
21+
22+
The Proxy is built on Caddy, a powerful, extensible web server with automatic HTTPS capabilities. The Gitpod proxy extends Caddy with custom plugins to handle specific Gitpod requirements:
23+
24+
1. **Core Proxy**: Handles general routing and TLS termination
25+
2. **Workspace Handler**: Routes workspace-specific requests
26+
3. **Custom Plugins**: Extend Caddy with Gitpod-specific functionality
27+
4. **Security Layer**: Enforces security headers and policies
28+
5. **Metrics Endpoint**: Provides monitoring capabilities
29+
30+
## Key Files and Structure
31+
32+
- `Dockerfile`: Builds the proxy container with Caddy and custom plugins
33+
- `conf/Caddyfile`: Main configuration file for the proxy
34+
- `conf/workspace-handler.full`: Configuration for handling workspace requests
35+
- `conf/workspace-handler.meta`: Configuration for handling workspace metadata
36+
- `plugins/`: Custom Caddy plugins for Gitpod-specific functionality
37+
38+
## Custom Plugins
39+
40+
The proxy includes several custom Caddy plugins to extend its functionality:
41+
42+
1. **corsorigin**: Handles Cross-Origin Resource Sharing (CORS) policies
43+
2. **secwebsocketkey**: Validates WebSocket connections
44+
3. **workspacedownload**: Manages workspace content downloads
45+
4. **headlesslogdownload**: Handles headless log downloads
46+
5. **configcat**: Integrates with ConfigCat feature flags
47+
6. **analytics**: Provides analytics functionality
48+
7. **logif**: Conditional logging
49+
8. **jsonselect**: JSON selection for logs
50+
9. **sshtunnel**: SSH tunneling support
51+
10. **frontend_dev**: Development mode for frontend
52+
53+
## Configuration
54+
55+
The proxy is configured via the Caddyfile, which includes:
56+
57+
### Main Domain Configuration
58+
- TLS settings
59+
- Security headers
60+
- Routing rules for the main Gitpod domain
61+
- API endpoints
62+
- Backend service routing
63+
64+
### Workspace Domain Configuration
65+
- Routing for workspace-specific domains
66+
- Port forwarding
67+
- WebSocket handling
68+
- IDE-specific routing
69+
70+
### Security Configuration
71+
- HTTP to HTTPS redirection
72+
- Security headers
73+
- CORS policies
74+
- WebSocket validation
75+
76+
## Routing Logic
77+
78+
The proxy implements sophisticated routing logic:
79+
80+
1. **Main Domain Routing**: Routes requests to the main Gitpod domain to appropriate backend services
81+
2. **Workspace Routing**: Routes workspace requests based on subdomain patterns
82+
3. **API Routing**: Routes API requests to the server component
83+
4. **Public API Routing**: Routes public API requests to the public-api-server
84+
5. **Static Content**: Routes static content requests to appropriate services
85+
6. **WebSocket Routing**: Handles WebSocket connections for real-time communication
86+
87+
## Workspace Routing
88+
89+
Workspace routing is particularly complex, handling several patterns:
90+
91+
1. **Standard Workspace**: `<workspace-id>.ws.<region>.<domain>`
92+
2. **Port Forwarding**: `<port>-<workspace-id>.ws.<region>.<domain>`
93+
3. **Debug Workspace**: `debug-<workspace-id>.ws.<region>.<domain>`
94+
4. **Foreign Content**: Special routes for VS Code webviews and webworkers
95+
96+
## Security Considerations
97+
98+
The proxy implements several security measures:
99+
100+
- TLS termination with secure configuration
101+
- HTTP Strict Transport Security (HSTS)
102+
- Content Security Policy (CSP)
103+
- Cross-Origin Resource Sharing (CORS) policies
104+
- XSS protection
105+
- Referrer policy
106+
- WebSocket validation
107+
108+
## Common Usage Patterns
109+
110+
The Proxy is typically used to:
111+
1. Route client requests to appropriate backend services
112+
2. Provide secure access to workspaces
113+
3. Handle WebSocket connections for real-time communication
114+
4. Enforce security policies
115+
5. Provide health checks and metrics
116+
117+
## Related Components
118+
119+
- **Server**: Receives API requests routed through the proxy
120+
- **Dashboard**: Serves the web UI through the proxy
121+
- **WS Proxy**: Handles workspace-specific traffic
122+
- **IDE Proxy**: Manages IDE-specific routing
123+
- **Public API Server**: Provides public API endpoints

‎memory-bank/components/server.md‎

Lines changed: 136 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,136 @@
1+
# Server Component
2+
3+
## Overview
4+
5+
The Server is a central component in Gitpod that serves as the main backend service, handling API requests, authentication, user management, workspace operations, and integration with various source code management systems. It acts as the core orchestrator for the Gitpod platform, connecting various components and providing a unified API for clients.
6+
7+
## Purpose
8+
9+
The primary purposes of the Server component are:
10+
- Provide API endpoints for client applications (dashboard, IDE, CLI)
11+
- Handle user authentication and session management
12+
- Manage user accounts and preferences
13+
- Coordinate workspace creation and management
14+
- Integrate with source code management systems (GitHub, GitLab, Bitbucket)
15+
- Process webhooks for prebuilds and other automated operations
16+
- Manage billing and subscription information
17+
- Provide real-time communication via WebSockets
18+
- Coordinate with other Gitpod components
19+
20+
## Architecture
21+
22+
The Server operates as an Express.js application with several key components:
23+
24+
1. **API Server**: Provides HTTP and WebSocket endpoints for client communication
25+
2. **Authentication System**: Handles user authentication and session management
26+
3. **Database Interface**: Interacts with the database for persistent storage
27+
4. **WebSocket Manager**: Manages real-time communication with clients
28+
5. **SCM Integrations**: Connects with GitHub, GitLab, Bitbucket, and other platforms
29+
6. **Workspace Coordinator**: Manages workspace lifecycle in coordination with ws-manager
30+
7. **Monitoring Endpoints**: Provides health checks and metrics
31+
32+
The server is designed as a modular application using dependency injection (Inversify) to manage components and their dependencies.
33+
34+
## Key Files and Structure
35+
36+
- `main.ts`: Entry point that initializes the container and starts the server
37+
- `init.ts`: Handles server initialization and setup
38+
- `server.ts`: Core server implementation
39+
- `src/api/`: API endpoints and handlers
40+
- `src/auth/`: Authentication and authorization
41+
- `src/workspace/`: Workspace management
42+
- `src/user/`: User management
43+
- `src/prebuilds/`: Prebuild functionality
44+
- `src/billing/`: Billing and subscription management
45+
- `src/github/`, `src/gitlab/`, `src/bitbucket/`: SCM integrations
46+
47+
## Dependencies
48+
49+
### Internal Dependencies
50+
- `components/gitpod-db`: Database access layer
51+
- `components/gitpod-protocol`: Shared protocol definitions
52+
- `components/content-service-api`: Content service API definitions
53+
- `components/ws-manager-api`: Workspace manager API definitions
54+
- `components/image-builder-api`: Image builder API definitions
55+
- Various other Gitpod component APIs
56+
57+
### External Dependencies
58+
- Express.js for HTTP server
59+
- WebSocket for real-time communication
60+
- Inversify for dependency injection
61+
- TypeORM for database access
62+
- Redis for caching and pub/sub
63+
- Prometheus for metrics
64+
- Various SCM platform SDKs
65+
66+
## Configuration
67+
68+
The Server is configured via environment variables and configuration files, including:
69+
70+
- Server address and port
71+
- Database connection details
72+
- Authentication providers
73+
- SCM integration settings
74+
- Feature flags
75+
- Monitoring and logging settings
76+
77+
## API Services
78+
79+
The Server exposes multiple API endpoints:
80+
81+
1. **User API**: User management, authentication, and preferences
82+
2. **Workspace API**: Workspace creation, management, and access
83+
3. **SCM Integration APIs**: GitHub, GitLab, Bitbucket webhooks and OAuth
84+
4. **Billing API**: Subscription and payment management
85+
5. **WebSocket API**: Real-time communication with clients
86+
6. **Health and Metrics API**: System health and monitoring
87+
88+
## Authentication and Authorization
89+
90+
The Server supports multiple authentication methods:
91+
92+
1. **Session-based Authentication**: For web clients
93+
2. **Bearer Token Authentication**: For API access
94+
3. **OAuth Integration**: With GitHub, GitLab, Bitbucket, etc.
95+
4. **Personal Access Tokens**: For programmatic access
96+
97+
Authorization is handled through a combination of user roles, permissions, and access controls.
98+
99+
## Integration Points
100+
101+
The Server integrates with:
102+
1. **Database**: For persistent storage
103+
2. **Redis**: For caching and pub/sub messaging
104+
3. **Workspace Manager**: For workspace lifecycle management
105+
4. **Image Builder**: For custom workspace images
106+
5. **Content Service**: For workspace content management
107+
6. **SCM Platforms**: For repository access and webhooks
108+
7. **Payment Providers**: For billing and subscriptions
109+
110+
## Security Considerations
111+
112+
- Implements CSRF protection for WebSocket connections
113+
- Handles authentication and session management securely
114+
- Validates and sanitizes user input
115+
- Implements proper error handling and logging
116+
- Uses HTTPS for secure communication
117+
- Manages sensitive data securely
118+
119+
## Common Usage Patterns
120+
121+
The Server is typically used to:
122+
1. Handle API requests from the dashboard and IDE
123+
2. Process authentication and session management
124+
3. Coordinate workspace creation and management
125+
4. Handle webhooks from SCM platforms
126+
5. Manage user accounts and preferences
127+
6. Process billing and subscription information
128+
129+
## Related Components
130+
131+
- **Dashboard**: Frontend interface that communicates with the server
132+
- **Workspace Manager**: Manages workspace instances
133+
- **Content Service**: Manages workspace content
134+
- **Image Builder**: Builds custom workspace images
135+
- **Database**: Stores persistent data
136+
- **IDE Service**: Provides IDE configuration

0 commit comments

Comments
 (0)