fix: updated readme and url maps describing how to redirect http to https (#184)

Author: bosbaber

.github/copilot-instructions.md

@@ -25,13 +25,24 @@ This is the Interledger Foundation website, a Drupal 10 CMS deployed on Google C
 
 ### Load Balancer Routing
 
-The URL map (`interledger-org`) routes traffic:
+**HTTPS Traffic (Port 443):**
+
+The URL map (`interledger-org`) routes HTTPS traffic:
 1. `interledger.org` → `drupal-sites-backend` (VM) for all paths except:
    - `/developers` → `nginx-rewrite-backend` (Cloud Run)
 2. `staging.interledger.org` → `drupal-sites-backend` (VM) for all paths except:
    - `/developers` → `nginx-rewrite-backend` (Cloud Run)
 3. `uwa.interledger.org` → `umami-analytics-backend` (Cloud Run)
 
+**HTTP Traffic (Port 80):**
+
+The HTTP redirect URL map (`interledger-org-http-redirect`) handles all HTTP traffic:
+- All HTTP requests receive a 301 permanent redirect to HTTPS
+- Preserves the original hostname and path
+- Configuration stored in `ci/deploy/http-redirect-urlmap.yaml`
+
+**Key Point**: We maintain TWO separate URL maps - one for HTTPS routing (the main `urlmap.yaml`) and one for HTTP→HTTPS redirect (`http-redirect-urlmap.yaml`). When adding new domains, update BOTH files.
+
 ### Database Credentials
 
 **Never hardcode database credentials in settings.php!**
@@ -60,6 +71,8 @@ $databases['default']['default'] = [
   - **`ci/Makefile`**: Commands for deploy, backup, restore operations
   - **`ci/backupmanager/`**: Go application for backup/restore with Cloud SQL
   - **`ci/deploy/`**: Environment-specific configs and load balancer configuration
+    - **`urlmap.yaml`**: HTTPS URL map configuration (port 443 routing)
+    - **`http-redirect-urlmap.yaml`**: HTTP redirect URL map (port 80 → HTTPS redirect)
 - **`web/`**: Drupal docroot
 - **`local/`**: Local development with Docker Compose
 - **`.github/workflows/`**: GitHub Actions workflows
@@ -183,25 +196,42 @@ sudo chmod -R 775 /var/www/production/web/sites/default/files/
 
 ## URL Map Management
 
-The URL map configuration is stored at `ci/deploy/urlmap.yaml` with read-only fields removed.
+We maintain two URL map configurations:
 
-**To modify routing:**
+**HTTPS URL Map** (`interledger-org`):
+- Stored at `ci/deploy/urlmap.yaml` with read-only fields removed
+- Routes HTTPS traffic to appropriate backends
+
+**HTTP Redirect URL Map** (`interledger-org-http-redirect`):
+- Stored at `ci/deploy/http-redirect-urlmap.yaml`
+- Redirects all HTTP traffic to HTTPS with 301 permanent redirects
+
+**To modify HTTPS routing:**
 1. Edit `ci/deploy/urlmap.yaml`
 2. Import: `gcloud compute url-maps import interledger-org --source=ci/deploy/urlmap.yaml --quiet`
 3. Verify: `gcloud compute url-maps describe interledger-org`
 
-**To update local file:**
+**To modify HTTP redirect (when adding new domains):**
+1. Edit `ci/deploy/http-redirect-urlmap.yaml`
+2. Import: `gcloud compute url-maps import interledger-org-http-redirect --source=ci/deploy/http-redirect-urlmap.yaml --quiet`
+3. Verify: `gcloud compute url-maps describe interledger-org-http-redirect`
+4. Test: `curl -I http://newdomain.com`
+
+**To update local files:**
 ```bash
 gcloud compute url-maps export interledger-org --destination=ci/deploy/urlmap.yaml
+gcloud compute url-maps export interledger-org-http-redirect --destination=ci/deploy/http-redirect-urlmap.yaml
 # Remove read-only fields: creationTimestamp, fingerprint, id, kind, selfLink
 ```
 
 ## Common Pitfalls
 
-1. **Certificate Map Entries**: When adding a new domain, remember to create both:
-   - The certificate: `gcloud certificate-manager certificates create ...`
-   - The certificate map entry: `gcloud certificate-manager maps entries create ...`
-   - Map entries take 5-10 minutes to become ACTIVE
+1. **HTTP and HTTPS for New Domains**: When adding a new domain, configure BOTH protocols:
+   - **HTTPS**: Create certificate, certificate map entry, and add to `ci/deploy/urlmap.yaml`
+   - **HTTP**: Add domain to `ci/deploy/http-redirect-urlmap.yaml` and import the updated config
+   - Forgetting HTTP redirect means `http://` requests will fail instead of redirecting to `https://`
+   - Both URL maps share the same IP address (34.111.215.251) but use different forwarding rules and target proxies
+   - Certificate map entries take 5-10 minutes to become ACTIVE
 
 2. **Database Name in SQL Dumps**: Cloud SQL exports include the source database name. The backupmanager automatically replaces this during restore, but if you manually restore, you must replace all occurrences of the source database name with the target database name.
 

ci/deploy/README.md

@@ -102,6 +102,35 @@ Client (Browser)
     └─ umami-analytics-backend → Cloud Run NEG
 ```
 
+### HTTP to HTTPS Redirect
+
+All HTTP (port 80) traffic is automatically redirected to HTTPS (port 443) using a separate forwarding rule and URL map:
+
+```
+Client (Browser) - HTTP Request
+    ↓
+[1] HTTP Forwarding Rule (34.111.215.251:80)
+    ↓ "Incoming HTTP traffic"
+    ↓
+[2] Target HTTP Proxy (interledger-org-http-proxy)
+    ↓
+[3] HTTP Redirect URL Map (interledger-org-http-redirect)
+    ↓ "301 Permanent Redirect to HTTPS"
+    ↓
+Client follows redirect to HTTPS
+```
+
+**Components:**
+- **Forwarding Rule**: `interledger-org-http` (port 80, same IP as HTTPS)
+- **Target HTTP Proxy**: `interledger-org-http-proxy`
+- **URL Map**: `interledger-org-http-redirect` (configuration stored in `ci/deploy/http-redirect-urlmap.yaml`)
+- **Reserved IP**: `interledger-org-ip` (34.111.215.251)
+
+**Configuration File**: `ci/deploy/http-redirect-urlmap.yaml` defines the redirect behavior:
+- All HTTP requests receive a 301 permanent redirect to HTTPS
+- Host rules ensure redirects work for all domains (production, staging, www)
+- Preserves the original path and query parameters
+
 **Key architectural decisions:**
 
 - **Single Backend for Both Environments**: Both production and staging use `drupal-sites-backend` because a VM instance can only belong to one load-balanced instance group
@@ -198,9 +227,17 @@ gcloud compute operations list \
 
 ### Update URL Map
 
-The URL Map defines all routing rules. The current configuration is stored in `ci/deploy/urlmap.yaml`.
+The URL Map defines all routing rules. We maintain two URL maps:
+
+1. **HTTPS URL Map** (`interledger-org`): Routes HTTPS traffic to backends
+   - Configuration: `ci/deploy/urlmap.yaml`
+   - Handles path-based routing to different backends
 
-#### Make Changes to URL Map
+2. **HTTP Redirect URL Map** (`interledger-org-http-redirect`): Redirects HTTP to HTTPS
+   - Configuration: `ci/deploy/http-redirect-urlmap.yaml`
+   - Simple redirect for all HTTP traffic
+
+#### Make Changes to HTTPS URL Map
 
 1. **Edit the local file**: Modify `ci/deploy/urlmap.yaml` as needed
    - Add/remove hostRules for new domains
@@ -223,10 +260,26 @@ The URL Map defines all routing rules. The current configuration is stored in `c
    # Clean up read-only fields manually (creationTimestamp, fingerprint, id, kind, selfLink)
    ```
 
+#### Make Changes to HTTP Redirect URL Map
+
+1. **Edit the local file**: Modify `ci/deploy/http-redirect-urlmap.yaml` as needed
+   - Add/remove hostRules for new domains that need HTTP→HTTPS redirect
+
+2. **Import the updated configuration**:
+   ```bash
+   gcloud compute url-maps import interledger-org-http-redirect --source=ci/deploy/http-redirect-urlmap.yaml --quiet
+   ```
+
+3. **Verify changes**:
+   ```bash
+   gcloud compute url-maps describe interledger-org-http-redirect
+   ```
+
 **Important Notes**:
 - Route rules are evaluated by priority (lower numbers first)
-- The local `urlmap.yaml` file has read-only fields removed for easy editing
+- The local `urlmap.yaml` files have read-only fields removed for easy editing
 - Always test changes before applying to production traffic
+- **When adding new domains, update BOTH URL maps** - one for HTTPS routing, one for HTTP redirect
 
 ### Add a New Cloud Run Service
 
@@ -454,10 +507,17 @@ Current backend services in use:
 
 ## Files in This Directory
 
-- **`urlmap.yaml`**: Current URL map configuration
+- **`urlmap.yaml`**: HTTPS URL map configuration (port 443 routing)
+  - Routes HTTPS traffic to appropriate backends
   - Export latest: `gcloud compute url-maps export interledger-org --destination=ci/deploy/urlmap.yaml`
   - Import changes: `gcloud compute url-maps import interledger-org --source=ci/deploy/urlmap.yaml --quiet`
   - Read-only fields removed (creationTimestamp, fingerprint, id, kind, selfLink)
+
+- **`http-redirect-urlmap.yaml`**: HTTP redirect URL map configuration (port 80 → HTTPS redirect)
+  - Redirects all HTTP traffic to HTTPS with 301 permanent redirects
+  - Export latest: `gcloud compute url-maps export interledger-org-http-redirect --destination=ci/deploy/http-redirect-urlmap.yaml`
+  - Import changes: `gcloud compute url-maps import interledger-org-http-redirect --source=ci/deploy/http-redirect-urlmap.yaml --quiet`
+  - **Remember**: When adding new domains, update BOTH this file and `urlmap.yaml`
 
 - **`staging/`**: Staging environment configuration files
   - `settings.php`, `htaccess`, `robots.txt`, `cleanup.sh`

ci/deploy/http-redirect-urlmap.yaml

@@ -0,0 +1,15 @@
+name: interledger-org-http-redirect
+defaultUrlRedirect:
+  httpsRedirect: true
+  redirectResponseCode: MOVED_PERMANENTLY_DEFAULT
+hostRules:
+- hosts:
+  - interledger.org
+  - www.interledger.org
+  - staging.interledger.org
+  pathMatcher: redirect-matcher
+pathMatchers:
+- name: redirect-matcher
+  defaultUrlRedirect:
+    httpsRedirect: true
+    redirectResponseCode: MOVED_PERMANENTLY_DEFAULT