Add localhost endpoints and add jobs to support local games and services

Add http localhost endpoints and add job to print hosts

Add ability to proxy local services into the cluster

Add job to update faf featured mod files with the existing files to launch games

Fix faf_data_dir being None

Add reloader auto to each pod

Allow alternative name for ergochat server

Undo ergochat servername change

Fix tiltfile

Support proxying services with no ingress

Add default data directory for linux

Make base domain configurable for local setup with tilt

Author: Sheikah45

README.MD

@@ -69,15 +69,21 @@ This repository aims to provide a ready-to-go [Tilt](https://docs.tilt.dev/) set
 ## Prerequisites
 * [Tilt](https://docs.tilt.dev/install.html)
 * Kubernetes ([Docker Desktop](https://docs.docker.com/desktop/kubernetes/) or [Minikube](https://minikube.sigs.k8s.io/docs/) are recommended for those new to kubernetes) [Tilt cluster setup](https://docs.tilt.dev/choosing_clusters)
-* For Windows users a bash program. By default git bash is used with an assumed installation directory of C:/Program Files/Git
 * [helm](https://helm.sh/docs/intro/install/) must be installed to generate the k8s from the helm charts in the gitops repo
+* For Windows users:
+    * A bash program. By default git bash is used with an assumed installation directory of C:/Program Files/Git
 
 ## Startup Services
 In the root directory of the repository run `tilt up`. This will start all the faf services in the correct order. The status of each service can be viewed in the tilt UI by visiting <http://localhost:10350>. This is the control server for tilt where you can restart services or disable them for substitution by services you would like to run from source code as you actively develop them.
 
 ## Development
-To develop against the FAF infrastructure you should disable the service in tilt that you are actively developing. Once disabled you can start up your developed version. Some tweaks may need to be made to the default configuration parameters in the source code. The proper values can be found in the configMaps in each of the services kubernetes deploy yaml files.
+To develop against the FAF infrastructure you can insert the service in tilt that you are actively developing by adding it to the local_services list in your tilt_config.json file or as --local_services when you run tilt up. This will proxy the connections in the cluster to that service to your locally running instance. Some tweaks may need to be made to the default configuration parameters in the source code. The proper values can be found in the configMaps in each of the services kubernetes deploy yaml files. You may need to specify your host-ip in the config or arguments in order to get the proxied connections to work
 
 ## Test Data
-The default test data that is loaded can be found in [faf-db](https://github.com/FAForever/db/blob/develop/test-data.sql)
+The default test data that is loaded can be found in [faf-db](https://github.com/FAForever/db/blob/develop/test-data.sql).
+You can load the data at anytime by manually triggering the popualte-db resource under the Database label. This needs to be done the first time for data to exist.
+
+## Endpoints
+Locally traefik uses the same ingress routes as we do in production just with a default base domain of faforever.localhost. So accessible domains will be api.faforever.localhost, hydra.faforever.localhost etc. By default these should resolve to 127.0.0.1. If they do not you can manually modify your hosts file to add the local dns records. For convenience the resource create-hosts-file-content
+will output the necessary lines for the hosts file when run after every service has started.
 

Tiltfile

@@ -9,7 +9,7 @@ stringData:
   OAUTH_SECRET: "banana"
   "config.json": |
     {
-      "url": "https://forum.localhost",
+      "url": "https://forum.{{ .Values.baseDomain }}",
       "secret": "banana",
       "database": "mongo",
       "port": 4567,

apps/nodebb/templates/local-secret.yaml

@@ -2,13 +2,20 @@ traefik:
   deployment:
     enabled: true
 
+  ingressRoute:
+    dashboard:
+      enabled: true
+      matchRule: Host(`traefik.faforever.localhost`)
+      entryPoints: ["websecure", "web"]
+
   additionalArguments:
     - "--entrypoints.websecure.http.encodequerysemicolons=true"
+    - "--entrypoints.web.http.encodequerysemicolons=true"
 
   updateStrategy:
     type: Recreate
 
-  hostNetwork: true
+  hostNetwork: false
 
   ports:
     metrics:
@@ -17,11 +24,9 @@ traefik:
       port: 9101
 
     web:
-      redirections:
-        entryPoint:
-          to: websecure
-          scheme: https
-          permanent: true
+      transport:
+        respondingTimeouts:
+          readTimeout: 60
 
     websecure:
       transport:
@@ -33,7 +38,7 @@ traefik:
 
   # HostNetwork
   service:
-    enabled: false
+    enabled: true
 
   logs:
     general:
@@ -43,3 +48,9 @@ traefik:
       # By default, the level is set to ERROR.
       # -- Alternative logging levels are DEBUG, PANIC, FATAL, ERROR, WARN, and INFO.
       level: INFO
+    access:
+      enabled: true
+  
+  providers:
+    kubernetesCRD:
+      allowExternalNameServices: true

cluster/traefik/values-local.yaml

@@ -1,6 +1,6 @@
 hostName: "localhost"
 environment: "local"
-baseDomain: "localhost"
+baseDomain: "faforever.localhost"
 infisical-secret:
   enabled: false
 traefik: {}

config/local.yaml

@@ -0,0 +1,3 @@
+apiVersion: v2
+name: host-proxy
+version: 1.0.0

tilt/helm/host-proxy/Chart.yaml

@@ -0,0 +1,11 @@
+{{- if .Values.hostIP }}
+apiVersion: v1
+kind: Endpoints
+metadata:
+  name: {{.Values.name}}
+subsets:
+  - addresses:
+      - ip: {{.Values.hostIP}}
+    ports:
+      - port: {{.Values.port}}
+{{- end }}

tilt/helm/host-proxy/templates/endpoint.yaml

@@ -0,0 +1,15 @@
+apiVersion: v1
+kind: Service
+metadata:
+  name: {{.Values.name}}
+spec:
+{{- if .Values.hostIP }}
+  clusterIP: None
+  ports:
+    - protocol: "TCP"
+      port: {{.Values.port}}
+      targetPort: {{.Values.port}}
+{{ else }}
+  type: ExternalName
+  externalName: {{.Values.hostname}}
+{{- end }}

tilt/helm/host-proxy/templates/service.yaml

@@ -1,10 +1,9 @@
 #!/bin/bash
 
-cache_dir=$1
+chart_cache_dir=$1
 shift
 
 chart=$1
-chart_cache_dir="$cache_dir/$chart"
 
 mkdir -p "$chart_cache_dir"
 

tilt/scripts/helm-with-cache.sh

@@ -0,0 +1 @@
+kubectl get IngressRoute -A -o yaml | grep -Po 'Host\(`(.*?)`\)' | sed 's/^.\{6\}\(.*\).\{2\}$/127.0.0.1 \1/' | uniq