Skip to content

Commit 990af1d

Browse files
SeeuSimSeeuSim
committed
PEER-242: Add working gcloud public internet deployment
Signed-off-by: SeeuSim <[email protected]>
1 parent 2f0ae7f commit 990af1d

File tree

9 files changed

+194
-17
lines changed

9 files changed

+194
-17
lines changed

.github/workflows/build-docker.yaml renamed to .github/workflows/build-deploy-docker.yaml

Lines changed: 25 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -105,7 +105,7 @@ jobs:
105105
echo "Outputs Generated: $formatted_matrix"
106106
echo "matrix=$formatted_matrix" >> $GITHUB_OUTPUT
107107
108-
build-and-push-image:
108+
build-push-deploy-image:
109109
needs: changes
110110
if: ${{ fromJson(needs.changes.outputs.matrix)[0] != null }}
111111
runs-on: ubuntu-latest
@@ -186,6 +186,30 @@ jobs:
186186
cache-from: type=gha
187187
cache-to: type=gha,mode=max
188188

189+
- name: Setup GCloud
190+
uses: google-github-actions/setup-gcloud@v
191+
if: ${{ contains(github.ref, 'main') && github.event.pull_request.title != 'Feedback' && false }}
192+
with:
193+
service_account_key: ${{ secrets.GKE_SA_KEY }}
194+
project_id: ${{ secrets.GKE_PROJECT }}
195+
196+
- name: Get GKE creds
197+
uses: google-github-actions/get-gke-credentials@v
198+
if: ${{ contains(github.ref, 'main') && github.event.pull_request.title != 'Feedback' && false }}
199+
with:
200+
cluster_name: ${{ env.GKE_CLUSTER }}
201+
location: ${{ env.GKE_ZONE }}
202+
credentials: ${{ secrets.GKE_SA_KEY }}
203+
204+
- name: Deploy to GKE
205+
if: ${{ contains(github.ref, 'main') && github.event.pull_request.title != 'Feedback' && false }}
206+
run: |-
207+
deployments=("collab-service" "matching-service" "question-service" "user-service" "frontend")
208+
for dplymnt in "${deployments[@]}"; do
209+
kubectl -n peerprep rollout restart deployment "$dplymnt"
210+
done
211+
212+
189213
results:
190214
if: ${{ always() && !github.event.pull_request.draft }}
191215
runs-on: ubuntu-latest

‎k8s/README.md‎

Lines changed: 112 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -90,7 +90,7 @@
9090
2. Run the command to set up the ingress controller:
9191

9292
```sh
93-
kubectl apply -f ./k8s/ingress/nginx-ingress.yaml
93+
kubectl apply -f ./k8s/local
9494
```
9595

9696
It should take a couple of minutes. Once done, you should run this command:
@@ -148,6 +148,116 @@
148148
149149
## GKE Instructions
150150
151-
To be added.
151+
1. Authenticate or ensure you are added as a user to the Google Cloud Project:
152152
153+
- Project ID: `cs3219-g16`
154+
- Project Zone: `asia-southeast1-c`
155+
156+
2. Install the `gcloud` C by following the instructions at this link:
157+
158+
- [Installation Instructions](https://cloud.google.com/sdk/docs/install)
159+
160+
3. Setup the CLI with the following commands:
161+
162+
```sh
163+
gcloud auth login
164+
165+
gcloud config set project cs3219-g16
166+
167+
gcloud config set compute/zone asia-southeast1-c
168+
169+
gcloud components install gke-gcloud-auth-plugin
170+
171+
export USE_GKE_GCLOUD_AUTH_PLUGIN=True
172+
```
173+
174+
4. Create the cluster with the following commands:
175+
176+
```sh
177+
gcloud container clusters create \
178+
cs3219-g16 \
179+
--preemptible \
180+
--machine-type e2-small \
181+
--enable-autoscaling \
182+
--num-nodes 1 \
183+
--min-nodes 1 \
184+
--max-nodes 25 \
185+
--region=asia-southeast1-c
186+
```
187+
188+
5. Once the cluster has been created, run the commands below to configure `kubectl` and connect to the cluster:
189+
190+
```sh
191+
gcloud container clusters get-credentials cs3219-g16
192+
193+
# You should see some output here
194+
kubectl get nodes -o wide
195+
```
196+
197+
6. Run the script (ensure you are in a Bash shell like on Mac or Linux):
198+
199+
```sh
200+
make k8s-up
201+
```
202+
203+
- Wait until the deployments all reach status running:
204+
205+
```sh
206+
kubectl -n peerprep rollout status deployment frontend
207+
```
208+
209+
7. If you haven't already, visit the GCloud console -> 'Cloud Domains' and verify that a domain name has been created.
210+
211+
- We currently have one as `peerprep-g16.net`.
212+
- We also associate a GCloud Global Web IP `web-ip` to this DNS record as an 'A' record.
213+
214+
8. Install the `cert-manager` plugin:
215+
216+
```sh
217+
kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.16.1/cert-manager.yaml
218+
```
219+
220+
9. Create the ingress and secrets in the prod environment:
221+
222+
```sh
223+
kubectl apply -f ./k8s/gcloud
224+
```
225+
226+
- After 15 minutes, you should be able to access the UI over HTTPS at this link:
227+
- `https://peerprep-g16.net`
228+
229+
10. Setup the following in Github Actions by:
230+
231+
- heading to the 'Settings' -> 'Secrets and variables' -> 'Actions' -> 'New repository secret'
232+
- Adding the following keys:
233+
234+
```txt
235+
GKE_SA_KEY: <redacted (get from the cloud console page)>
236+
GKE_PROJECT: cs3219-g16
237+
GKE_CLUSTER: cs3219-g16
238+
GKE_ZONE: asia-southeast1-c
239+
240+
```
241+
242+
11. Merge a PR to `main` and verify that the cluster is redeployed with the latest images:
243+
244+
```sh
245+
kubectl -n peerprep get deployment
246+
```
247+
248+
12. Cleanup:
249+
250+
- Delete the cluster:
251+
252+
```sh
253+
gcloud container clusters delete cs3219-g16
254+
```
255+
256+
- When done with the project, delete the web records:
257+
258+
```sh
259+
gcloud dns record-sets delete peerprep-g16 --type A
260+
261+
gcloud compute addresses delete web-ip --global
262+
```
153263
<!-- https://cert-manager.io/docs/tutorials/getting-started-with-cert-manager-on-google-kubernetes-engine-using-lets-encrypt-for-ingress-ssl/ -->

k8s/gcloud/02-issuer-le-staging.yaml renamed to k8s/gcloud-staging/01-issuer-le-staging.yaml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,10 +2,14 @@ apiVersion: cert-manager.io/v1
22
kind: Issuer
33
metadata:
44
name: letsencrypt-staging
5+
namespace: peerprep
6+
labels:
7+
project: peerprep
8+
peerprep.service: app-cert-issuer-staging
59
spec:
610
acme:
711
server: https://acme-staging-v02.api.letsencrypt.org/directory
8-
email: <email-address> # ❗ Replace this with your email address
12+
email: [email protected] # ❗ Replace this with your email address
913
privateKeySecretRef:
1014
name: letsencrypt-staging
1115
solvers:

k8s/gcloud/01-web-ssl-secret.yaml renamed to k8s/gcloud-staging/02-web-ssl-secret.yaml

File renamed without changes.

‎k8s/gcloud-staging/03-ingress.yaml‎

Lines changed: 27 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,27 @@
1+
# ingress.yaml
2+
apiVersion: networking.k8s.io/v1
3+
kind: Ingress
4+
metadata:
5+
name: peerprep-ingress
6+
namespace: peerprep
7+
labels:
8+
project: peerprep
9+
peerprep.service: app-ingress
10+
annotations:
11+
# This tells Google Cloud to create an External Load Balancer to realize this Ingress
12+
kubernetes.io/ingress.class: gce
13+
# This enables HTTP connections from Internet clients
14+
kubernetes.io/ingress.allow-http: "true"
15+
# This tells Google Cloud to associate the External Load Balancer with the static IP which we created earlier
16+
kubernetes.io/ingress.global-static-ip-name: web-ip
17+
cert-manager.io/issuer: letsencrypt-staging
18+
spec:
19+
tls:
20+
- secretName: web-ssl
21+
hosts:
22+
- peerprep-g16.net
23+
defaultBackend:
24+
service:
25+
name: frontend
26+
port:
27+
number: 3000

k8s/gcloud/02-issuer-le-prod.yaml renamed to k8s/gcloud/01-issuer-le-prod.yaml

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -3,10 +3,14 @@ apiVersion: cert-manager.io/v1
33
kind: Issuer
44
metadata:
55
name: letsencrypt-production
6+
namespace: peerprep
7+
labels:
8+
project: peerprep
9+
peerprep.service: app-cert-issuer-prod
610
spec:
711
acme:
812
server: https://acme-v02.api.letsencrypt.org/directory
9-
email: <email-address> # ❗ Replace this with your email address
13+
email: [email protected] # ❗ Replace this with your email address
1014
privateKeySecretRef:
1115
name: letsencrypt-production
1216
solvers:

‎k8s/gcloud/02-web-ssl-secret.yaml‎

Lines changed: 13 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,13 @@
1+
# Placeholder Secret to store TLS keys
2+
3+
apiVersion: v1
4+
kind: Secret
5+
metadata:
6+
name: web-ssl
7+
namespace: peerprep
8+
labels:
9+
project: peerprep
10+
type: kubernetes.io/tls
11+
stringData:
12+
tls.key: ""
13+
tls.crt: ""

‎k8s/gcloud/03-ingress.yaml‎

Lines changed: 7 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -19,15 +19,10 @@ spec:
1919
tls:
2020
- secretName: web-ssl
2121
hosts:
22-
- $DOMAIN_NAME
23-
rules:
24-
- host: peerprep-g16.net
25-
http:
26-
paths:
27-
- path: /
28-
pathType: Prefix
29-
backend:
30-
service:
31-
name: frontend
32-
port:
33-
number: 3000
22+
- peerprep-g16.net
23+
defaultBackend:
24+
service:
25+
name: frontend
26+
port:
27+
number: 3000
28+

k8s/ingress/nginx-ingress.yaml renamed to k8s/local/nginx-ingress.yaml

File renamed without changes.

0 commit comments

Comments
 (0)