Skip to content

Commit e143ce5

Browse files
NogbitNogbit
authored
Autoscale OKE Node Pools with Agones Game Servers and Fleets (oracle-livelabs#443)
* adding OKE with Agones * fixing code blocks * testing markdown * fixing readme * fix readme * linting * linting * Update 4-create-agones-system.md adding -v to nc so user sees a success on the connect --------- Co-authored-by: nogbit <[email protected]>
1 parent e57d2ba commit e143ce5

File tree

12 files changed

+868
-0
lines changed

12 files changed

+868
-0
lines changed

‎oke-with-agones/0-workshop-introduction/0-workshop-introduction.md‎

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
# Introduction
2+
3+
## **What is OKE and Agones?**
4+
5+
### OKE Intro
6+
7+
Oracle Kubernetes Engine (OKE) is the runtime within OCI for the running and operations of enterprise-grade Kubernetes at scale. You can easily deploy and manage resource-intensive workloads such as dedicated game servers with automatic scaling, patching, and upgrades.
8+
9+
### Agones Intro
10+
11+
Agones is an open source platform, for deploying, hosting, scaling, and orchestrating dedicated game servers for large scale multiplayer games, built on top of the industry standard, distributed system platform Kubernetes.
12+
13+
Agones replaces bespoke or proprietary cluster management and game server scaling solutions with an open source solution that can be utilized and communally developed - so that you can focus on the important aspects of building a multiplayer game, rather than developing the infrastructure to support it.
14+
15+
### Workshop Lab Objectives
16+
17+
* Ensure you have installed the prerequisites
18+
* Create the OCI infrastructure
19+
* Setup OKE Autoscaling
20+
* Setup Agones system pods with Helm
21+
* Deploy an Agones Fleet (dedicated game servers)
22+
* Scale a Agones Fleet and OKE nodes
23+
* Teardown
24+
25+
### Labs
26+
27+
| Module | Est. Time |
28+
| ------------- | :-----------: |
29+
| [Workshop Introduction](?lab=0-workshop-introduction) | 5 minutes |
30+
| [Get Started](?lab=1-get-started) | 15 minutes |
31+
| [Creating OCI Resources With Terraform](?lab=2-create-infrastructure-with-terraform) | 30 minutes |
32+
| [Installing the OKE Autoscaler Addon](?lab=3-install-oke-autoscaler-addon) | 20 minutes |
33+
| [Create the Agones System Pods with Helm](?lab=4-create-agones-system) | 15 minutes |
34+
| [Deploy an Agones Fleet and Autoscale OKE Nodes](?lab=5-create-scale-agones-fleet) | 25 minutes |
35+
| [Teardown](?lab=6-teardown) | 10 minutes |
36+
37+
Total estimated time: 120 minutes
38+
39+
## Task 1: Begin The Labs
40+
41+
Use the left navigation on this page to begin the labs in this workshop.
42+
43+
You may now **proceed to the next lab**
44+
45+
## Learn More - *Useful Links*
46+
47+
- [Kubernetes](https://kubernetes.io/)
48+
- [OKE](https://www.oracle.com/cloud/cloud-native/kubernetes-engine/)
49+
- [OKE Terraform Module](https://github.com/oracle-terraform-modules/terraform-oci-oke)
50+
- [Agones](https://agones.dev/site/docs/)
51+
52+
## **Acknowledgements**
53+
54+
- **Author** - Marcellus Miles, Master Cloud Architect
55+
- **Last Updated By/Date** - Marcellus Miles, Dec 2024

‎oke-with-agones/1-get-started/1-get-started.md‎

Lines changed: 89 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,89 @@
1+
# Get Started
2+
3+
In this lab you will install the necessary components for this workshop.
4+
5+
## Introduction
6+
7+
In order to complete this workshop you will need to have the necessary tooling to connect to and deploy OCI resources.
8+
9+
You will be using Terraform to deploy to OCI and will also need the OCI CLI.
10+
11+
Estimated Time: 15 minutes
12+
13+
### Objectives
14+
15+
In this lab, you will:
16+
- Install the OCI CLI
17+
- Install Terraform
18+
- Download the Terraform files
19+
- Initialize Terraform
20+
21+
### Prerequisites
22+
23+
Please ensure you have the following before continuing
24+
25+
- An OCI Tenancy
26+
- A Shell ([OCI Cloud Shell](https://docs.oracle.com/en-us/iaas/Content/API/Concepts/cloudshellintro.htm), Linux, MacOS, Windows with WSL)
27+
- A user in a group with Tenancy Admin and downloaded API Key
28+
29+
## Task 1: Install the OCI CLI
30+
31+
Install the OCI CLI
32+
33+
1. Make sure you have the policy for Tenancy admin. This is required because the Terraform OKE module creates a dynamic group policy, other than that everything gets created in a OCI Compartment you specify in the `terraform.tfvars` file.
34+
35+
1. Follow the [install steps from Oracle](https://docs.oracle.com/en-us/iaas/Content/API/SDKDocs/cliinstall.htm)
36+
37+
2. Make sure you followed the steps above and are fully setup with API Keys (this will be the case if you did `oci setup config`) for the user of the above mentioned Tenancy admin.
38+
39+
## Task 2: Install Terraform
40+
41+
Complete the [install steps form Hashicorp](https://developer.hashicorp.com/terraform/tutorials/aws-get-started/install-cli)
42+
43+
## Task 3: Download the Terraform Files
44+
45+
Download the Terraform files
46+
47+
1. Create a directory called `infrastructure` in your system
48+
49+
````shell
50+
<copy>
51+
mkdir infrastructure
52+
cd infrastructure
53+
</copy>
54+
````
55+
56+
2. Download the terraform files from [terraform.tar.gz](./files/terraform.tar.gz) to `infrastructure`.
57+
58+
3. Untar the downloaded file
59+
60+
````shell
61+
<copy>
62+
tar -xvzf terraform.tar.gz
63+
</copy>
64+
````
65+
66+
3. From within the infrastructure folder, initialize the Terraform
67+
68+
````shell
69+
<copy>
70+
terraform init
71+
</copy>
72+
````
73+
74+
You may now **proceed to the next lab**
75+
76+
## Learn More - *Useful Links*
77+
78+
- [Kubernetes](https://kubernetes.io/)
79+
- [OKE](https://www.oracle.com/cloud/cloud-native/kubernetes-engine/)
80+
- [OKE Terraform Module](https://oracle-terraform-modules.github.io/terraform-oci-oke/)
81+
82+
## **Summary**
83+
84+
You have now initialized the dependencies for this Workshop.
85+
86+
## **Acknowledgements**
87+
88+
- **Author** - Marcellus Miles, Master Cloud Architect
89+
- **Last Updated By/Date** - Marcellus Miles, Dec 2024

‎oke-with-agones/1-get-started/files/terraform.tar.gz‎

3.37 KB
Binary file not shown.

‎oke-with-agones/2-create-infrastructure-with-terraform/2-create-infrastructure-with-terraform.md‎

Lines changed: 91 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,91 @@
1+
# Create OCI Resources With Terraform
2+
3+
In this lab you will create the OCI Network, Bastion, Operator and OKE cluster using Terraform and the OKE Terraform module. It's important to read through the [OKE Module documentation](https://oracle-terraform-modules.github.io/terraform-oci-oke/) as there are numerous options that can apply to your specific OCI deployment.
4+
5+
## Introduction
6+
7+
This Terraform deployment creates the following resources
8+
9+
- Private OKE Control plane
10+
- Private Operator (with kubectl installed)
11+
- Public bastion (for SSH tunneling to Operator)
12+
- Three node pools (one public for game servers, and two private for the Autoscaler and Agones system pods respectively)
13+
- Security Group Rules for UDP access (game server to game client connectivity)
14+
- VCN Logs for logging traffic
15+
16+
Estimated Time: 30 minutes
17+
18+
### Objectives
19+
20+
In this lab, you will:
21+
- Configure your Terraform variables
22+
- Run a Terraform plan and apply changes
23+
24+
### Prerequisites
25+
26+
- Completed Lab 1 which walked through sourcing the Terraform files
27+
28+
## Task 1: Update Terraform Variables
29+
30+
Customize the infrastructure to fit your tenancy and compartment.
31+
32+
1. You will want to edit `infrastructre/terraform.tfvars` with relevant information to match your account OCID's and API Keys. For a full description of each variable in that file refer to `infrasctucture/variables.tf`.
33+
34+
2. You can (optional) edit `infrastructure/module.tf` to tweak the OKE settings as needed for your deployment. The current settings will work as is for the purpose of this lab. Its a good idea to have a look at these settings since the OKE Terraform module does a lot and is very customizable.
35+
36+
## Task 2: Create a Terraform Plan
37+
38+
Get a Terraform plan and check the output of that plan to make sure its what you expect. After you validate the plan you can move onto the next task.
39+
40+
````shell
41+
<copy>
42+
cd infrastructure
43+
terraform plan
44+
</copy>
45+
````
46+
47+
## Task 3: Apply the Terraform Plan
48+
49+
You can now apply the plan and the infrastructure will get created. Wait some time for the apply to complete.
50+
51+
````shell
52+
<copy>
53+
terraform apply
54+
</copy>
55+
````
56+
57+
## Task 4: Connect to the bastion
58+
59+
Connect to the Bastion to ensure you have access to the OKE control plane.
60+
61+
1. Get the terraform output. It will give an example command that you can use to SSH to the bastion and jump to the operator.
62+
63+
````shell
64+
<copy>
65+
terraform output
66+
</copy>
67+
````
68+
69+
2. The operator has kubectl installed with connectivity to the OKE control plane. **An example of that output is below**. You should test this command and make sure it works before proceeding to the next section of this workshop.
70+
71+
````shell
72+
<copy>
73+
ssh -J opc@<bastion public IP> opc@<operator private ip>
74+
</copy>
75+
````
76+
77+
You may now **proceed to the next lab**
78+
79+
## **Summary**
80+
81+
You have now deployed the necessary infrastructure and connected to the Bastion and jumped to the Operator. You are ready to begin installing the autoscaler and more.
82+
83+
## Learn More - *Useful Links*
84+
85+
- [OKE Terraform Module](https://oracle-terraform-modules.github.io/terraform-oci-oke/)
86+
- [Terraform Variables](https://developer.hashicorp.com/terraform/language/values/variables)
87+
88+
## **Acknowledgements**
89+
90+
- **Author** - Marcellus Miles, Master Cloud Architect
91+
- **Last Updated By/Date** - Marcellus Miles, Dec 2024

‎oke-with-agones/3-install-oke-autoscaler-addon/3-install-oke-autoscaler-addon.md‎

Lines changed: 124 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,124 @@
1+
# Install the OKE Autoscaler Addon
2+
3+
In this lab you will install and verify the OKE Cluster Autoscaler Add-on.
4+
5+
## Introduction
6+
7+
The OKE Cluster Autoscaler Add-on is what is used to watch and manage specified node pools that should be autoscaled. The node pool you will configure this add on for is the pool that will be running the Agones fleet in future labs of this workshop.
8+
9+
The add-on itself is installed into its own node pool to isolate it from its own scaling events.
10+
11+
12+
Estimated Time: 20 minutes
13+
14+
### Objectives
15+
16+
In this lab, you will:
17+
- Verify the installation of the Autoscaler
18+
- Create a config file for the Autoscaler Add-on
19+
- Install the Autoscaler Add-on
20+
21+
### Prerequisites
22+
23+
- Completed Lab 2 which walked through deploying the infrastructure
24+
25+
## Task 1: Verify if the Autoscaler is already installed
26+
27+
Depending on the OKE Terraform module used and your connectivity when creating the infrastructure the OKE Terraform may have installed the addon for you. Its configured to do so, but does not always work in some scenarios.
28+
29+
You should verify the current state of addons to see if the autoscaler was installed by following the steps below.
30+
31+
1. SSH to your Operater using the output from `terraform output`, example below.
32+
33+
```bash
34+
<copy>
35+
ssh -J opc@<bastion public IP> opc@<operator private ip>
36+
</copy>
37+
```
38+
39+
2. Get the OCID of your cluster by running this command and looking for the `id` (which is the OCID) of the cluster you just created. This can also be obtained from the web console.
40+
41+
````shell
42+
<copy>
43+
oci ce cluster list -c <OCID of Compartment you used in terraform.tfvars>
44+
</copy>
45+
````
46+
47+
3. Get the Addons installed on your cluster
48+
49+
````shell
50+
<copy>
51+
oci ce cluster list-addons --cluster-id <OCID of your OKE Cluster from previous step>
52+
</copy>
53+
````
54+
55+
If `Autoscaler` is listed as one of the addons you can go to the next lab in this workshop. If not, proceed with the remaining tasks here to install it.
56+
57+
## Task 2: Install the Autoscaler Addon
58+
59+
Assuming the task before this indicated the addon was not installed you can now install the addon. You can also optionally do the following steps manually in the web console for OKE.
60+
61+
1. SSH to your Operater using the output from `terraform output`, example below.
62+
63+
```bash
64+
<copy>
65+
ssh -J opc@<bastion public IP> opc@<operator private ip>
66+
</copy>
67+
```
68+
69+
2. Get the OCID of the `node_pool_workers` node pool. This is the pool that will run the Agones fleet in subsequent labs of this workshop.
70+
71+
````shell
72+
<copy>
73+
kubectl get node -l oke.oraclecloud.com/pool.name=node_pool_workers -o json |grep node-pool-id
74+
</copy>
75+
````
76+
77+
**This OCID is the Node Pool OCID that you will use in the next step.**
78+
79+
3. The file [addon.json](./files/addon.json) will be used as an example. It's format is `<min nodes>:<max nodes>:<node pool id>`. It's important to remember that as your node pools change (renaming, changing terraform etc) their respective OCID's will change and you will need to update this config.
80+
81+
Create the config as `addon.json` and paste the contents from [addon.json](./files/addon.json). Replace from the file `<NODE POOL OCID>` with the OCID from the previous step above.
82+
83+
```bash
84+
<copy>
85+
# using vim or nano
86+
vim addon.json
87+
88+
# paste from addon.json into this new file and save
89+
# Paste in the correct Node Pool OCID
90+
</copy>
91+
```
92+
93+
4. Install the addon using the newly created config file. This should run without error and a resulting work request ID will be displayed.
94+
95+
96+
````shell
97+
<copy>
98+
oci ce cluster install-addon --addon-name ClusterAutoscaler --from-json file://addon.json --cluster-id <ocid of cluster>
99+
</copy>
100+
````
101+
102+
5. Verify there are no errors with the newly installed addon. The result should say `ACTIVE`.
103+
104+
````shell
105+
<copy>
106+
oci ce cluster get-addon --addon-name ClusterAutoscaler --cluster-id <ocid of cluster> | grep lifecycle-state
107+
</copy>
108+
````
109+
110+
You may now **proceed to the next lab**
111+
112+
## **Summary**
113+
114+
You have installed the OKE Cluster Autoscaler addon on and configured it to watch the node pool that will be running the Agones fleet in subsequent labs of this workshop.
115+
116+
## Learn More - *Useful Links*
117+
118+
- [Working with the OKE Cluster Autoscaler Add-on](https://docs.oracle.com/en-us/iaas/Content/ContEng/Tasks/contengusingclusterautoscaler_topic-Working_with_Cluster_Autoscaler_as_Cluster_Add-on.htm)
119+
- ["oci ce cluster addon" documentation](https://docs.oracle.com/en-us/iaas/tools/oci-cli/3.50.3/oci_cli_docs/cmdref/ce/cluster.html)
120+
121+
## **Acknowledgements**
122+
123+
- **Author** - Marcellus Miles, Master Cloud Architect
124+
- **Last Updated By/Date** - Marcellus Miles, Dec 2024

‎oke-with-agones/3-install-oke-autoscaler-addon/files/addon.json‎

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
{
2+
"addonName": "ClusterAutoscaler",
3+
"configurations": [
4+
{
5+
"key": "nodes",
6+
"value": "2:10:<NODE POOL OCID>"
7+
}
8+
]
9+
}

0 commit comments

Comments
 (0)