Skip to content

Commit a70ff25

Browse files
marinakogmarkxnelson
marinakog
authored and
markxnelson
committed
WIP: added sample terraform scripts to create OKE (#807)
* added example of terraform scripts config files to create OKE cluster * removed code for building terraform oci provider, added copiright info, corrected typos * corrected format * corrected format
1 parent a2cf6da commit a70ff25

File tree

10 files changed

+781
-0
lines changed

10 files changed

+781
-0
lines changed

‎kubernetes/samples/README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ While these samples may be useful and usable as is, it is intended that you woul
1414
* [Sample for creating a WebLogic domain home inside a Docker image](scripts/create-weblogic-domain/domain-home-in-image/README.md), and the domain resource YAML file for deploying the generated WebLogic domain.
1515
* [Sample for configuring the Elasticsearch and Kibana](scripts/elasticsearch-and-kibana/README.md) deployments and services for the operator's logs.
1616
* [Sample for generating a self-signed certificate and private key](scripts/rest/README.md) that can be used for the operator's external REST API.
17+
* [Sample for creating an OKE cluster using Terraform](scripts/terraform/README.md).
1718

1819
## Sample Helm charts
1920

‎kubernetes/samples/scripts/terraform/README.md

Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Sample to create an OKE cluster using Terraform scripts
2+
3+
The provided sample will create:
4+
5+
* A new Virtual Cloud Network (VCN) for the cluster
6+
* Two LoadBalancer subnets with security lists
7+
* Three Worker subnets with security lists
8+
* A Kubernetes Cluster with one Node Pool
9+
* A `kubeconfig` file to allow access using `kubectl`
10+
11+
Nodes and network settings will be configured to allow SSH access, and the cluster networking policies will allow `NodePort` services to be exposed. This cluster can be used for testing and development purposes only. The provided samples of Terraform scripts should not be considered for creating production clusters, without more of a review.
12+
13+
All OCI Container Engine masters are Highly Available (HA) and fronted by load balancers.
14+
15+
16+
17+
## Prerequisites
18+
19+
To use these Terraform scripts, you will need fulfill the following prerequisites:
20+
* Have an existing tenancy with enough compute and networking resources available for the desired cluster.
21+
* Have an [Identity and Access Management](https://docs.cloud.oracle.com/iaas/Content/ContEng/Concepts/contengpolicyconfig.htm#PolicyPrerequisitesService) policy in place within that tenancy to allow the OCI Container Engine for Kubernetes service to manage tenancy resources.
22+
* Have a user defined within that tenancy.
23+
* Have an API key defined for use with the OCI API, as documented [here](https://docs.cloud.oracle.com/iaas/Content/Identity/Tasks/managingcredentials.htm).
24+
* Have an [SSH key pair](https://docs.oracle.com/en/cloud/iaas/compute-iaas-cloud/stcsg/generating-ssh-key-pair.html) for configuring SSH access to the nodes in the cluster.
25+
26+
Copy provided `oci.props.template` file to `oci.props` and add all required values:
27+
* `user.ocid` - OCID for the tenancy user - can be obtained from the user settings in the OCI console.
28+
* `tfvars.filename` - File name for generated tfvar file.
29+
* `okeclustername` - The name for OCI Container Engine for Kubernetes cluster.
30+
* `tenancy.ocid` - OCID for the target tenancy.
31+
* `region` - name of region in the target tenancy.
32+
* `compartment.ocid` - OCID for the target compartment.
33+
* `compartment.name` - Name for the target compartment.
34+
* `ociapi.pubkey.fingerprint` - Fingerprint of the OCI user's public key.
35+
* `ocipk.path` - API Private Key -- local path to the private key for the API key pair.
36+
* `vcn.cidr.prefix` - Prefix for VCN CIDR, used when creating subnets -- you should examine the target compartment find a CIDR that is available.
37+
* `vcn.cidr` - Full CIDR for the VCN, must be unique within the compartment, first 2 octets should match the vcn_cidr_prefix.
38+
* `nodepool.shape` - A valid OCI VM Shape for the cluster nodes.
39+
* `k8s.version` - SSH public key (key contents as a string).
40+
* `nodepool.imagename` - A valid image name for Node Pool creation.
41+
* `terraform.installdir` - Location to install Terraform binaries.
42+
43+
To run the script, use the command:
44+
```
45+
$ kubernetes/samples/scripts/terraform/oke.create.sh oci.props
46+
```
47+
The script collects the values from `oci.props` file and performs the following steps:
48+
* Creates a new tfvars file based on the values from the provided `oci.props` file.
49+
* Downloads and installs all needed binaries for Terraform, Terraform OCI Provider, based on OS system (macOS or Linux)
50+
* Applies the configuration and creates OKE Cluster using Terraform
51+
52+

‎kubernetes/samples/scripts/terraform/cluster.tf

Lines changed: 62 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,62 @@
1+
/*
2+
# Copyright 2018, 2019, Oracle Corporation and/or its affiliates. All rights reserved.
3+
# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl.
4+
*/
5+
variable "cluster_kubernetes_version" { default = "v1.11.5" }
6+
variable "cluster_name" { default = "tfTestCluster" }
7+
variable "cluster_options_add_ons_is_kubernetes_dashboard_enabled" { default = true }
8+
variable "cluster_options_add_ons_is_tiller_enabled" { default = true }
9+
variable "cluster_options_kubernetes_network_config_pods_cidr" { default = "10.1.0.0/16" }
10+
variable "cluster_options_kubernetes_network_config_services_cidr" { default = "10.2.0.0/16" }
11+
variable "node_pool_initial_node_labels_key" { default = "key" }
12+
variable "node_pool_initial_node_labels_value" { default = "value" }
13+
variable "node_pool_kubernetes_version" { default = "v1.11.5" }
14+
variable "node_pool_name" { default = "tfTestCluster_workers" }
15+
variable "node_pool_node_image_name" { default = "Oracle-Linux-7.4" }
16+
variable "node_pool_node_shape" { default = "VM.Standard2.1" }
17+
variable "node_pool_quantity_per_subnet" { default = 2 }
18+
variable "node_pool_ssh_public_key" { }
19+
20+
data "oci_identity_availability_domains" "tfsample_availability_domains" {
21+
compartment_id = "${var.compartment_ocid}"
22+
}
23+
24+
25+
resource "oci_containerengine_cluster" "tfsample_cluster" {
26+
#Required
27+
compartment_id = "${var.compartment_ocid}"
28+
kubernetes_version = "${var.cluster_kubernetes_version}"
29+
name = "${var.cluster_name}"
30+
vcn_id = "${oci_core_virtual_network.oke-vcn.id}"
31+
32+
#Optional
33+
options {
34+
service_lb_subnet_ids = ["${oci_core_subnet.oke-subnet-loadbalancer-1.id}", "${oci_core_subnet.oke-subnet-loadbalancer-2.id}"]
35+
36+
#Optional
37+
add_ons {
38+
#Optional
39+
is_kubernetes_dashboard_enabled = "${var.cluster_options_add_ons_is_kubernetes_dashboard_enabled}"
40+
is_tiller_enabled = "${var.cluster_options_add_ons_is_tiller_enabled}"
41+
}
42+
}
43+
}
44+
45+
resource "oci_containerengine_node_pool" "tfsample_node_pool" {
46+
#Required
47+
cluster_id = "${oci_containerengine_cluster.tfsample_cluster.id}"
48+
compartment_id = "${var.compartment_ocid}"
49+
kubernetes_version = "${var.node_pool_kubernetes_version}"
50+
name = "${var.node_pool_name}"
51+
node_image_name = "${var.node_pool_node_image_name}"
52+
node_shape = "${var.node_pool_node_shape}"
53+
subnet_ids = ["${oci_core_subnet.oke-subnet-worker-1.id}", "${oci_core_subnet.oke-subnet-worker-2.id}","${oci_core_subnet.oke-subnet-worker-3.id}"]
54+
55+
#Optional
56+
quantity_per_subnet = "${var.node_pool_quantity_per_subnet}"
57+
ssh_public_key = "${var.node_pool_ssh_public_key}"
58+
}
59+
60+
output "cluster_id" {
61+
value = "${oci_containerengine_cluster.tfsample_cluster.id}"
62+
}

‎kubernetes/samples/scripts/terraform/kube_config.tf

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
/*
2+
# Copyright 2018, 2019, Oracle Corporation and/or its affiliates. All rights reserved.
3+
# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl.
4+
*/
5+
6+
variable "cluster_kube_config_expiration" { default = 2592000 }
7+
variable "cluster_kube_config_token_version" { default = "1.0.0" }
8+
9+
data "oci_containerengine_cluster_kube_config" "tfsample_cluster_kube_config" {
10+
#Required
11+
cluster_id = "${oci_containerengine_cluster.tfsample_cluster.id}"
12+
}
13+
14+
resource "local_file" "tfsample_cluster_kube_config_file" {
15+
content = "${data.oci_containerengine_cluster_kube_config.tfsample_cluster_kube_config.content}"
16+
filename = "${path.module}/${var.cluster_name}_kubeconfig"
17+
}

‎kubernetes/samples/scripts/terraform/oci.props.example

Lines changed: 17 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,17 @@
1+
user.ocid=ocid1.user.oc1..aaaaaaaast7s6jdho6mh2dqvyqcychofaiv5lhztkx7u5jlr5wwuhhmewq
2+
okeclustername=myokecluster
3+
tfvars.filename=myokeclustertf
4+
region=us-phoenix-1
5+
tenancy.ocid=ocid1.tenancy.oc1..aaaaaahmcbb5mp2h6toh4vj7ax526xtmihrneoumyat557rvlolsx63imq
6+
compartment.ocid=ocid1.compartment.oc1..aaaaaaaaxzwkinzejhkncuvfy67pmb6wb46ifrixtuikkrgnnrp4wswsu4xq
7+
compartment.name=QualityAssurance
8+
ociapi.pubkey.fingerprint=c8\:b2\:da\:b2\:e8\:96\:7e\:bf\:ac\:ee\:ce\:bc\:a8\:7f\:07\:c5
9+
ocipk.path=/scratch/mkogan/.oci/oci_api_key.pem
10+
vcn.cidr.prefix=10.1
11+
vcn.cidr=10.1.0.0/16
12+
nodepool.shape=VM.Standard2.1
13+
nodepool.imagename=Oracle-Linux-7.4
14+
k8s.version=v1.10.11
15+
nodepool.ssh.pubkey=ssh-rsa AAAAB3NzaC1yc2EAAAAQABAAABAQC9FSfGdjjL+EZre2p5yLTAgtLsnp49AUVX1yY9V8guaXHol6UkvJWnyFHhL7s0qvWj2M2BYo6WAROVc0/054UFtmbd9zb2oZtGVk82VbT6aS74cMlqlY91H/rt9/t51Om9Sp5AvbJEzN0mkI4ndeG/5p12AUyg9m5XOdkgI2n4J8KFnDAI33YSGjxXb7UrkWSGl6XZBGUdeaExo3t2Ow8Kpl9T0Tq19qI+IncOecsCFj1tbM5voD8IWE2l0SW7V6oIqFJDMecq4IZusXdO+bPc+TKak7g82RUZd8PARpvYB5/7EOfVadxsXGRirGAKPjlXDuhwJYVRj1+IjZ+5Suxz mkogan@slc13kef
16+
terraform.installdir=/scratch/mkogan/myterraformtest
17+

‎kubernetes/samples/scripts/terraform/oci.props.template

Lines changed: 45 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,45 @@
1+
# Copyright 2018, 2019, Oracle Corporation and/or its affiliates. All rights reserved.
2+
# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl.
3+
4+
# Properties to generate TF variables file for cluster creation from property file oci.props
5+
#
6+
# Copy this file to oci.props and update it with your own info, see oci.props.example as sample for values
7+
#
8+
9+
# OCID can be obtained from the user info page in the OCI console
10+
user.ocid=
11+
12+
# name of OKE cluster
13+
okeclustername=
14+
15+
# name of tfvars file (no extention) to generate
16+
tfvars.filename=
17+
18+
# Required tenancy info
19+
tenancy.ocid=
20+
compartment.ocid=
21+
compartment.name=
22+
region=
23+
24+
# API key fingerprint and private key location, needed for API access -- you should have added a public API key through the OCI console first, add escape backslash \ for each colon signt
25+
ociapi.pubkey.fingerprint=
26+
27+
# path to private OCI API key
28+
ocipk.path=
29+
30+
# VCN CIDR -- must be unique within the compartment in the tenancy
31+
# - assuming 1:1 cluster:vcn
32+
# BE SURE TO SET BOTH VARS -- the first 2 octets for each variable have to match
33+
vcn.cidr.prefix=
34+
vcn.cidr=
35+
36+
# Node pool info
37+
nodepool.shape=
38+
nodepool.ssh.pubkey=
39+
nodepool.imagename=
40+
41+
# K8S version
42+
k8s.version=
43+
44+
#location for terraform installation
45+
terraform.installdir=

‎kubernetes/samples/scripts/terraform/oke.create.sh

Lines changed: 109 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,109 @@
1+
#!/bin/bash
2+
# Copyright 2018, 2019, Oracle Corporation and/or its affiliates. All rights reserved.
3+
# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl.
4+
5+
function prop {
6+
grep "${1}" ${propsFile}|cut -d'=' -f2
7+
}
8+
9+
function generateTFVarFile {
10+
tfVarsFiletfVarsFile=${terraformVarDir}/${clusterTFVarsFile}.tfvars
11+
rm -f ${tfVarsFiletfVarsFile}
12+
cp ${terraformVarDir}/template.tfvars $tfVarsFiletfVarsFile
13+
chmod 777 ${terraformVarDir}/template.tfvars $tfVarsFiletfVarsFile
14+
15+
sed -i -e "s:@TENANCYOCID@:${tenancy_ocid}:g" ${tfVarsFiletfVarsFile}
16+
sed -i -e "s:@USEROCID@:${user_ocid}:g" ${tfVarsFiletfVarsFile}
17+
sed -i -e "s:@COMPARTMENTOCID@:${compartment_ocid}:g" ${tfVarsFiletfVarsFile}
18+
sed -i -e "s:@COMPARTMENTNAME@:${compartment_name}:g" ${tfVarsFiletfVarsFile}
19+
sed -i -e "s:@OKECLUSTERNAME@:${okeclustername}:g" ${tfVarsFiletfVarsFile}
20+
sed -i -e "s:@OCIAPIPUBKEYFINGERPRINT@:"${ociapi_pubkey_fingerprint}":g" ${tfVarsFiletfVarsFile}
21+
sed -i -e "s:@OCIPRIVATEKEYPATH@:${ocipk_path}:g" ${tfVarsFiletfVarsFile}
22+
sed -i -e "s:@VCNCIDRPREFIX@:${vcn_cidr_prefix}:g" ${tfVarsFiletfVarsFile}
23+
sed -i -e "s:@VCNCIDR@:${vcn_cidr_prefix}.0.0/16:g" ${tfVarsFiletfVarsFile}
24+
sed -i -e "s:@OKEK8SVERSION@:${k8s_version}:g" ${tfVarsFiletfVarsFile}
25+
sed -i -e "s:@NODEPOOLSHAPE@:${nodepool_shape}:g" ${tfVarsFiletfVarsFile}
26+
sed -i -e "s:@NODEPOOLIMAGENAME@:${nodepool_imagename}:g" ${tfVarsFiletfVarsFile}
27+
sed -i -e "s:@NODEPOOLSSHPUBKEY@:${nodepool_ssh_pubkey}:g" ${tfVarsFiletfVarsFile}
28+
sed -i -e "s:@REGION@:${region}:g" ${tfVarsFiletfVarsFile}
29+
echo "Generated TFVars file [${tfVarsFiletfVarsFile}]"
30+
31+
}
32+
33+
function setupTerraform () {
34+
mkdir ${terraformDir}
35+
cd ${terraformDir}
36+
if [[ "${OSTYPE}" == "darwin"* ]]; then
37+
curl -O https://releases.hashicorp.com/terraform/0.11.10/terraform_0.11.10_darwin_amd64.zip
38+
unzip terraform_0.11.10_darwin_amd64.zip
39+
elif [[ "${OSTYPE}" == "linux"* ]]; then
40+
curl -O https://releases.hashicorp.com/terraform/0.11.8/terraform_0.11.8_linux_amd64.zip
41+
unzip terraform_0.11.8_linux_amd64.zip
42+
else
43+
echo "Unsupported OS"
44+
fi
45+
chmod 777 ${terraformDir}/terraform
46+
export PATH=${terraformDir}:${PATH}
47+
48+
}
49+
50+
function deleteOlderVersionTerraformOCIProvider() {
51+
if [ -d ~/.terraform.d/plugins ]; then
52+
echo "Deleting older version of terraform plugins dir"
53+
rm -rf ~/.terraform.d/plugins
54+
fi
55+
if [ -d ${terraformVarDir}/.terraform ]; then
56+
rm -rf ${terraformVarDir}/.terraform
57+
fi
58+
if [ -e ~/.terraformrc ]; then
59+
rm ~/.terraformrc
60+
fi
61+
}
62+
63+
function createCluster () {
64+
cd ${terraformVarDir}
65+
echo "terraform init -var-file=${terraformVarDir}/${clusterTFVarsFile}.tfvars"
66+
terraform init -var-file=${terraformVarDir}/${clusterTFVarsFile}.tfvars
67+
terraform plan -var-file=${terraformVarDir}/${clusterTFVarsFile}.tfvars
68+
terraform apply -auto-approve -var-file=${terraformVarDir}/${clusterTFVarsFile}.tfvars
69+
}
70+
71+
#MAIN
72+
propsFile=${1:-$PWD/oci.props}
73+
terraformVarDir=${2:-$PWD}
74+
75+
#grep props's values from oci.props file
76+
77+
clusterTFVarsFile=$(prop 'tfvars.filename')
78+
tenancy_ocid=$(prop 'tenancy.ocid')
79+
user_ocid=$(prop 'user.ocid')
80+
compartment_ocid=$(prop 'compartment.ocid')
81+
compartment_name=$(prop 'compartment.name')
82+
okeclustername=$(prop 'okeclustername')
83+
ociapi_pubkey_fingerprint=$(prop 'ociapi.pubkey.fingerprint')
84+
ocipk_path=$(prop 'ocipk.path')
85+
vcn_cidr_prefix=$(prop 'vcn.cidr.prefix')
86+
k8s_version=$(prop 'k8s.version')
87+
nodepool_shape=$(prop 'nodepool.shape')
88+
nodepool_imagename=$(prop 'nodepool.imagename')
89+
nodepool_ssh_pubkey=$(prop 'nodepool.ssh.pubkey')
90+
region=$(prop 'region')
91+
terraformDir=$(prop 'terraform.installdir')
92+
93+
# generate terraform configuration file with name $(clusterTFVarsFile).tfvar
94+
generateTFVarFile
95+
96+
# cleanup previously installed terraform binaries
97+
rm -rf ${terraformDir}
98+
99+
# download terraform binaries into ${terraformDir}
100+
setupTerraform
101+
102+
# clean previous versions of terraform oci provider
103+
deleteOlderVersionTerraformOCIProvider
104+
105+
chmod 600 ${ocipk_path}
106+
107+
# run terraform init,plan,apply to create OKE cluster based on the provided tfvar file ${clusterTFVarsFile).tfvar
108+
createCluster
109+
export KUBECONFIG=${terraformVarDir}/${okeclustername}_kubeconfig

‎kubernetes/samples/scripts/terraform/provider.tf

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
/*
2+
* Copyright 2018, 2019, Oracle Corporation and/or its affiliates. All rights reserved.
3+
* Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl.
4+
* This example file shows how to configure the oci provider to target the a single region.
5+
*/
6+
7+
// These variables would commonly be defined as environment variables or sourced in a .env file
8+
variable "tenancy_ocid" {}
9+
variable "user_ocid" {}
10+
variable "fingerprint" {}
11+
variable "private_key_path" {}
12+
variable "region" { default = "us-phoenix-1" }
13+
14+
provider "oci" {
15+
version = ">= 3.0.0"
16+
region = "${var.region}"
17+
tenancy_ocid = "${var.tenancy_ocid}"
18+
user_ocid = "${var.user_ocid}"
19+
fingerprint = "${var.fingerprint}"
20+
private_key_path = "${var.private_key_path}"
21+
}

‎kubernetes/samples/scripts/terraform/template.tfvars

Lines changed: 46 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,46 @@
1+
# Copyright 2018, 2019, Oracle Corporation and/or its affiliates. All rights reserved.
2+
# Licensed under the Universal Permissive License v 1.0 as shown at http://oss.oracle.com/licenses/upl.
3+
4+
# Template to generate TF variables file for cluster creation from property file oci.props
5+
#
6+
# User-specific vars - you can get these easily from the OCI console from your user page
7+
#
8+
9+
# OCID can be obtained from the user info page in the OCI console
10+
user_ocid="@USEROCID@"
11+
# API key fingerprint and private key location, needed for API access -- you should have added a public API key through the OCI console first
12+
fingerprint="@OCIAPIPUBKEYFINGERPRINT@"
13+
private_key_path="@OCIPRIVATEKEYPATH@"
14+
15+
# Required tenancy vars
16+
tenancy_ocid="@TENANCYOCID@"
17+
compartment_ocid="@COMPARTMENTOCID@"
18+
compartment_name="@COMPARTMENTNAME@"
19+
region="@REGION@"
20+
21+
#
22+
# Cluster-specific vars
23+
#
24+
25+
# VCN CIDR -- must be unique within the compartment in the tenancy
26+
# - assuming 1:1 cluster:vcn
27+
# BE SURE TO SET BOTH VARS -- the first 2 octets for each variable have to match
28+
vcn_cidr_prefix="@VCNCIDRPREFIX@"
29+
vcn_cidr="@VCNCIDR@"
30+
31+
# Cluster name and k8s version
32+
cluster_kubernetes_version="@OKEK8SVERSION@"
33+
cluster_name="@OKECLUSTERNAME@"
34+
35+
# Node pool info
36+
node_pool_kubernetes_version="@OKEK8SVERSION@"
37+
node_pool_name="@OKECLUSTERNAME@_workers"
38+
node_pool_node_shape="@NODEPOOLSHAPE@"
39+
node_pool_node_image_name="@NODEPOOLIMAGENAME@"
40+
node_pool_quantity_per_subnet=1
41+
42+
# SSH public key, for SSH access to nodes in the cluster
43+
node_pool_ssh_public_key="@NODEPOOLSSHPUBKEY@"
44+
45+
46+

0 commit comments

Comments
 (0)