Set AWS_REGION (#63)

Author: runakash

CONTRIBUTING.md

@@ -1,5 +1,24 @@
 # Contributing Guidelines
 
+* [Getting Started](#getting-started)
+    + [Local Setup](#local-setup)
+        - [Prerequisites](#prerequisites)
+        - [Cluster Setup](#cluster-setup)
+        - [Run the controller from outside the cluster](#run-the-controller-from-outside-the-cluster)
+        - [Build and deploy to the cluster](#build-and-deploy-to-the-cluster)
+        - [Run unit tests](#run-unit-tests)
+        - [Cleanup](#cleanup)
+    + [Deploying to a cluster](#deploying-to-a-cluster)
+    + [Build and push docker image to ECR](#build-and-push-docker-image-to-ecr)
+        - [Deployment](#deployment)
+        - [Uninstallation](#uninstallation)
+* [Reporting Bugs/Feature Requests](#reporting-bugsfeature-requests)
+* [Contributing via Pull Requests](#contributing-via-pull-requests)
+* [Finding contributions to work on](#finding-contributions-to-work-on)
+* [Code of Conduct](#code-of-conduct)
+* [Security issue notifications](#security-issue-notifications)
+* [Licensing](#licensing)
+
 Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional
 documentation, we greatly value feedback and contributions from our community.
 
@@ -8,38 +27,50 @@ information to effectively respond to your bug report or contribution.
 
 ## Getting Started
 
-### Build and Run locally
+### Local Setup
 
 #### Prerequisites
 
 In order to build and run locally:
 
 * Make sure to have `kubectl` [installed](https://kubernetes.io/docs/tasks/tools/install-kubectl/), at least version `1.15` or above.
 * Make sure to have `kind` [installed](https://kind.sigs.k8s.io/docs/user/quick-start/#installation).
-* Make sure, you have created a [HttpNamespace](https://docs.aws.amazon.com/cloud-map/latest/api/API_CreateHttpNamespace.html) in AWS Cloud Map. The examples below assumes the namespace name to be `demo`
+* Make sure, you have access to AWS Cloud Map. As per exercise below, AWS Cloud Map namespace `example` of the type [HttpNamespace](https://docs.aws.amazon.com/cloud-map/latest/api/API_CreateHttpNamespace.html) will be automatically created.
 
 Note that this walk-through assumes throughout to operate in the `us-west-2` region.
 
 ```sh
 export AWS_REGION=us-west-2
 ```
 
-#### Cluster provisioning
+#### Cluster Setup
 
 Spin up a local Kubernetes cluster using `kind`
+
 ```sh
 kind create cluster --name my-cluster
 # Creating cluster "my-cluster" ...
 # ...
 ```
 
 When completed, set the kubectl context
+
 ```sh
 kind export kubeconfig --name my-cluster
 # Set kubectl context to "kind-my-cluster"
 ```
 
+Create `example` namespace in the cluster
+
+```sh
+kubectl create namespace example
+# namespace/example created
+```
+
+#### Run the controller from outside the cluster
+
 To register the custom CRDs (`ServiceImport`, `ServiceExport`) in the cluster and create installers
+
 ```sh
 make install
 # ...
@@ -48,48 +79,88 @@ make install
 ```
 
 To run the controller, run the following command. The controller runs in an infinite loop so open another terminal to create CRDs.
+
 ```sh
 make run 
 ```
 
-Create `demo` namespace
-```sh
-kubectl create namespace demo
-# namespace/demo created
-```
+Apply deployment, service and serviceexport configs
 
-Apply deployment, service and export configs
 ```sh
-kubectl apply -f samples/demo-deployment.yaml
+kubectl apply -f samples/example-deployment.yaml
 # deployment.apps/nginx-deployment created
-kubectl apply -f samples/demo-service.yaml
-# service/demo-service created
-kubectl apply -f samples/demo-export.yaml
-# serviceexport.multicluster.x-k8s.io/demo-service created
+kubectl apply -f samples/example-service.yaml
+# service/my-service created
+kubectl apply -f samples/example-serviceexport.yaml
+# serviceexport.multicluster.x-k8s.io/my-service created
 ```
 
 Check running controller if it correctly detects newly created resources
+
+```
+controllers.ServiceExport	updating Cloud Map service	{"serviceexport": "example/my-service", "namespace": "example", "name": "my-service"}
+cloudmap	                fetching a service	{"namespaceName": "example", "serviceName": "my-service"}
+cloudmap	                creating a new service	{"namespace": "example", "name": "my-service"}
+```
+
+#### Build and deploy to the cluster
+
+Build a `controller:local`  (`--no-cache` args can be set to update the existing docker image)
+
+```shell
+make docker-build IMG=controller:local ARGS="--no-cache"
+# ...
+# docker build --no-cache -t controller:local .
+# ...
+# 
+```
+
+Load the controller docker image into the kind cluster `my-cluster`
+```shell
+kind load docker-image controller:local --name my-cluster
+# Image: "controller:local" with ID "sha256:xxx" not yet present on node "my-cluster-control-plane", loading...
+```
+
+> ⚠ **The controller still needs credentials to interact to AWS SDK.** We are not supporting this configuration out of box. There are multiple ways to achieve this within the cluster.
+
+Finally, create the controller resources in the cluster.
+```shell
+make deploy IMG=controller:local AWS_REGION=us-west-2
+# customresourcedefinition.apiextensions.k8s.io/serviceexports.multicluster.x-k8s.io configured
+# customresourcedefinition.apiextensions.k8s.io/serviceimports.multicluster.x-k8s.io created
+# ...
+# deployment.apps/cloud-map-mcs-controller-manager created
 ```
-controllers.ServiceExport	updating Cloud Map service	{"serviceexport": "demo/demo-service", "namespace": "demo", "name": "demo-service"}
-cloudmap	                fetching a service	{"namespaceName": "demo", "serviceName": "demo-service"}
-cloudmap	                creating a new service	{"namespace": "demo", "name": "demo-service"}
+
+Stream the controller logs
+```shell
+kubectl logs -f -l control-plane=controller-manager -c manager -n cloud-map-mcs-system
 ```
 
 #### Run unit tests
 
 Use command below to run the unit test
+
 ```sh
 make test
 ```
 
 #### Cleanup
 
 Use the command below to clean all the generated files
+
 ```sh
 make clean
 ```
 
+Use the command below to remove the CRDs from the cluster
+
+```sh
+make uninstall
+```
+
 Use the command below to delete the cluster `my-cluster`
+
 ```sh
 kind delete cluster --name my-cluster
 ```
@@ -108,23 +179,17 @@ make docker-build docker-push IMG=<YOUR ACCOUNT ID>.dkr.ecr.<ECR REGION>.amazona
 
 #### Deployment
 
-You must specify AWS access credentials for the operator. You can do so via environment variables, or by creating them.
+You must setup the AWS access credentials. Also, set the AWS_REGION environment variable like `export AWS_REGION=us-west-2` 
 
-Any one of below three options will work:
 ```sh
 # With an IAM user.
 make deploy
-
-# Use an existing access key
-OPERATOR_AWS_ACCESS_KEY_ID=xxx OPERATOR_AWS_SECRET_KEY=yyy make deploy
-
-# Use an AWS profile
-OPERATOR_AWS_PROFILE=default make deploy
 ```
 
 #### Uninstallation
 
-To remove the operator from your cluster, run
+To remove the controller from your cluster, run
+
 ```sh
 make undeploy
 ```

Makefile

@@ -6,6 +6,8 @@ PKG:=github.com/aws/aws-cloud-map-mcs-controller-for-k8s/pkg/version
 IMG ?= controller:latest
 # Produce CRDs that work back to Kubernetes 1.11 (no version conversion)
 CRD_OPTIONS ?= "crd:trivialVersions=true,preserveUnknownFields=false"
+# AWS Region
+AWS_REGION ?= us-east-1
 
 # Get the currently used golang install path (in GOPATH/bin, unless GOBIN is set)
 ifeq (,$(shell go env GOBIN))
@@ -67,10 +69,10 @@ run: manifests generate generate-mocks fmt vet ## Run a controller from your hos
 	go run ./main.go
 
 docker-build: test ## Build docker image with the manager.
-	docker build -t ${IMG} .
+	docker build $(ARGS) -t ${IMG} .
 
 docker-push: ## Push docker image with the manager.
-	docker push ${IMG}
+	docker push $(ARGS) ${IMG}
 
 clean:
 	rm -rf $(MOCKS_DESTINATION) bin/ testbin/ cover.out
@@ -85,7 +87,7 @@ uninstall: manifests kustomize ## Uninstall CRDs from the K8s cluster specified
 
 deploy: manifests kustomize ## Deploy controller to the K8s cluster specified in ~/.kube/config.
 	cd config/manager && $(KUSTOMIZE) edit set image controller=${IMG}
-	$(KUSTOMIZE) build config/default | kubectl apply -f -
+	AWS_REGION=${AWS_REGION} $(KUSTOMIZE) build config/default | kubectl apply -f -
 
 undeploy: ## Undeploy controller from the K8s cluster specified in ~/.kube/config.
 	$(KUSTOMIZE) build config/default | kubectl delete -f -

README.md

@@ -21,6 +21,8 @@ AWS Cloud Map multi-cluster service discovery for Kubernetes (K8s) is a controll
 
 First, install the controller with latest release on at least 2 AWS EKS clusters. Nodes must have sufficient IAM permissions to perform CloudMap operations.
 
+> **_NOTE:_** AWS region environment variable should be set like `export AWS_REGION=us-west-2`
+
 ```sh
 kubectl apply -k "github.com/aws/aws-cloud-map-mcs-controller-for-k8s/config/controller_install_release"
 ```
@@ -51,9 +53,9 @@ metadata:
 *See the `samples` directory for a set of example yaml files to set up a service and export it. To apply the sample files run*
 ```sh
 kubectl create namespace demo
-kubectl apply -f https://raw.githubusercontent.com/aws/aws-cloud-map-mcs-controller-for-k8s/main/samples/demo-deployment.yaml
-kubectl apply -f https://raw.githubusercontent.com/aws/aws-cloud-map-mcs-controller-for-k8s/main/samples/demo-service.yaml
-kubectl apply -f https://raw.githubusercontent.com/aws/aws-cloud-map-mcs-controller-for-k8s/main/samples/demo-export.yaml
+kubectl apply -f https://raw.githubusercontent.com/aws/aws-cloud-map-mcs-controller-for-k8s/main/samples/example-deployment.yaml
+kubectl apply -f https://raw.githubusercontent.com/aws/aws-cloud-map-mcs-controller-for-k8s/main/samples/example-service.yaml
+kubectl apply -f https://raw.githubusercontent.com/aws/aws-cloud-map-mcs-controller-for-k8s/main/samples/example-serviceexport.yaml
 ```
 
 ### Import services
@@ -67,6 +69,8 @@ kubectl get ServiceImport -A
 
 AWS Cloud Map MCS Controller for K8s adheres to the [SemVer](https://semver.org/) specification. Each release updates the major version tag (eg. `vX`), a major/minor version tag (eg. `vX.Y`) and a major/minor/patch version tag (eg. `vX.Y.Z`). To see a full list of all releases, refer to our [Github releases page](https://github.com/aws/aws-cloud-map-mcs-controller-for-k8s/releases).
 
+> **_NOTE:_** AWS region environment variable should be set like `export AWS_REGION=us-west-2`
+
 To install from a release run
 ```sh
 kubectl apply -k "github.com/aws/aws-cloud-map-mcs-controller-for-k8s/config/controller_install_release[?ref=*git version tag*]"

config/controller_install_latest/kustomization.yaml

@@ -1,4 +1,4 @@
-bases:
+resources:
 - ../default
 
 images:

config/controller_install_release/kustomization.yaml

@@ -1,4 +1,4 @@
-bases:
+resources:
 - ../default
 
 images:

config/default/kustomization.yaml

@@ -12,7 +12,7 @@ namePrefix: cloud-map-mcs-
 #commonLabels:
 #  someName: someValue
 
-bases:
+resources:
 - ../crd
 - ../rbac
 - ../manager

config/manager/aws.properties

@@ -0,0 +1 @@
+AWS_REGION

config/manager/kustomization.yaml

@@ -8,8 +8,10 @@ configMapGenerator:
 - files:
   - controller_manager_config.yaml
   name: manager-config
-apiVersion: kustomize.config.k8s.io/v1beta1
-kind: Kustomization
+- envs:
+  - aws.properties
+  name: aws-config
+
 images:
 - name: controller
   newName: controller

config/manager/manager.yaml

@@ -52,5 +52,11 @@ spec:
           requests:
             cpu: 100m
             memory: 20Mi
+        env:
+          - name: AWS_REGION
+            valueFrom:
+              configMapKeyRef:
+                name: aws-config
+                key: AWS_REGION
       serviceAccountName: controller-manager
       terminationGracePeriodSeconds: 10

main.go

@@ -86,27 +86,30 @@ func main() {
 	}
 
 	// TODO: configure session
+	awsRegion := os.Getenv("AWS_REGION")
+	setupLog.Info("configuring AWS session", "AWS_REGION", awsRegion)
 	awsCfg, err := config.LoadDefaultConfig(context.TODO(),
-		config.WithRegion(os.Getenv("AWS_REGION")),
+		config.WithRegion(awsRegion),
 	)
 	if err != nil {
-		setupLog.Error(err, "unable to configure AWS session")
+		setupLog.Error(err, "unable to configure AWS session", "AWS_REGION", awsRegion)
 		os.Exit(1)
 	}
 
+	serviceDiscoveryClient := cloudmap.NewServiceDiscoveryClient(&awsCfg)
 	if err = (&controllers.ServiceExportReconciler{
 		Client:   mgr.GetClient(),
 		Log:      ctrl.Log.WithName("controllers").WithName("ServiceExport"),
 		Scheme:   mgr.GetScheme(),
-		Cloudmap: cloudmap.NewServiceDiscoveryClient(&awsCfg),
+		Cloudmap: serviceDiscoveryClient,
 	}).SetupWithManager(mgr); err != nil {
 		setupLog.Error(err, "unable to create controller", "controller", "ServiceExport")
 		os.Exit(1)
 	}
 
 	cloudMapReconciler := &controllers.CloudMapReconciler{
 		Client:   mgr.GetClient(),
-		Cloudmap: cloudmap.NewServiceDiscoveryClient(&awsCfg),
+		Cloudmap: serviceDiscoveryClient,
 		Logger:   ctrl.Log.WithName("controllers").WithName("CloudMap"),
 	}