Add support for Ready, Hostname and Nodename Endpoint attributes. And some other refactoring changes. (#210)

* Add support for Ready, Hostname and Nodename Endpoint attributes. And some other refactoring changes.

Changes:
* Add support for Ready, Hostname and Nodename Endpoint attributes.
* Refactor cluster properties changes - move to model package, simplify code.
* Add controller watches for Service event. Filter out events if cluster properties is not defined.
* Improve kind integration test - add curl test, use dnsutils pod for dig, use ngnix app for deployment

* Fix build command test

* Update the Leader resource name

Author: runakash

CONTRIBUTING.md

@@ -1,30 +1,31 @@
 # Contributing Guidelines
 
-* [Architecture Overview](#architecture-overview)
-* [Getting Started](#getting-started)
-    + [Local Setup](#local-setup)
-        - [Prerequisites](#prerequisites)
-        - [Cluster Setup](#cluster-setup)
-        - [Run the controller from outside the cluster](#run-the-controller-from-outside-the-cluster)
-        - [Build and deploy to the cluster](#build-and-deploy-to-the-cluster)
-        - [Run unit tests](#run-unit-tests)
-        - [Cleanup](#cleanup)
-    + [Deploying to a cluster](#deploying-to-a-cluster)
-* [Integration testing](#integration-testing)
-* [Build and push docker image to ECR](#build-and-push-docker-image-to-ecr)
-* [Reporting Bugs/Feature Requests](#reporting-bugs-feature-requests)
-* [Contributing via Pull Requests](#contributing-via-pull-requests)
-* [Finding contributions to work on](#finding-contributions-to-work-on)
-* [Code of Conduct](#code-of-conduct)
-* [Security issue notifications](#security-issue-notifications)
-* [Licensing](#licensing)
-
 Thank you for your interest in contributing to our project. Whether it's a bug report, new feature, correction, or additional
 documentation, we greatly value feedback and contributions from our community.
 
 Please read through this document before submitting any issues or pull requests to ensure we have all the necessary
 information to effectively respond to your bug report or contribution.
 
+<!-- TOC -->
+* [Contributing Guidelines](#contributing-guidelines)
+  * [Architecture Overview](#architecture-overview)
+  * [Getting Started](#getting-started)
+    * [Build and Unit Tests](#build-and-unit-tests)
+    * [Local Setup](#local-setup)
+      * [Prerequisites](#prerequisites)
+      * [Kind Cluster Setup](#kind-cluster-setup)
+      * [Run the controller from outside the cluster](#run-the-controller-from-outside-the-cluster)
+      * [Build and deploy controller into the cluster](#build-and-deploy-controller-into-the-cluster)
+    * [Local integration testing](#local-integration-testing)
+  * [Build and push docker image](#build-and-push-docker-image)
+  * [Reporting Bugs/Feature Requests](#reporting-bugsfeature-requests)
+  * [Contributing via Pull Requests](#contributing-via-pull-requests)
+  * [Finding contributions to work on](#finding-contributions-to-work-on)
+  * [Code of Conduct](#code-of-conduct)
+  * [Security issue notifications](#security-issue-notifications)
+  * [Licensing](#licensing)
+<!-- TOC -->
+
 ## Architecture Overview
 
 ![Architecture diagram](docs/architecture-overview.png?raw=true)
@@ -34,6 +35,23 @@ information to effectively respond to your bug report or contribution.
 
 ## Getting Started
 
+### Build and Unit Tests
+
+Use command below to run the unit test:
+```sh
+make test
+```
+
+Use command below to build:
+```sh
+make build
+```
+
+Use the command below to perform cleanup:
+```sh
+make clean
+```
+
 ### Local Setup
 
 #### Prerequisites
@@ -50,7 +68,7 @@ Note that this walk-through assumes throughout to operate in the `us-west-2` reg
 export AWS_REGION=us-west-2
 ```
 
-#### Cluster Setup
+#### Kind Cluster Setup
 
 Spin up a local Kubernetes cluster using `kind`:
 
@@ -92,7 +110,7 @@ kubectl apply -f samples/example-clusterproperty.yaml
 > ⚠ **Note:** If you are creating multiple clusters, ensure you create unique `id.k8s.io` identifiers for each cluster.
 
 
-To run the controller, run the following command. The controller runs in an infinite loop so open another terminal to create CRDs.
+To run the controller, run the following command. The controller runs in an infinite loop so open another terminal to create CRDs. (Ctrl+C to exit)
 ```sh
 make run 
 ```
@@ -114,7 +132,12 @@ cloudmap	                fetching a service	{"namespaceName": "example", "servic
 cloudmap	                creating a new service	{"namespace": "example", "name": "my-service"}
 ```
 
-#### Build and deploy to the cluster
+Use the command below to remove the CRDs from the cluster:
+```sh
+make uninstall
+```
+
+#### Build and deploy controller into the cluster
 
 Build local `controller` docker image:
 ```sh
@@ -136,7 +159,8 @@ kind load docker-image controller:local --name my-cluster
 Finally, create the controller resources in the cluster:
 ```sh
 make deploy IMG=controller:local AWS_REGION=us-west-2
-# customresourcedefinition.apiextensions.k8s.io/serviceexports.multicluster.x-k8s.io configured
+# customresourcedefinition.apiextensions.k8s.io/clusterproperties.about.k8s.io created
+# customresourcedefinition.apiextensions.k8s.io/serviceexports.multicluster.x-k8s.io created
 # customresourcedefinition.apiextensions.k8s.io/serviceimports.multicluster.x-k8s.io created
 # ...
 # deployment.apps/cloud-map-mcs-controller-manager created
@@ -152,38 +176,13 @@ To remove the controller from your cluster, run:
 make undeploy
 ```
 
-#### Run unit tests
-
-Use command below to run the unit test:
-```sh
-make test
-```
-
-#### Cleanup
-
-Use the command below to perform cleanup:
-```sh
-make clean
-```
-
-Use the command below to remove the CRDs from the cluster:
-```sh
-make uninstall
-```
-
 Use the command below to delete the cluster `my-cluster`:
 ```sh
 kind delete cluster --name my-cluster
 ```
 
-### Deploying to a cluster
-
-You must first push a Docker image containing the changes to a Docker repository like ECR, Github packages, or DockerHub. The repo is configured to use Github Actions to automatically publish the docker image upon push to `main` branch. The image URI will be `ghcr.io/[Your forked repo name here]` You can enable this for forked repos by enabling Github actions on your forked repo in the "Actions" tab of forked repo.
-
-If you are deploying to cluster using kustomize templates from the `config` directory, you will need to override the image URI away from `ghcr.io/aws/aws-cloud-map-mcs-controller-for-k8s` in order to use your own docker images.
-
+### Local integration testing
 
-## Local integration testing
 The end-to-end integration test suite can be run locally to validate controller core functionality. This will provision a local Kind cluster and build and run the AWS Cloud Map MCS Controller for K8s. The test will verify service endpoints sync with AWS Cloud Map. If successful, the suite will then de-provision the local test cluster and delete AWS Cloud Map namespace `aws-cloud-map-mcs-e2e` along with test service and service instance resources:
 ```sh
 make kind-integration-suite
@@ -194,10 +193,15 @@ If integration test suite fails for some reason, you can perform a cleanup:
 make kind-integration-cleanup
 ```
 
-## Build and push docker image to ECR
+## Build and push docker image
+
+You must first push a Docker image containing the changes to a Docker repository like ECR, Github packages, or DockerHub. The repo is configured to use Github Actions to automatically publish the docker image upon push to `main` branch. The image URI will be `ghcr.io/[Your forked repo name here]` You can enable this for forked repos by enabling Github actions on your forked repo in the "Actions" tab of forked repo.
+
+If you are deploying to cluster using kustomize templates from the `config` directory, you will need to override the image URI away from `ghcr.io/aws/aws-cloud-map-mcs-controller-for-k8s` in order to use your own docker images.
 
+To push the docker image into personal repo:
 ```sh
-make docker-build docker-push IMG=<YOUR ACCOUNT ID>.dkr.ecr.<ECR REGION>.amazonaws.com/<ECR REPOSITORY>
+make docker-build docker-push IMG=[Your personal repo]
 ```
 
 ## Reporting Bugs/Feature Requests

Makefile

@@ -1,3 +1,6 @@
+# Silence update recommendations for Ginkgo
+export ACK_GINKGO_DEPRECATIONS:=1.16.5
+
 GIT_COMMIT:=$(shell git describe --dirty --always)
 GIT_TAG:=$(shell git describe --dirty --always --tags)
 PKG:=github.com/aws/aws-cloud-map-mcs-controller-for-k8s/pkg/version

config/manager/controller_manager_config.yaml

@@ -8,4 +8,4 @@ webhook:
   port: 9443
 leaderElection:
   leaderElect: true
-  resourceName: db692913.x-k8s.io
+  resourceName: aws-cloud-map-mcs-controller-for-k8s-lock

integration/janitor/api.go

@@ -4,7 +4,6 @@ import (
 	"context"
 
 	"github.com/aws/aws-cloud-map-mcs-controller-for-k8s/pkg/cloudmap"
-	"github.com/aws/aws-cloud-map-mcs-controller-for-k8s/pkg/common"
 	"github.com/aws/aws-sdk-go-v2/aws"
 	sd "github.com/aws/aws-sdk-go-v2/service/servicediscovery"
 )
@@ -20,9 +19,9 @@ type serviceDiscoveryJanitorApi struct {
 	janitorFacade SdkJanitorFacade
 }
 
-func NewServiceDiscoveryJanitorApiFromConfig(cfg *aws.Config, clusterUtils common.ClusterUtils) ServiceDiscoveryJanitorApi {
+func NewServiceDiscoveryJanitorApiFromConfig(cfg *aws.Config) ServiceDiscoveryJanitorApi {
 	return &serviceDiscoveryJanitorApi{
-		ServiceDiscoveryApi: cloudmap.NewServiceDiscoveryApiFromConfig(cfg, clusterUtils),
+		ServiceDiscoveryApi: cloudmap.NewServiceDiscoveryApiFromConfig(cfg),
 		janitorFacade:       NewSdkJanitorFacadeFromConfig(cfg),
 	}
 }

integration/janitor/api_test.go

@@ -5,7 +5,6 @@ import (
 	"testing"
 
 	janitorMock "github.com/aws/aws-cloud-map-mcs-controller-for-k8s/mocks/integration/janitor"
-	"github.com/aws/aws-cloud-map-mcs-controller-for-k8s/pkg/common"
 	"github.com/aws/aws-cloud-map-mcs-controller-for-k8s/test"
 	"github.com/aws/aws-sdk-go-v2/aws"
 	sd "github.com/aws/aws-sdk-go-v2/service/servicediscovery"
@@ -14,7 +13,7 @@ import (
 )
 
 func TestNewServiceDiscoveryJanitorApiFromConfig(t *testing.T) {
-	assert.NotNil(t, NewServiceDiscoveryJanitorApiFromConfig(&aws.Config{}, common.ClusterUtils{}))
+	assert.NotNil(t, NewServiceDiscoveryJanitorApiFromConfig(&aws.Config{}))
 }
 
 func TestServiceDiscoveryJanitorApi_DeleteNamespace_HappyCase(t *testing.T) {

integration/janitor/janitor.go

@@ -6,7 +6,7 @@ import (
 	"os"
 
 	"github.com/aws/aws-cloud-map-mcs-controller-for-k8s/pkg/cloudmap"
-	"github.com/aws/aws-cloud-map-mcs-controller-for-k8s/pkg/common"
+	"github.com/aws/aws-cloud-map-mcs-controller-for-k8s/pkg/model"
 	"github.com/aws/aws-sdk-go-v2/aws"
 	"github.com/aws/aws-sdk-go-v2/config"
 )
@@ -18,8 +18,10 @@ type CloudMapJanitor interface {
 }
 
 type cloudMapJanitor struct {
-	sdApi ServiceDiscoveryJanitorApi
-	fail  func()
+	clusterId    string
+	clusterSetId string
+	sdApi        ServiceDiscoveryJanitorApi
+	fail         func()
 }
 
 // NewDefaultJanitor returns a new janitor object.
@@ -32,8 +34,10 @@ func NewDefaultJanitor(clusterId string, clusterSetId string) CloudMapJanitor {
 	}
 
 	return &cloudMapJanitor{
-		sdApi: NewServiceDiscoveryJanitorApiFromConfig(&awsCfg, common.NewClusterUtilsForTest(clusterId, clusterSetId)),
-		fail:  func() { os.Exit(1) },
+		clusterId:    clusterId,
+		clusterSetId: clusterSetId,
+		sdApi:        NewServiceDiscoveryJanitorApiFromConfig(&awsCfg),
+		fail:         func() { os.Exit(1) },
 	}
 }
 
@@ -73,7 +77,11 @@ func (j *cloudMapJanitor) Cleanup(ctx context.Context, nsName string) {
 }
 
 func (j *cloudMapJanitor) deregisterInstances(ctx context.Context, nsName string, svcName string, svcId string) {
-	insts, err := j.sdApi.DiscoverInstances(ctx, nsName, svcName)
+	queryParameters := map[string]string{
+		model.ClusterSetIdAttr: j.clusterSetId,
+	}
+
+	insts, err := j.sdApi.DiscoverInstances(ctx, nsName, svcName, &queryParameters)
 	j.checkOrFail(err,
 		fmt.Sprintf("service has %d instances to clean", len(insts)),
 		"could not list instances to cleanup")

integration/janitor/janitor_test.go

@@ -21,7 +21,7 @@ type testJanitor struct {
 }
 
 func TestNewDefaultJanitor(t *testing.T) {
-	assert.NotNil(t, NewDefaultJanitor(test.ClusterId1, test.ClusterSetId1))
+	assert.NotNil(t, NewDefaultJanitor(test.ClusterId1, test.ClusterSet))
 }
 
 func TestCleanupHappyCase(t *testing.T) {
@@ -32,7 +32,9 @@ func TestCleanupHappyCase(t *testing.T) {
 		Return(map[string]*model.Namespace{test.HttpNsName: test.GetTestHttpNamespace()}, nil)
 	tj.mockApi.EXPECT().GetServiceIdMap(context.TODO(), test.HttpNsId).
 		Return(map[string]string{test.SvcName: test.SvcId}, nil)
-	tj.mockApi.EXPECT().DiscoverInstances(context.TODO(), test.HttpNsName, test.SvcName).
+	tj.mockApi.EXPECT().DiscoverInstances(context.TODO(), test.HttpNsName, test.SvcName, &map[string]string{
+		model.ClusterSetIdAttr: test.ClusterSet,
+	}).
 		Return([]types.HttpInstanceSummary{{InstanceId: aws.String(test.EndptId1)}}, nil)
 
 	tj.mockApi.EXPECT().DeregisterInstance(context.TODO(), test.SvcId, test.EndptId1).
@@ -67,8 +69,10 @@ func getTestJanitor(t *testing.T) *testJanitor {
 	failed := false
 	return &testJanitor{
 		janitor: &cloudMapJanitor{
-			sdApi: api,
-			fail:  func() { failed = true },
+			clusterId:    test.ClusterId1,
+			clusterSetId: test.ClusterSet,
+			sdApi:        api,
+			fail:         func() { failed = true },
 		},
 		mockApi: api,
 		failed:  &failed,

integration/kind-test/configs/e2e-coredns-deployment.yaml renamed to integration/kind-test/configs/coredns-deployment.yaml

@@ -58,7 +58,7 @@ spec:
       - args:
         - -conf
         - /etc/coredns/Corefile
-        image: ghcr.io/aws/aws-cloud-map-mcs-controller-for-k8s/coredns-multicluster/coredns:v1.8.4
+        image: ghcr.io/aws/aws-cloud-map-mcs-controller-for-k8s/coredns-multicluster/coredns:v1.8.6
         imagePullPolicy: IfNotPresent
         livenessProbe:
           failureThreshold: 5
@@ -134,4 +134,4 @@ spec:
           - key: Corefile
             path: Corefile
           name: coredns
-        name: config-volume
+        name: config-volume

integration/kind-test/configs/dnsutils-pod.yaml

@@ -0,0 +1,14 @@
+apiVersion: v1
+kind: Pod
+metadata:
+  name: dnsutils
+  namespace: default
+spec:
+  containers:
+    - command:
+        - sleep
+        - "3600"
+      image: registry.k8s.io/e2e-test-images/jessie-dnsutils:1.3
+      name: dnsutils
+      imagePullPolicy: IfNotPresent
+  restartPolicy: Always