By default requeue the resource with 10 hour delay (#97)

Issue #, if available: aws-controllers-k8s/community#1355

Description of changes:
* implement ReSync using `requeue.NeededAfter` instead of upstream controlle-runime's `SyncPeriod`
* ACK controllers do not use cached go-client for reading objects from APIServer, hence upstream controller-runime ReSync was not working. Upstream controller-runtime ReSync only works for cached objects.
* As part of this change, whenever the `RequeueOnSuccess` duration for an ACK resource is 0 , requeue it with a 10 hour delay to trigger ReSync.
* I did not change the default value of `RequeueOnSuccessSeconds` from 0 to 10 hours because RequeueOnSuccessSeconds = 0 still means resource will not be requeued again to the developers. This 10 hour ReSync is an ACK runtime behavior and hence i only made change in ACK runtime. There will no change needed in existing ACK controllers. 

By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license.

Author: vijtrip2

pkg/runtime/reconciler.go

@@ -43,6 +43,10 @@ import (
 
 const (
 	backoffReadOneTimeout = 10 * time.Second
+	// The default duration to trigger the sync for an ACK resource after
+	// the successful reconciliation. This behavior for a resource can be
+	// overriden by RequeueOnSuccessSeconds configuration for that resource.
+	resyncPeriod = 10 * time.Hour
 )
 
 // reconciler describes a generic reconciler within ACK.
@@ -836,6 +840,25 @@ func (r *resourceReconciler) handleRequeues(
 					)
 					return latest, requeue.NeededAfter(nil, time.Duration(duration)*time.Second)
 				}
+				// Since RequeueOnSuccessSeconds <= 0, requeue the resource
+				// with "resyncPeriod" to perform periodic drift detection and
+				// sync the desired state.
+				//
+				// Upstream controller-runtime provides SyncPeriod functionality
+				// which flushes the go-client cache and triggers Sync for all
+				// the objects in cache every 10 hours by default.
+				//
+				// ACK controller use non-cached client to read objects
+				// from API Server, hence controller-runtime's SyncPeriod
+				// functionality does not work.
+				// https://github.com/aws-controllers-k8s/community/issues/1355
+				//
+				// ACK controllers use api-reader(non-cached client) to avoid
+				// reading stale copies of ACK resource that can cause
+				// duplicate resource creation when resource identifier is
+				// not present in stale copy of resource.
+				// https://github.com/aws-controllers-k8s/community/issues/894#issuecomment-911876354
+				return latest, requeue.NeededAfter(nil, resyncPeriod)
 			} else {
 				rlog.Debug(
 					"requeueing resource after finding resource synced condition false",