Create a Discovery Controller that adds service/instance information in config maps

[ryanjbaxter]

To eliminate the need for the application to be able to access the K8S API server we should provide a ServiceDiscovery implementation that does not rely on the use of the API.

We could do this by just leveraging the platform service discovery and then fetch the port via a lookup.

Another option would be to implement a service discovery server which exclusively talks to the k8s api server and have clients fetch service information from this server (kind of like a eureka server).

[spencergibb]

There's an open issue in commons for dns/srv discovery client

[ojhughes]

I don't think it is possible to implement a DiscoveryClient using DNS alone. You can't populate List getServices() because you cannot list all of the services available to the Pod via DNS.

We could extract the known services IDs from the Pod's environment variables but these do not get updated when a services changes, the Pod has to be redeployed.

I think the best way to do this (without the Pod contacting the API) would be to provide a Kubernetes controller that uses a shared Informer to capture all the events about services and act as a ServiceRegistry.

There is already an implementation for a KubernetesServiceRegistry in the project that is effectively a NoOp. This ServiceRegistry would be populated using the events stream from a Shared Informer

Projects using spring-cloud-kubernetes will then need to be configured with a property such as spring.cloud.kubernetes.registry.uri to access the ServiceRegistryservice. The registry controller will be implemented as a standalone Deployment in Kubernetes. This Deployment would also include the necessary RBAC bindings required so that Pods have access to the Service Registry and to allow the Service Registry access to the required k8s API.

The downside with this approach is that users of spring-cloud-kubernetes will need to install an extra component in the cluster. The installation could be as simple as a single command such as kubectl apply -f https://raw.githubusercontent.com/spring-cloud/spring-cloud-kubernetes/master/k8s/service-registry.yml. The service registry could also be made available as a Helm chart.
[#461]
cc @markfisher

[markfisher]

I think the best way to do this (without the Pod contacting the API) would be to provide a Kubernetes controller that uses a shared Informer to capture all the events about services and act as a ServiceRegistry.

I agree, but I would first explore the ecosystem for existing projects. I would also expect that implementing an Informer-driven ServiceRegistry would be simpler/lighter in Go.

[spencergibb]

After recent discussions with @TYsewyn, @dsyer, and @ryanjbaxter, I'd like to repurpose this to a controller that creates config map[s] with the information available from the k8s API. This could be extended for the multi-cluster use case when those APIs mature. Combined with spring-native this would be a good opportunity to target this project as a way to write k8s controllers in java.

Spring Cloud Commons already has a configuration-based DiscoveryClient that could work for those that want to continue to use that abstraction.

[spencergibb]

https://github.com/scratches/spring-controller (native, config maps) and https://github.com/Haybu/spring-cloud-k8s-eureka-controller (discovery) can be useful proofs of concept to take from.