Skip to content

Commit 1e653eb

Browse files
yue9944882yue9944882
committed
doc: re-writing a kubernetes replicaset-controller in java
1 parent 351ffa1 commit 1e653eb

File tree

2 files changed

+216
-1
lines changed

2 files changed

+216
-1
lines changed

‎README.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -113,7 +113,7 @@ We prepared a few examples for common use-cases which are shown below:
113113
- ([6.0.0+](https://github.com/kubernetes-client/java/tree/client-java-parent-6.0.0)) [LeaderElectionExample](https://github.com/kubernetes-client/java/blob/master/examples/src/main/java/io/kubernetes/client/examples/LeaderElectionExample.java):
114114
Leader election utilities to help implement HA controllers.
115115
- ([9.0.0+](https://github.com/kubernetes-client/java/tree/client-java-parent-9.0.0)) [SpringIntegrationControllerExample](https://github.com/kubernetes-client/java/blob/master/examples/src/main/java/io/kubernetes/client/examples/SpringControllerExample.java):
116-
Building a kubernetes controller based on spring framework's bean injection.
116+
Building a kubernetes controller based on spring framework's bean injection, see additional documentation [here](./docs/java-controller-tutorial-rewrite-rs-controller.md).
117117
- ([9.0.0+](https://github.com/kubernetes-client/java/tree/client-java-parent-9.0.0)) [GenericKubernetesClientExample](https://github.com/kubernetes-client/java/blob/master/examples/src/main/java/io/kubernetes/client/examples/GenericClientExample.java):
118118
Construct a generic client interface for any kubernetes types, including CRDs.
119119

‎docs/java-controller-tutorial-rewrite-rs-controller.md

Lines changed: 215 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,215 @@
1+
# Re-Write a Kubernetes Replicas-Controller in Java
2+
3+
__TL;DR__: This tutorial will show you how to re-write a replicaset-controller
4+
(kube-controller-manager) in Java. This tutorial requires you to have a basic knowledge
5+
of spring-boot framework and maven commands and you're will be able write your own custom
6+
Java kubernetes controller similarly after reading this document. The complete example project
7+
is available at [yue9944882/replicaset-controller](https://github.com/yue9944882/replicaset-controller)
8+
and a working image example is also at [ghcr.io/yue9944882/java-replicaset-controller](https://github.com/users/yue9944882/packages/container/package/java-replicaset-controller).
9+
10+
11+
### Setup Environment
12+
13+
Please check that you have the following required development toolings installed in your local
14+
environment:
15+
16+
- __JDK 8+__: Necessary Java development environment.
17+
- __Maven__: Greater than 3.0.0+.
18+
- __KinD__: (Optional) For verifying the controller in a real kubernetes cluster. https://kind.sigs.k8s.io/docs/user/quick-start/
19+
20+
21+
### Writing Project
22+
23+
24+
#### Necessary Plugins for pom.xml
25+
26+
Creating a new repo following the standard maven project structure, and ensure the following
27+
plugins are present in the [pom.xml](https://github.com/yue9944882/replicaset-controller/blob/master/pom.xml):
28+
29+
- __Maven Compiler Plugin__: Setting language level to 8.
30+
```xml
31+
<plugin>
32+
<groupId>org.apache.maven.plugins</groupId>
33+
<artifactId>maven-compiler-plugin</artifactId>
34+
<configuration>
35+
<source>8</source>
36+
<target>8</target>
37+
</configuration>
38+
</plugin>
39+
```
40+
- __Spring Boot Maven Plugin__: For building thinner docker images following the standard spring-boot plugins:
41+
```xml
42+
<plugin>
43+
<groupId>org.springframework.boot</groupId>
44+
<artifactId>spring-boot-maven-plugin</artifactId>
45+
<version>2.3.4.RELEASE</version>
46+
</plugin>
47+
```
48+
49+
#### Adding a Main Class
50+
51+
So that the project can be packaged as a executable jar. See the example class [here](https://github.com/yue9944882/replicaset-controller/blob/master/src/main/java/com/github/yue9944882/kubernetes/Application.java).
52+
53+
#### Loading Post-Processor Spring Beans
54+
55+
Adding the following annotation to whatever Java class under spring context, so that processors can be activated.
56+
57+
```java
58+
@ComponentScan("io.kubernetes.client.spring.extended.controller")
59+
```
60+
61+
Corresponding example is available [here](https://github.com/yue9944882/replicaset-controller/blob/master/src/main/java/com/github/yue9944882/kubernetes/config/ControllerConfiguration.java#L20).
62+
63+
__NOTE__: In the future releases (already landed on master, will release after 11.0.0), you will be able to activate the processors by configuration-
64+
beans. To previewing the feature, you can take a glance at the test codes for [KubernetesReconcilerConfigurer](https://github.com/kubernetes-client/java/blob/351ffa13d49cb76445788d78bf83e03a5edf139a/spring/src/test/java/io/kubernetes/client/spring/extended/controller/KubernetesReconcilerCreatorTest.java#L75-L79) and
65+
[KubernetesInformerConfigurer](https://github.com/kubernetes-client/java/blob/351ffa13d49cb76445788d78bf83e03a5edf139a/spring/src/test/java/io/kubernetes/client/spring/extended/controller/KubernetesInformerCreatorTest.java#L73-L76).
66+
67+
#### Declaring Your Informer Factory
68+
69+
You're supposed to create a new class (can be a inner-class) extending `io.kubernetes.client.informer.SharedInformerFactory`,
70+
and then put `@KubernetesInformers` annotation on the new class to configure the informer-factory list-watching the specified
71+
kubernetes resources. See the example at [here](https://github.com/yue9944882/replicaset-controller/blob/c8dda02fe444d7154117b9bf0583557502694e1b/src/main/java/com/github/yue9944882/kubernetes/config/ControllerConfiguration.java#L46-L64).
72+
The following code is an example of registering a pod-informer and replicaset-informer to the informer-factory.
73+
74+
```java
75+
@KubernetesInformers({
76+
@KubernetesInformer( // Adding a pod-informer to the factory for list-watching pod resources
77+
apiTypeClass = V1Pod.class,
78+
apiListTypeClass = V1PodList.class,
79+
groupVersionResource =
80+
@GroupVersionResource(
81+
apiGroup = "",
82+
apiVersion = "v1",
83+
resourcePlural = "pods")),
84+
@KubernetesInformer( // Adding a replicaset-informer to the factory for list-watching replicaset resources
85+
apiTypeClass = V1ReplicaSet.class,
86+
apiListTypeClass = V1ReplicaSetList.class,
87+
groupVersionResource =
88+
@GroupVersionResource(
89+
apiGroup = "apps",
90+
apiVersion = "v1",
91+
resourcePlural = "replicasets")),
92+
})
93+
class ControllerSharedInformerFactory extends SharedInformerFactory {
94+
}
95+
```
96+
97+
And don't forget to register `ControllerSharedInformerFactory` as a bean to the spring context. See [here](https://github.com/yue9944882/replicaset-controller/blob/c8dda02fe444d7154117b9bf0583557502694e1b/src/main/java/com/github/yue9944882/kubernetes/config/ControllerConfiguration.java#L28-L33).
98+
The registered informer-factory won't be running unless you explcitly calls `startAllRegisteredInformers` somewhere else,
99+
the method is the trigger to run the controller, so hold it carefully until you're ready :). In the example project, the
100+
informer-factory was started inside `ControllerManager#run`.
101+
102+
The [KubernetesInformerFactoryProcessor](https://github.com/kubernetes-client/java/blob/master/spring/src/main/java/io/kubernetes/client/spring/extended/controller/KubernetesInformerFactoryProcessor.java)
103+
will be parsing the `@KubernetesInformers` annotation on informer-factory class and then register `SharedInformer` and
104+
`Lister` beans for the kubernetes resource classes. You can easily acquire them by declaring them as parameters in the
105+
bean method. See this example source to see the per-resource informer/lister bean registration [here](https://github.com/yue9944882/replicaset-controller/blob/c8dda02fe444d7154117b9bf0583557502694e1b/src/main/java/com/github/yue9944882/kubernetes/config/ControllerConfiguration.java#L35-L43).
106+
107+
108+
#### Declaring Your Reconciler
109+
110+
Now the informer-factory's ready, it will keep receiving watching events from kubernetes cluster after started. And by
111+
this step you will be actually writing a controller subscribing events from the cluster. First of all, create a new class
112+
implementing the `Reconciler` interface and register the class as a bean in the spring context:
113+
114+
115+
```java
116+
public class ReplicaSetReconciler implements Reconciler {...}
117+
```
118+
119+
Then add a `@KubernetesReconciler` annotation on the reconciler class to explictly mark the class so that it can be
120+
processed by [KubernetesReconcilerConfigurer](https://github.com/kubernetes-client/java/blob/master/spring/src/main/java/io/kubernetes/client/spring/extended/controller/KubernetesReconcilerConfigurer.java).
121+
The processor will be wiring the reconciler class to the informer-factory together so that the event can be delivered to
122+
the reconciler. In the `@KubernetesReconciler` annotation, you're also requried to uniquely name the reconciler by setting
123+
`value` property and set `@KubernetesReconcilerWatches` property to set what kind of kubernetes resources the reconciler
124+
need to keep notified. Note that the `@KubernetesReconcilerWatches` configuration won't raise a new watch connection, all
125+
the watch connections are managed by informer-factory and they're multiplex'd. Simply put, it's justing adding an in-memory watcher
126+
to the watch connection. For more detail, take a look at the example code [here](https://github.com/yue9944882/replicaset-controller/blob/c8dda02fe444d7154117b9bf0583557502694e1b/src/main/java/com/github/yue9944882/kubernetes/ReplicaSetReconciler.java#L27-L40).
127+
128+
129+
#### Running the Reconcilers
130+
131+
Now both the informer-factory and the reconciler are set, the last step is to stitch them together by adding a starter
132+
(or runner) bean implemeting `InitializingBean` as is shown in the following approaches:
133+
134+
135+
- (Option 1) Run it immediately.
136+
```java
137+
@Component
138+
public class ControllerStarter implements InitializingBean {
139+
140+
@Resource
141+
private SharedInformerFactory sharedInformerFactory;
142+
143+
@Resource(name = "replicaset-reconciler")
144+
private Controller replicasetController;
145+
146+
@Override
147+
public void afterPropertiesSet() throws Exception {
148+
sharedInformerFactory.startAllRegisteredInformers();
149+
Executors.newSingleThreadExecutor().submit(replicasetController);
150+
}
151+
}
152+
```
153+
154+
- (Option 2) Pack the informer-factory and the reconciler into a controller-manager instance.
155+
```java
156+
@Component
157+
public class ControllerStarter implements InitializingBean {
158+
159+
@Resource
160+
private SharedInformerFactory sharedInformerFactory;
161+
162+
@Resource(name = "replicaset-reconciler")
163+
private Controller replicasetController;
164+
165+
@Override
166+
public void afterPropertiesSet() throws Exception {
167+
ControllerManager controllerManager = new ControllerManager(sharedInformerFactory, replicasetController);
168+
Executors.newSingleThreadExecutor().submit(controllerManager);
169+
}
170+
}
171+
```
172+
173+
- (Option 3) Pack the informer-factory and the reconciler into a leader-election HA controller-manager.
174+
```java
175+
@Component
176+
public class ControllerStarter implements InitializingBean {
177+
178+
@Resource
179+
private SharedInformerFactory sharedInformerFactory;
180+
181+
@Resource(name = "replicaset-reconciler")
182+
private Controller replicasetController;
183+
184+
@Override
185+
public void afterPropertiesSet() throws Exception {
186+
ControllerManager controllerManager = new ControllerManager(sharedInformerFactory, replicasetController);
187+
188+
String appNamespace = "default";
189+
String appName = "java-replicaset-controller";
190+
String lockHolderIdentityName = UUID.randomUUID().toString(); // Anything unique
191+
EndpointsLock lock = new EndpointsLock(appNamespace, appName, lockHolderIdentityName);
192+
193+
LeaderElectionConfig leaderElectionConfig =
194+
new LeaderElectionConfig(
195+
lock, Duration.ofMillis(10000), Duration.ofMillis(8000), Duration.ofMillis(2000));
196+
ExecutorService pool = Executors.newSingleThreadExecutor();
197+
try (LeaderElector leaderElector = new LeaderElector(leaderElectionConfig)) {
198+
leaderElector.run(
199+
() -> pool.submit(controllerManager),
200+
() -> pool.shutdown());
201+
}
202+
}
203+
}
204+
```
205+
206+
207+
#### *Optional*: Using spring-boot maven plugin to Build Image
208+
209+
Run the command to build a thinner image for your controller:
210+
211+
```bash
212+
mvn spring-boot:build-image
213+
```
214+
215+
For running the image in a real cluster, please following the instructions [here](https://github.com/yue9944882/replicaset-controller#run-this-project-in-a-new-kind-cluster).

0 commit comments

Comments
 (0)