Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
121 changes: 121 additions & 0 deletions stories/kube-apis/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
# Kubernetes-Style APIs

Welcome to yet another adventure through the Kubernetes codebase.

This time we will see how Kubernetes implements its APIs.
You know all those pods, deployments, services, and all that that you used all
the time?
Well, the plan for today is to explore all the API machinery (ha!) that makes
that possible.

Kubernetes-style APIs are at their core a convination of Go `struct`s and some
code generation to wrangle data into and out of said Go `struct`s.

We will start this trip into kubenernetes by eploring the code generating tools
that are currently used.

## Deep Copy Generator

We already hinted that the implementation of Kubernetes-style APIs relies on
handling data and passing it into and out of Go `struct`s.
If you have worked with multiple nested structs then you may have already ran
into the problem of moving data from one struct into another.
If the terms "shallow" and "deep" copy don't ring a bell then we recommend you
give https://flaviocopes.com/go-copying-structs/ a quick read.

In Kubernetes, we have a fancy tool called `deepcopy-gen` which does some code
generation that results in `DeepCopy` methods.
Like everything, hopefully an example will make it make more sense.

`deepcopy-gen` can be found in staging,
https://github.com/kubernetes/kubernetes/tree/master/staging/src/k8s.io/code-generator/cmd/deepcopy-gen.


For the sake of experimentation, we provide a small bash script,
[build.sh](./build.sh) to get the `deepcopy-gen` binary.
The build script will compile a binary for linux on amd64 but you can tweak it
to cross-compile something else.
This script is the simple version of
https://github.com/kubernetes-sigs/kind/blob/edecdfee8878ac00c7ae00485fca3f95d351e1ac/hack/go_container.sh
which is used in https://github.com/kubernetes-sigs/kind.


Anyway, a
```
./build.sh go build k8s.io/code-generator/cmd/deepcopy-gen
```

will get you a `deepcopy-gen` in the current working directory.


The entrypoint to `deepcopy-gen` is currently in
https://github.com/kubernetes/gengo/blob/master/examples/deepcopy-gen/main.go

There isn't a lot of documentation for `deepcopy-gen` but between
https://github.com/kubernetes/gengo/blob/master/examples/deepcopy-gen/main.go
and asking `deepcopy-gen` for help
```
./tools/deepcopy-gen --help
```

We should be able to get by.

A couple details that we will need to explicitly mention are

* `deepcopy-gen` relies on comments to know what types it should build deep
copy methods for (we will show an example here shortly).


```
./tools/deepcopy-gen -i ./apis/v1alpha1/ -O zz_generated.deepcopy --go-header-file boilerplate.go.txt
```

This will result in the creation of
[pkg/deepcopy_generated.go](./pkg/deepcopy_generated.go), which we already
included in here.

And to test our deep copiers we included [main.go](./main.go).
If you run it, you will see something like the following
```
$ go run main.go
cluster1: &pkg.Cluster{Name:"kube", Nodes:[]string{"1", "2", "3"}}
cluster2: &pkg.Cluster{Name:"kube", Nodes:[]string{"1", "2", "3"}}
cluster3: &pkg.Cluster{Name:"", Nodes:[]string(nil)}
cluster3: &pkg.Cluster{Name:"kube", Nodes:[]string{"1", "2", "3"}}
cluster1 and cluster3 the same: false
```

and if you look at the code,
```go
c1 := &api.Cluster{Name: "kube", Nodes: []string{"1", "2", "3"}}
...

c2 := c1.DeepCopy()
...

c3 := &api.Cluster{}
c2.DeepCopyInto(c3)
...

fmt.Printf("cluster1 and cluster3 the same: %v\n", c1 == c3)
```

we essentially create 1 `Cluster` object and then we use the generated
`DeepCopy()` and `DeepCopyInto()` methods to create copies.

The very last line compares if the first and last "cluster" objects are the
same and we indeed find they aren't - which is good, because we want deep
copies (we want the data alone).

## Conversion Generator

```
$ ./tools/conversion-gen -i ./apis/v1alpha1/ -O zz_generated.conversion --go-header-file boilerplate.go.txt
E0718 21:08:28.977033 25990 conversion.go:755] Warning: could not find nor generate a final Conversion function for github.com/contributing-to-kubernetes/gnosis/stories/kube-apis/apis/v1alpha2.Cluster -> github.com/contributing-to-kubernetes/gnosis/stories/kube-apis/apis/v1alpha1.Cluster
E0718 21:08:28.977988 25990 conversion.go:756] the following fields need manual conversion:
E0718 21:08:28.977996 25990 conversion.go:758] - Provider
```

The `zz_generated.conversion.go` needs a wrapper to fully convert to v1alpha2
because there are fields that exist in v1alpha2 that don't exist in v1alpha1
and have to be set manually.
6 changes: 6 additions & 0 deletions stories/kube-apis/apis/v1alpha1/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
// This package implements a sample apiVersion of the `Cluster` Config
// along with some common abstractions.
//
// +k8s:deepcopy-gen=package
// +k8s:conversion-gen=github.com/contributing-to-kubernetes/gnosis/stories/kube-apis/apis/v1alpha2
package v1alpha1
6 changes: 6 additions & 0 deletions stories/kube-apis/apis/v1alpha1/types.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
package v1alpha1

type Cluster struct {
Name string
Nodes []string
}
58 changes: 58 additions & 0 deletions stories/kube-apis/apis/v1alpha1/zz_generated.conversion.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

33 changes: 33 additions & 0 deletions stories/kube-apis/apis/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

5 changes: 5 additions & 0 deletions stories/kube-apis/apis/v1alpha2/doc.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
// This package implements a sample apiVersion of the `Cluster` Config
// along with some common abstractions.
//
// +k8s:deepcopy-gen=package
package v1alpha2
7 changes: 7 additions & 0 deletions stories/kube-apis/apis/v1alpha2/types.go
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
package v1alpha2

type Cluster struct {
Name string
Nodes []string
Provider string
}
33 changes: 33 additions & 0 deletions stories/kube-apis/apis/v1alpha2/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

6 changes: 6 additions & 0 deletions stories/kube-apis/boilerplate.go.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
/*
This is required for deepcopy-gen to work.

This oilerplate will make a lot more sense in the context of Kubernetes, where
all the source code must have a copyright message at the top.
*/
18 changes: 18 additions & 0 deletions stories/kube-apis/build.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
#!/bin/bash

# Source directory, will be mounted to /src, defaults to ./tools.
SOURCE_DIR="${SOURCE_DIR:-$(pwd -P)/tools}"


docker run --rm \
`# run as the user/group running this script` \
--user "$(id -u):$(id -g)" \
`# mount golang cache - required for 'go build'ing` \
-v "${CACHE_VOLUME}:/go" -e XDG_CACHE_HOME=/go/cache \
`# mount source directory` \
-v "${SOURCE_DIR}:/src" \
`# set environment variables to allow for cross-compilation (if desired)` \
-e CGO_ENABLED=0 -e GOOS=linux -e GOARCH=amd64 \
`# set working directory to the source directory` \
-w "/src" \
golang:1.14 "$@"
5 changes: 5 additions & 0 deletions stories/kube-apis/go.mod
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
module github.com/contributing-to-kubernetes/gnosis/stories/kube-apis

go 1.14

require k8s.io/apimachinery v0.18.6
Loading