Pass conformance tests

[dlorenc]

https://coreos.com/kubernetes/docs/latest/conformance-tests.html

Or document known failures that we decide we can live with.

[dlorenc]

Got conformance tests running by following these steps:
https://github.com/kubernetes/kubernetes/blob/master/docs/devel/e2e-tests.md#conformance-tests

I had to hack e2e.go to allow the tests to run with a single node.

I also had to pass check_version_skew=false: https://github.com/kubernetes/kubernetes/blob/master/hack/e2e.go#L43

[dlorenc]

Update on conformance tests. They seem to require SSL to be configured (probably for service accounts).

I have a branch of localkube that configures the apiserver and controllermanager to listen securely, and a branch of minikube that generates certificates using easyrsa. Once we get localkube merged into minikube I can submit these fixes for real.

With this, the tests run and some of them even pass. I noticed a bunch of errors parsing the server version though, like this:

Apr 28 13:54:34.199: INFO: update-demo-nautilus-7ansz is running right image but validator function failed: Unable to parse server version "v1.2.1+$Format:%h$": Invalid character(s) found in build meta data "$Format:%h$"

It looks like this string comes from here: https://github.com/kubernetes/kubernetes/blob/master/pkg/version/base.go#L54

but in the vendored copy inside localkube. I hacked those to be more readable, and tests are running more successfully. It looks like this file is updated dynamically during the k8s build/release process. We'll need to do something similar in localkube.

Seeing this error now:
I0428 21:06:34.007282 5183 hairpin.go:51] Unable to find pair interface, setting up all interfaces: exec: "nsenter": executable file not found in $PATH

I'll try adding nsenter to the boot2docker iso and see if that resolves these errors.

[dlorenc]

Results here: https://gist.github.com/dlorenc/91720521e6722948bf580c5c5d36a0d1

[yvespp]

I tried the Configmap example with Redis on minikube and got this error:

  FirstSeen LastSeen    Count   From            SubobjectPath   Type        Reason      Message
  --------- --------    -----   ----            -------------   --------    ------      -------
  4m        4m      1   {default-scheduler }            Normal      Scheduled   Successfully assigned redis to minikubevm
  4m        5s      23  {kubelet minikubevm}            Warning     FailedMount Unable to mount volumes for pod "redis_default(4abd4bc5-0f0d-11e6-9bc2-46d4ae96d391)": executable file not found in $PATH
  4m        5s      23  {kubelet minikubevm}            Warning     FailedSync  Error syncing pod, skipping: executable file not found in $PATH

I guessed that it's looking for nsenter an installed it via docker run --rm -v /usr/local/bin:/target jpetazzo/nsenter and now it works!

I really just guessed and actually don't know that nsenter is needed...
I tested with commit 1b12968 on a Mac with Virtualbox.

[yvespp]

To pass
[Fail] [k8s.io] ConfigMap [It] should be consumable from pods in volume with mappings as non-root [Conformance]
localkube needs to be update to kubernetes 1.2.3 (currently 1.2.1) which fixes that: kubernetes/kubernetes#23793

[dlorenc]

Thanks for the help debugging! I wasn't actually able to observe an increase in test pass rates with nsenter, but I'll give it another try. We should probably include it in the .iso either way.

I'm working on getting everything updated to latest k8s. We'll probably wait until we merge localkube into this repo, but here's a rough proof of concept that updates to latest: redspread/localkube#59

[dlorenc]

Latest results (with #81): https://gist.github.com/dlorenc/9fbb967bc5799439a7e63ea979e72d6d

Many look SecurityContext related, I'll try another run with SecurityContextDeny removed from the APIServer admission control.

[dlorenc]

Much better after enabling SecurityContext:
https://gist.github.com/dlorenc/c16fef9e86f4191acffd02aa26958b3f

Down to 6 failures.

[luxas]

What commands are you using to run the conformance tests?
This should be documented, and if possible, added as a non-required test for every PR

[dlorenc]

It still requires a little hackery. I definitely plan on documenting it ASAP.