v0.3.0

This is the third release of the 2023 after nearly four months of active contribution from the community. This release packs a great deal of interesting features, fixes, and documentation updates. Let's highlight some of them below.

Thread Safe Test Context

When Parallel test features where enabled and integrated into the framework, it was enabled with room for running into race conditions. In this release we have included changes that helps us mitigate these issues when the tests are being run in parallel. This has been achieved by doing the following mechanism.

1. Populate the test context via the BeforeEachTest handler
2. Generate a child context from the parent context and provide that for individual tests/assessments
3. Discard the context from step 2 after the tests are done
4. Pass the original parent context to the AfterEachTest handler

As part of the said changes to mitigate the race condition, we also extended the Environment.Test and Environment.TestInParallel function to return a context.Context back after the execution of the tests have been completed. This also allows for better debugging of test context to analyze failures better with the added advantage that this is not breaking any of the existing contracts. However, if you are using a golint infra and have errcheck linter enabled, you will have to make a few changes to your code to account for the newly returned value from the Environment.Test and Environment.TestInParallel functions.

Limitations

1. The Finish phase will only be able to see the context from the Setup phase and not the one from the features themselves.

Related Issues

1. #216
2. #258

Related Pull Requests

1. #292

Continubutors

@phisco @maruina

FluxCD Integration

e2e-framework has had helm workflow integrated under the third_party support package for a while and now, we are adding FluxCD to that arsenal. With this integration, now you can integrate your e2e tests to run against a repository using fluxcd constructs. The supported features include the following.

1. Install FluxCD components onto your cluster
2. Create and Delete GitRepository resource
3. Create and Delete Kustomization resource

Related Pull Requests

1. #255
2. #283

Contributors

@matrus2

kwok Integration as a Cluster Provider

By default, e2e-framework has two ways to run tests. One could bring up a new cluster using kind and run the e2e tests against them using the framework or integrate against a real cluster and run tests against them. In this release,
kwok is being added to that list of supported cluster providers.

Since kwok provided a simulated kubelet instead of running a real kubelet, this can helm create a cluster with large number of nodes with very quick turn around time, reducing the turn around time to run the e2e tests. With this support, end users of the framework can now standup a simple kwok based cluster and run their tests against it. This also supports discovering the kowkctl binary from non standard location in order to simplify the integration.

Releated Issues

1. #214

Related Pull Requests

1. #239

Contributors

@reetasingh

Support for Custom Binary Paths

With this release, e2e-framework provides a mechanism where by, end users can provide a custom path from where the binaries such as kwokctl, helm or kind can be discovered. This helps consumers of the framework who wants to keep their binaries in non standard path outside of the $PATH and consume them for the integration needs.

Related Issues

1. #282

Related Pull Requests

1. #286

Contributors

@harshanarayana

Cluster Provider Interface for easy inclusion of additional Cluster providers

Until now, e2e-framework had a series of custom built hand crafted helper functions defined under envfuncs that enabled end users to perform operation such as instanciating a new cluster, destoring a cluster, collecting logs etc. However, with the interest in adding additional cluster providesr such as kwok, k3d and possibly many others in the future, it became critical that we define a set of common interfaces that can be implemented by the provider so that
we can avoid having duplication in the code provided under envfuncs package. What started as a discussion during the review of kwok provider integration turned into a full blown feature to enable better integration of providers in the
future.

As part of this work, e2e-framework not provides an interface named E2EClusterProvider which can be implemented by any cluster provider that we want to integrate into the framework and the existing envfuncs can be used as is for the new provider without having to add new code to it.

Providers can also implement an additonal optional interface E2EClusterProviderWithImageLoader which extends the E2EClusterProvider and adds two more additional supported feature around being able to load a container image into the cluster. Either as individual images or as a archieve. (kind currenrly supports this workflow)

Deprecation

As part of this implementation, the following envfuncs have been deprecated and should be replaced with the respective alternative in the future.

1. GetKindClusterFromContext can be replaced with GetClusterFromContext
2. CreateKindCluster can be replaced with CreateCluster
3. CreateKindClusterWithConfig can be replaced with CreateClusterWithConfig
4. DestroyKindCluster can be replaced with DestroyCluster
5. ExportKindClusterLogs can be replaced with ExportClusterLogs

Following section has a few example of what this replacement would look like. (Code snippets taken from examples)

Deprecated kind based Cluster setup

func TestMain(m *testing.M) {
    testenv = env.New()
	kindClusterName := envconf.RandomName("decoder", 16)
	testenv.Setup(
		envfuncs.CreateKindCluster(kindClusterName),
	)
	testenv.Finish(
		envfuncs.DestroyKindCluster(kindClusterName),
	)
    os.Exit(testenv.Run(m))
}

Suggested kind based Cluster Setup

func TestMain(m *testing.M) {
	testenv = env.New()
	kindClusterName := envconf.RandomName("decoder", 16)
	testenv.Setup(
		envfuncs.CreateCluster(kind.NewProvider(), kindClusterName),
	)
	testenv.Finish(
		envfuncs.DestroyCluster(kindClusterName),
	)
	os.Exit(testenv.Run(m))
}

Related Issues

1. #245

Related Pull Requests

1. #246

Contributors

@harshanarayana

Subresource Update support via the klient's resources package

klient's resources now supports a few new helper method that are aimed towards updating the sub resources.

1. func (r *Resources) UpdateSubresource(ctx context.Context, obj k8s.Object, subresource string, opts ...UpdateOption) error {}
2. func (r *Resources) UpdateStatus(ctx context.Context, obj k8s.Object, opts ...UpdateOption) error {}
3. func (r *Resources) PatchSubresource(ctx context.Context, obj k8s.Object, subresource string, patch k8s.Patch, opts ...PatchOption) error {}
4. func (r *Resources) PatchStatus(ctx context.Context, objs k8s.Object, patch k8s.Patch, opts ...PatchOption) error {}

These new helper functions make it very easy to interact with subresources as part of your e2e-framework based tests

Related Pull Requests

1. #249

Contributors

@wzshiming

Other notable changes

1. Updated kubernetes component dependency to v1.27.x by @harshanarayana in #244
2. Enabled ability to use --labels where the selectors have the same key (i.e --labels="feature=foo,feature=bar") by @embano1 in #248
3. Added support for DeploymentAvailable condition check helper by @ryankwilliams in #251
4. Enhanced example for using namespaces passed to the test via context by @maruina in #253
5. Improved error reporting for kind Cluster provided by @mmanciop in #256
6. Improved error result for helm command workflows by @bradbeam in #262
7. Improved conversion of resource handler options properly between metav1 and controller-runtime by @harshanarayana in #278
8. Enabled linters on examples along with actual code by @harshanarayana in #281
9. Added documentation about e2e-framework adopters by @vladimirvivien in #285
10. Added support for adding descriptions under table driven test definition model by @harshanarayana in #284
11. Removed unused random source seeding property by @matrus2 in #294
12. Enabled Github Issue and PR templates by @harshanarayana in #298
13. Enabled printing the stacktrace when the graceful teardown mode by @reetasingh in #299

New Contributors

• @phisco made their first contribution in #292
• @wzshiming made their first contribution in #249
• @ryankwilliams made their first contribution in #251
• @mmanciop made their first contribution in #256

Special Thanks

A special thanks to @cpanato, @ShwethaKumbla and @vladimirvivien for their incredible and continued support by providing valuable suggestions during the change reviews and their continued work in making sure the dependencies are upto date with the help of dependabot.

And special thanks to @harshanarayana for his invaluable help and contributions to the project and specially with preparing and getting this release out.

Full Changelog: v0.2.0...v0.3.0