Rethink the Subroutine interface

[aaronschweig]

In our current implementation of the lifecycle package we defined the following interface for the Subroutines.

type Subroutine interface {
	Process(ctx context.Context, instance runtimeobject.RuntimeObject) (ctrl.Result, errors.OperatorError)
	Finalize(ctx context.Context, instance runtimeobject.RuntimeObject) (ctrl.Result, errors.OperatorError)
	GetName() string
	Finalizers(instance runtimeobject.RuntimeObject) []string
}

This ticket focuses especially on the return values for the Process and Finalize methods defined. Until now we were relying on the standard controller-runtime Result type to indicate what behavior we want after a subroutine has run - if it should requeue, cancel the run or do something specific that we want.

A few months ago ctrl decided to deprecate the Requeue bool flag in the result in favor of a more explicit approach directly using a limiter. While it might make sense on the level that plain controller-runtime is operating on, the scope of the higher level lifecycle package should not expose the necessity of thinking of proper rate limiting and requeueing intervals from a developer.

Unfourtunately as long as we stick to the types provided by ctrl we are also tied to their decisions on which fields are available to us.

Therefore we want to explore how a custom return type might look like, what helpers might make sense with a simple goal of removing that burden from the developer again.

In the same approach we can also have a look into the requirement of return a specific OperatorError type instead of relying on the built-in error interface golang provides. We still have the ability inside the lifecycle package to wrap any error that is received inside an operator error, but again it frees the developer of the burden of having to worry about complying with a specific type or structure.

Any other improvements can also be evaluated within the scope of this ticket to evolve the Subroutine interface to its next version