charts/redpanda: refactor console integration

[chrisseto]

Refactor the redpanda console integration to first convert redpanda.RenderState into a StaticConfiguration. The StaticConfiguration is then transformed into a PartialConsoleValues which is then merged into the subchart values to perform the final rendering.

This is done to allow allow the charts' integration to be leveraged by the upcoming console controller.

Notable changes include:

• Using the bootstrap user for console's auth instead of the first listed SASL user.
• Generating a partial version of Console's config to add typing to the configuration code.
• The introduction of operator/pkg/ir to host non-user facing datastructures. These structs may evolve without impact to our public APIs and help resolve cyclic imports.

[chrisseto]

Looks like this PR may partially address redpanda-data/helm-charts#835

The console integration now accounts for mTLS but there's still an open issue with how the chart generates client certs.

[RafalKorepta]

Unfortunately gen package or rather go list does not work with console/backend go package.

For example tags are not valid go versions for module. Console tags are e.g. v3.2.1 where go requires backend/v3.2.1.

$ go list -m -versions -json  github.com/redpanda-data/console/[email protected]
go: github.com/redpanda-data/console/[email protected]: invalid version: unknown revision backend/v3.2.1

I forked the repo and pushed tag named backend/v3.2.1 -> https://github.com/RafalKorepta/console/releases/tag/backend%2Fv3.2.1

That doesn't work either because module path should container v3 as major version.

$ go list -m -versions -json  github.com/RafalKorepta/console/[email protected]
go: github.com/RafalKorepta/console/[email protected]: invalid version: module contains a go.mod file, so module path must match major version ("github.com/RafalKorepta/console/backend/v3")

Yet I need to find a way to correctly reference it in the following command;

$ go list -e -json=Name,ImportPath,Error,Dir,GoFiles,IgnoredGoFiles,IgnoredOtherFiles,CFiles,CgoFiles,CXXFiles,MFiles,HFiles,FFiles,SFiles,SwigFiles,SwigCXXFiles,SysoFiles,CompiledGoFiles,DepOnly,Imports,ImportMap,Export,Module -compiled=true -test=false -export=true -deps=true -find=false -pgo=off -- github.com/RafalKorepta/console/backend/[email protected]
{
	"ImportPath": "github.com/RafalKorepta/console/backend/[email protected]",
	"Error": {
		"ImportStack": [
			"github.com/RafalKorepta/console/backend/[email protected]"
		],
		"Pos": "",
		"Err": "can only use path@version syntax with 'go get' and 'go install' in module-aware mode"
	}
}

[chrisseto]

Hmm. I've not seen the errors in CI locally. I'll dig into those tomorrow.

[chrisseto]

I bumped up to v3.2.2 as v3.0.1 has some massive headaches due to using koanf/[email protected].

[chrisseto]

Ugh, looks like something in test containers is malfunctioning due to the upgrade...

[RafalKorepta]

Any update to the console go module would require to pin to exact git sha, right? The v3.2.2 go module reference will not work.

[chrisseto]

Yes, that's correct.

[github-actions]

💚 All backports created successfully

Status	Branch	Result
✅	release/v25.1.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation and see the Github Action logs for details