Shall we use Golang to manage the cluster?

[avamingli]

Description

As we are working on cherry-picking commits from the GPDB project, I came across the commit greenplum-db/gpdb-archive@0801058 here, which marks the beginning of using Golang as the cluster management tool instead of Python.

**The program flow:**
- `cli/init.go`: `initCmd()` Starting point for init command command `gp init cluster`. Performs basic flag checks, and config file validity checks. Forms cluster creation request and calls RPC service on Hub to create the cluster.
- `hub/make_cluster.go`: `MakeCluster()` Starting point for `MakeCluster` RPC to create the actual cluster
- `MakeCluster` calls for validating the host environment where multiple checks are done returning error/warning
- `MakeCluster` creates a coordinator segment followed by the start segment
- Register primaries to the coordinator and get content-id and dbid for the segments.
- Post registration, MakeCluster creates primary segments in parallel on all hosts
- Stop the coordinator to restart the cluster.
- Start the cluster using `gpstart`
- Create GPDB extensions
- Import collations
- Setup GPDB superuser password
- 

As far as I know, the GPDB's cluster-tool team has been working on this replacement for some time, and this commit represents the initial steps.

This commit is the last 170th commit—yes, we'r close to finish the most cherry-pick work.
However, GPDB was archived soon after this commit. We are uncertain about how much work is left and what actions to take if any bugs arise, as well as who will address those issues.

The core question is:
Are we going to use Golang as the cluster management tool, similar to the ongoing GPDB project?

Use case/motivation

No response

Related issues

No response

Are you willing to submit a PR?

• Yes I am willing to submit a PR!

[wkvolkman]

I would vote no. I've had to go in an modify the code a couple of times in an running cluster to get segment recovery to work. Being able to make required changes and not have to have a full source to rebuild from I think out weighs any benefit. Which comes to the question of what was the benefit of switching to go? This process is not something that really gets used in day to day work.

[yihong0618]

[hyongtao-code]

HAHAHA, laugh die.
If you want to start go install suddenly when starting gpdb, please feel free to cherry-pick these commits.

[tuhaihe]

Hey @yihong0618 , We should be careful with the words. Try to be kind, and do not insult or put down other participants or people by regions or countries per our CODE OF CONDUCT [1].

It would be better if you updated your comments to be more considerate. Thanks!

[1] https://www.apache.org/foundation/policies/conduct

[yihong0618]

I just to be honest, not sure when political correctness needs to get in the way of honesty

[yihong0618]

and if so I really do not see any difference between a company closed open source project from this for now, will not response any more

[yjhjstz]

since gpdb not opensource anymore , so golang dependent components also closed.

import (
	"context"
	"reflect"
	"strings"
	"testing"
	"github.com/greenplum-db/gp-common-go-libs/testhelper"
	"github.com/greenplum-db/gpdb/gp/agent"
	"github.com/greenplum-db/gpdb/gp/idl"
	"github.com/greenplum-db/gpdb/gp/testutils/exectest"
	"github.com/greenplum-db/gpdb/gp/utils"
)

that's key point not to pick.

[tuhaihe]

For the common-go-libs, we have one forked repo here: https://github.com/apache/cloudberry-go-libs. FYI.

[my-ship-it]

I do not recommend cherry pick go language commits. One reason is that greenplum has closed source, and we cannot obtain later fixes and updates unless we rely on ourselves to implement them.

[avamingli]

Given the feedback, we skip the commits which use Golang as cluster management tool for now.

[boogabee]

There was going to be a move to golang for the greenplum utilities, and there had been a significant amount of work put into that effort. The issue in front of us at the time was lots of conflict with the environment in the python tools and an aging code base. We could remove some of the dependency and conflicts we had and get a more energetic set of developers if we went with Go at Pivotal. A reimplementation in Python was heavily considered but moved off the board by consensus of the developers that were working on the management utilities at the time.

[avamingli]

Hi @boogabee

Thank you so much for sharing the backstory behind this decision—it truly brings back memories of our time at Pivotal. Those were such formative years, working alongside talented teams on meaningful technical challenges.

I believe the move to Go was the right choice at the time, and it's unfortunate that GPDB has since become archived. Personally, I remain fond of Go's elegance for such tools.

Sometimes I miss the energy of those Pivotal days: the collaborative spirit, the willingness to rethink architectures, and of course the brilliant people. Your insight reminds me how much we learned through those tough technical calls.