Skip to content

feat: Add Tenant and Warehouse CRD. #2

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
hantmac merged 9 commits into from
Dec 13, 2024
Merged

feat: Add Tenant and Warehouse CRD. #2

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
9 commits
Select commit Hold shift + click to select a range
b1b633d
feat: fill in Tenant CRD & add comments and annotations.
Electronic-Waste Dec 11, 2024
f4be058
chore(tenant): add more annotations to tenant crd.
Electronic-Waste Dec 12, 2024
1f182be
chore(warehouse): add fields and annotations to Warehouse CRD.
Electronic-Waste Dec 12, 2024
4de99a1
fix(lint): fix lint error.
Electronic-Waste Dec 12, 2024
0afffcd
fix: go mod tidy.
Electronic-Waste Dec 12, 2024
bbabe7c
chore: generate crd yaml files.
Electronic-Waste Dec 12, 2024
4bb72e2
chore: generate deepcopy funcs.
Electronic-Waste Dec 12, 2024
cfe3498
fix: remove CI tests temporarily.
Electronic-Waste Dec 12, 2024
4c259eb
fix: add secret ref for AuthString & update some annotations.
Electronic-Waste Dec 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
35 changes: 0 additions & 35 deletions .github/workflows/test-e2e.yml
View file
Open in desktop

This file was deleted.

23 changes: 0 additions & 23 deletions .github/workflows/test.yml
View file
Open in desktop

This file was deleted.

285 changes: 280 additions & 5 deletions config/crd/bases/databendlabs.io_tenants.yaml
View file
Open in desktop

Large diffs are not rendered by default.

332 changes: 328 additions & 4 deletions config/crd/bases/databendlabs.io_warehouses.yaml
View file
Open in desktop

Large diffs are not rendered by default.

2 changes: 1 addition & 1 deletion go.mod
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,7 @@ go 1.22.0
require (
github.com/onsi/ginkgo/v2 v2.19.0
github.com/onsi/gomega v1.33.1
k8s.io/api v0.31.0
k8s.io/apimachinery v0.31.0
k8s.io/client-go v0.31.0
sigs.k8s.io/controller-runtime v0.19.1
Expand Down Expand Up @@ -84,7 +85,6 @@ require (
gopkg.in/inf.v0 v0.9.1 // indirect
gopkg.in/yaml.v2 v2.4.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
k8s.io/api v0.31.0 // indirect
k8s.io/apiextensions-apiserver v0.31.0 // indirect
k8s.io/apiserver v0.31.0 // indirect
k8s.io/component-base v0.31.0 // indirect
Expand Down
120 changes: 114 additions & 6 deletions pkg/api/v1alpha1/tenant_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,29 +17,137 @@ limitations under the License.
package v1alpha1

import (
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN!
// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized.

type UserAuthType string

const (
MD5 UserAuthType = "md5"
NoPassword UserAuthType = "no_password"
)

type Storage struct {
// Specification of S3 storage.
S3 *S3Storage `json:"s3,omitempty"`
}

type S3Storage struct {
// Authentication configuration of S3 storage.
S3Auth `json:",inline"`

// Whether to allow insecure connections to S3 storage.
// If set to true, users can establish HTTP connections to S3 storage.
// Otherwise, only HTTPS connections are allowed. Default to true.
// +kubebuilder:default=true
AllowInsecure bool `json:"allowInsecure,omitempty"`

// Root path of S3.
RootPath string `json:"rootPath,omitempty"`

// Name of S3 bucket.
BucketName string `json:"bucketName,omitempty"`

// Region of S3 storage.
Region string `json:"region,omitempty"`

// Endpoint of S3 storage.
Copy link

@flaneur2020 flaneur2020 Dec 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

add an optional for the endpoint? when you're using s3, endpoint is not an required parameter in most of the time.

Copy link
Contributor Author

@Electronic-Waste Electronic-Waste Dec 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's optional now, since I added omitempty tag after the field.

Endpoint string `json:"endpoint,omitempty"`
}

type S3Auth struct {
// Secret Access Key of S3 storage.
SecretKey string `json:"secretKey,omitempty"`

// Access Key ID of S3 storage.
AccessKey string `json:"accessKey,omitempty"`

// Reference to the secret with SerectKey and AccessKey to S3 storage.
// Secret can be created in any namespace.
SecretRef *corev1.ObjectReference `json:"secretRef,omitempty"`
}

type MetaConfig struct {
// Authentication configurations to connect to Meta cluster.
MetaAuth `json:",inline"`

// Exposed endpoints of Meta cluster (must list all pod endpoints in the Meta cluster).
Endpoints []string `json:"endpoints,omitempty"`

// Timeout seconds of connections to Meta cluster.
// +kubebuilder:default=3
TimeoutInSeconds int `json:"timeoutInSecond,omitempty"`

// Interval for warehouse to sync data from Meta cluster.
// +kubebuilder:default=60
AutoSyncInterval int `json:"autoSyncInterval,omitempty"`
}

type MetaAuth struct {
// User of Meta cluster.
User string `json:"user,omitempty"`

// Password of Meta cluster.
Password string `json:"password,omitempty"`

// Reference to the secret with User and Password to Meta cluster.
// Secret can be created in any namespace.
PasswordSecretRef *corev1.ObjectReference `json:"passwordSecretRef,omitempty"`
}

type User struct {
// Name of warehouse user.
Name string `json:"name,omitempty"`

// Authentication type of warehouse password.
// Currently we support: md5, no_password.
// +kubebuilder:default="no_password"
AuthType UserAuthType `json:"authType,omitempty"`

// Password encrypted with AuthType.
AuthString string `json:"authString,omitempty"`
Comment on lines +111 to +112
Copy link
Contributor Author

@Electronic-Waste Electronic-Waste Dec 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was wondering, whether we need to add a field named AuthStringRef to reference Secret like what we do in S3 and meta configurations.

Copy link

@flaneur2020 flaneur2020 Dec 12, 2024 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

i feel having a reference to secret is nice, might be like support both of styles:

authString:
  value: "a md5 hash"

authString:
  secretRef:
    name: "blah"
    namespace: "xxx"

Copy link
Contributor Author

@Electronic-Waste Electronic-Waste Dec 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think so. What about maintaining consistency with S3 and Meta configs? I think it would be better if we add a new field named AuthStringRef. WDYT👀？

Copy link

@flaneur2020 flaneur2020 Dec 12, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What about maintaining consistency with S3 and Meta configs? I think it would be better if we add a new field named AuthStringRef. WDYT👀？

looks good to me 👍


// Reference to the secret with AuthString of user.
// Secret can be created in any namespace.
AuthStringSecretRef *corev1.ObjectReference `json:"authStringSecretRef,omitempty"`
}

// TenantSpec defines the desired state of Tenant.
type TenantSpec struct {
// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
// Important: Run "make" to regenerate code after modifying this file
// Object storage specifications. Currently we only support S3.
Storage `json:",inline"`

// Configurations to open connections to a Meta cluster.
Meta MetaConfig `json:"meta,omitempty"`

// Foo is an example field of Tenant. Edit tenant_types.go to remove/update
Foo string `json:"foo,omitempty"`
// Built-in users in the warehouse created by this tenant.
// If not set, we'll create "admin" user with password "admin".
// +listType=map
// +listMapKey=name
BuiltinUsers []User `json:"builtinUsers,omitempty"`
}

// TenantStatus defines the observed state of Tenant.
type TenantStatus struct {
// INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
// Important: Run "make" to regenerate code after modifying this file
// Conditions for the Tenant.
// +listType=map
// +listMapKey=type
// +patchStrategy=merge
// +patchMergeKey=type
Conditions []metav1.Condition `json:"conditions,omitempty" patchStrategy:"merge" patchMergeKey:"type"`
}

// +kubebuilder:object:root=true
// +kubebuilder:subresource:status
// +kubebuilder:printcolumn:name="State",type=string,JSONPath=`.status.conditions[-1:].type`
// +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp`
// +genclient
// +k8s:openapi-gen=true
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// Tenant is the Schema for the tenants API.
type Tenant struct {
Expand Down
139 changes: 133 additions & 6 deletions pkg/api/v1alpha1/warehouse_types.go
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,29 +17,156 @@ limitations under the License.
package v1alpha1

import (
corev1 "k8s.io/api/core/v1"
metav1 "k8s.io/apimachinery/pkg/apis/meta/v1"
)

// EDIT THIS FILE! THIS IS SCAFFOLDING FOR YOU TO OWN!
// NOTE: json tags are required. Any new fields you add must have json tags for the fields to be serialized.

type DiskCacheSpec struct {
// Whether to enable cache in disk.
// +kubebuilder:default=false
Enabled bool `json:"enabled,omitempty"`

// Max bytes of cache in disk.
MaxBytes int `json:"size,omitempty"`

// Path to cache directory in disk.
// If not set, default to /var/lib/databend/cache.
Path string `json:"path,omitempty"`

// Provide storage class to allocate disk cache automatically.
StorageClass string `json:"storageClass,omitempty"`
}

type LogSpec struct {
// Specifications for logging in files.
File FileLogSpec `json:"file,omitempty"`

// Specifications for stderr logging.
Stderr FileLogSpec `json:"stderr,omitempty"`

// Specifications for query logging.
Query OTLPLogSpec `json:"query,omitempty"`

// Specifications for profile logging.
Profile OTLPLogSpec `json:"profile,omitempty"`
}

type FileLogSpec struct {
// Whether to enable file logging.
// +kubebuilder:default=false
Enabled bool `json:"enabled,omitempty"`

// Log level.
Level string `json:"level,omitempty"`

// Path to log directory.
Directory string `json:"directory,omitempty"`
}

type OTLPLogSpec struct {
// Whether to enable OTLP logging.
// +kubebuilder:default=false
Enabled bool `json:"enabled,omitempty"`

// OpenTelemetry Protocol
// +kubebuilder:default="http"
Protocol string `json:"protocol,omitempty"`

// Endpoint for OpenTelemetry Protocol
Endpoint string `json:"endpoint,omitempty"`
}

type WarehouseServiceSpec struct {
// Type of service [ClusterIP | NodePort | ExternalName | LoadBalance].
// +kubebuilder:default="ClusterIP"
Type string `json:"type,omitempty"`

// External name is needed when Type is set to "ExternalName"
ExternalName string `json:"externalName,omitempty"`
}

type WarehouseIngressSpec struct {
// Annotations for Ingress.
Annotations map[string]string `json:"annotations,omitempty"`

// Name of IngressClass.
IngressClassName string `json:"ingressClassName,omitempty"`

// Host name of ingress.
HostName string `json:"hostName,omitempty"`
}

// WarehouseSpec defines the desired state of Warehouse.
type WarehouseSpec struct {
// INSERT ADDITIONAL SPEC FIELDS - desired state of cluster
// Important: Run "make" to regenerate code after modifying this file
// Desired replicas of Query
// +kubebuilder:default=1
Replicas int `json:"replicas,omitempty"`

// Time for Query clsuter to suspend when no query requests are received.
// Set to 0 if you don't want Query cluster to be suspended.
// +kubebuilder:default=0
AutoSuspendAfterSecs int `json:"autoSuspendAfterSecs,omitempty"`

// Image for Query.
QueryImage string `json:"queryImage,omitempty"`

// Reference to the Tenant CR, which provides the configuration of storage and Meta cluster.
// Warehouse must be created in the Tenant's namespace.
Tenant *corev1.LocalObjectReference `json:"tenant,omitempty"`

// Foo is an example field of Warehouse. Edit warehouse_types.go to remove/update
Foo string `json:"foo,omitempty"`
// Configurations of cache in disk.
Cache DiskCacheSpec `json:"diskCacheSize,omitempty"`

// Configurations of logging.
Log LogSpec `json:"log,omitempty"`

// Additional labels added to Query pod.
PodLabels map[string]string `json:"labels,omitempty"`

// Resource required for each Query pod.
PodResource corev1.ResourceRequirements `json:"resourcesPerNode,omitempty"`

// Taint tolerations for Query pod.
PodTolerations []corev1.Toleration `json:"tolerations,omitempty"`

// Node selector for Query pod.
NodeSelector map[string]string `json:"nodeSelector,omitempty"`

// Service specifications for Query cluster.
Service WarehouseServiceSpec `json:"service,omitempty"`

// Ingress specifications for Query cluster.
Ingress WarehouseIngressSpec `json:"ingress,omitempty"`

// Custom settings that will append to the config file of Query.
Settings map[string]string `json:"settings,omitempty"`
}

// WarehouseStatus defines the observed state of Warehouse.
type WarehouseStatus struct {
// INSERT ADDITIONAL STATUS FIELD - define observed state of cluster
// Important: Run "make" to regenerate code after modifying this file
// Number of the ready Query.
ReadyReplicas int `json:"readyReplicas,omitempty"`

// Conditions for the Tenant.
// +listType=map
// +listMapKey=type
// +patchStrategy=merge
// +patchMergeKey=type
Conditions []metav1.Condition `json:"status,omitempty" patchStrategy:"merge" patchMergeKey:"type"`
}

// +kubebuilder:object:root=true
// +kubebuilder:subresource:status
// +kubebuilder:printcolumn:name="Ready",type=number,JSONPath=`.status.readyReplicas`
// +kubebuilder:printcolumn:name="Replicas",type=number,JSONPath=`.spec.replicas`
// +kubebuilder:printcolumn:name="State",type=string,JSONPath=`.status.conditions[-1:].type`
// +kubebuilder:printcolumn:name="Age",type=date,JSONPath=`.metadata.creationTimestamp`
// +genclient
// +k8s:openapi-gen=true
// +k8s:deepcopy-gen:interfaces=k8s.io/apimachinery/pkg/runtime.Object

// Warehouse is the Schema for the warehouses API.
type Warehouse struct {
Expand Down
Loading
Loading