Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
56 changes: 56 additions & 0 deletions api/v1alpha1/pulsargeoreplication_types.go
Original file line number Diff line number Diff line change
Expand Up @@ -36,6 +36,12 @@ type PulsarGeoReplicationSpec struct {
// +kubebuilder:validation:Enum=CleanUpAfterDeletion;KeepAfterDeletion
// +optional
LifecyclePolicy PulsarResourceLifeCyclePolicy `json:"lifecyclePolicy,omitempty"`

// ClusterParamsOverride allows overriding specific cluster parameters when setting up
// geo-replication. This is useful when the destination cluster requires different
// configuration than what's defined in the DestinationConnectionRef.
// +optional
ClusterParamsOverride *ClusterParamsOverride `json:"clusterParamsOverride,omitempty"`
}

// PulsarGeoReplicationStatus defines the observed state of PulsarGeoReplication
Expand Down Expand Up @@ -89,3 +95,53 @@ type ClusterInfo struct {
// ConnectionRef is the connection reference that can connect to the pulsar cluster
ConnectionRef corev1.LocalObjectReference `json:"connectionRef"`
}

// ClusterParamsOverride allows overriding specific parameters when creating/updating cluster info
// for geo-replication. This provides flexibility to customize cluster configuration without
// modifying the underlying PulsarConnection.
type ClusterParamsOverride struct {
// ServiceURL overrides the HTTP(S) URL for the Pulsar cluster's admin service
// +optional
ServiceURL *string `json:"serviceURL,omitempty"`

// ServiceSecureURL overrides the HTTPS URL for secure connections to the Pulsar admin service
// +optional
ServiceSecureURL *string `json:"serviceSecureURL,omitempty"`

// BrokerServiceURL overrides the non-TLS URL for connecting to Pulsar brokers
// +optional
BrokerServiceURL *string `json:"brokerServiceURL,omitempty"`

// BrokerServiceSecureURL overrides the TLS-enabled URL for secure connections to Pulsar brokers
// +optional
BrokerServiceSecureURL *string `json:"brokerServiceSecureURL,omitempty"`

// BrokerClientTrustCertsFilePath overrides the file path to the trusted TLS certificate
// for outgoing connections to Pulsar brokers
// +optional
BrokerClientTrustCertsFilePath *string `json:"brokerClientTrustCertsFilePath,omitempty"`

// Authentication overrides the authentication configuration for the cluster.
// When this field is set, the secret update check will be skipped for this geo-replication.
// +optional
Authentication *ClusterAuthOverride `json:"authentication,omitempty"`
}

// ClusterAuthOverride allows overriding authentication parameters for cluster configuration.
// This is useful when the geo-replication target requires different authentication than
// the source connection.
type ClusterAuthOverride struct {
// AuthPlugin specifies the authentication plugin class name
// Common values: "org.apache.pulsar.client.impl.auth.AuthenticationToken",
// "org.apache.pulsar.client.impl.auth.oauth2.AuthenticationOAuth2"
// +optional
AuthPlugin *string `json:"authPlugin,omitempty"`

// AuthParameters contains the authentication parameters as a string.
// Format depends on the AuthPlugin:
// - For Token: "token:your-token-here"
// - For Token: "file://your-token-file-path-on-brokers"
// - For OAuth2: JSON string with client credentials
// +optional
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does it support configuring tokens using file format? such as: file:///your-token-file-path

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yes it is depends on broker side. i have update the comments to address the file path format, ptal when you have time, thanks.

AuthParameters *string `json:"authParameters,omitempty"`
}
77 changes: 76 additions & 1 deletion api/v1alpha1/zz_generated.deepcopy.go

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

Original file line number Diff line number Diff line change
Expand Up @@ -54,6 +54,54 @@ spec:
spec:
description: PulsarGeoReplicationSpec defines the desired state of PulsarGeoReplication
properties:
clusterParamsOverride:
description: |-
ClusterParamsOverride allows overriding specific cluster parameters when setting up
geo-replication. This is useful when the destination cluster requires different
configuration than what's defined in the DestinationConnectionRef.
properties:
authentication:
description: |-
Authentication overrides the authentication configuration for the cluster.
When this field is set, the secret update check will be skipped for this geo-replication.
properties:
authParameters:
description: |-
AuthParameters contains the authentication parameters as a string.
Format depends on the AuthPlugin:
- For Token: "token:your-token-here"
- For Token: "file://your-token-file-path-on-brokers"
- For OAuth2: JSON string with client credentials
type: string
authPlugin:
description: |-
AuthPlugin specifies the authentication plugin class name
Common values: "org.apache.pulsar.client.impl.auth.AuthenticationToken",
"org.apache.pulsar.client.impl.auth.oauth2.AuthenticationOAuth2"
type: string
type: object
brokerClientTrustCertsFilePath:
description: |-
BrokerClientTrustCertsFilePath overrides the file path to the trusted TLS certificate
for outgoing connections to Pulsar brokers
type: string
brokerServiceSecureURL:
description: BrokerServiceSecureURL overrides the TLS-enabled
URL for secure connections to Pulsar brokers
type: string
brokerServiceURL:
description: BrokerServiceURL overrides the non-TLS URL for connecting
to Pulsar brokers
type: string
serviceSecureURL:
description: ServiceSecureURL overrides the HTTPS URL for secure
connections to the Pulsar admin service
type: string
serviceURL:
description: ServiceURL overrides the HTTP(S) URL for the Pulsar
cluster's admin service
type: string
type: object
connectionRef:
description: ConnectionRef is the reference to the source PulsarConnection
properties:
Expand Down
62 changes: 62 additions & 0 deletions docs/pulsar_geo_replication.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ The `PulsarGeoReplication` resource has the following specifications:
| `connectionRef` | Reference to the PulsarConnection resource used to connect to the source Pulsar cluster. | Yes |
| `destinationConnectionRef` | Reference to the PulsarConnection resource used to connect to the destination Pulsar cluster. | Yes |
| `lifecyclePolicy` | Determines whether to keep or delete the geo-replication configuration when the Kubernetes resource is deleted. Options: `CleanUpAfterDeletion`, `KeepAfterDeletion`. Default is `CleanUpAfterDeletion`. | No |
| `clusterParamsOverride` | Allows overriding specific cluster parameters when setting up geo-replication. This is useful when the destination cluster requires different configuration than what's defined in the `destinationConnectionRef`. See [Cluster Parameters Override](#cluster-parameters-override) for details. | No |

The `PulsarGeoReplication` resource is designed to configure geo-replication between separate Pulsar instances. It creates a new "Cluster" in the destination Pulsar cluster identified by `destinationConnectionRef`. This setup allows configuring the replication of data from the source cluster (identified by `connectionRef`) to the destination cluster. By establishing this connection, the brokers in the source cluster can communicate with and replicate data to the brokers in the destination cluster, enabling geo-replication between the two separate Pulsar instances.

Expand All @@ -49,6 +50,67 @@ Note: When configuring geo-replication between `connectionRef` and `destinationC

1. The brokers in the `connectionRef` cluster are able to communicate with the `destinationConnectionRef` cluster, and the `destinationConnectionRef` cluster is able to authenticate the connections from the `connectionRef` cluster.

### Cluster Parameters Override

The `clusterParamsOverride` field provides a powerful way to customize cluster configuration for geo-replication without modifying the underlying `PulsarConnection` resource. This is particularly useful when:

1. **Different authentication is required** for geo-replication compared to regular cluster operations
2. **Alternative URLs need to be used** for inter-cluster communication
3. **Specific TLS configurations** are needed for cross-cluster connections

#### Supported Override Parameters

The `clusterParamsOverride` supports the following fields:

- **URL Configuration**:
- `serviceURL`: Override the HTTP(S) URL for the Pulsar cluster's admin service
- `serviceSecureURL`: Override the HTTPS URL for secure admin connections
- `brokerServiceURL`: Override the non-TLS URL for broker connections
- `brokerServiceSecureURL`: Override the TLS-enabled URL for secure broker connections

- **TLS Configuration**:
- `brokerClientTrustCertsFilePath`: Override the path to trusted TLS certificates

- **Authentication Configuration**:
- `authentication.authPlugin`: Override the authentication plugin class name
- `authentication.authParameters`: Override the authentication parameters

#### Authentication Override Benefits

When `authentication` is specified in the override, the system automatically:
- **Skips secret validation checks** for the destination connection
- **Avoids unnecessary Secret API calls** for improved performance
- **Uses the override authentication directly** without processing the destinationConnectionRef authentication

#### Example Usage

```yaml
apiVersion: resource.streamnative.io/v1alpha1
kind: PulsarGeoReplication
metadata:
name: us-east-to-west-geo-replication
namespace: us-east
spec:
connectionRef:
name: us-east-local-connection
destinationConnectionRef:
name: us-east-to-west-connection
clusterParamsOverride:
# Override URLs for cross-cluster communication
serviceURL: "https://geo-replication-admin.us-west.example.com:8443"
brokerServiceURL: "pulsar://geo-replication-broker.us-west.example.com:6650"
# Override authentication for geo-replication
authentication:
authPlugin: "org.apache.pulsar.client.impl.auth.AuthenticationToken"
authParameters: "token:geo-replication-specific-token"
```

**Important Notes**:
- Override parameters take precedence over the corresponding fields in `destinationConnectionRef`
- Only non-null override values will replace the destination connection values
- Authentication override is particularly useful for scenarios requiring different credentials for geo-replication
- The override does not affect how the operator connects to manage other resources in the destination cluster

### Lifecycle Policy

The `lifecyclePolicy` field determines what happens to the geo-replication configuration when the Kubernetes PulsarGeoReplication resource is deleted:
Expand Down
Loading