Skip to content
Merged
Show file tree
Hide file tree
Changes from 8 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 0 additions & 3 deletions antora.yml
Original file line number Diff line number Diff line change
Expand Up @@ -5,9 +5,6 @@ start_page: 'index.adoc'

nav:
- modules/ROOT/nav.adoc
- modules/install-upgrade/nav.adoc
- modules/components/nav.adoc
- modules/operations/nav.adoc

asciidoc:
attributes:
Expand Down
Binary file removed modules/ROOT/images/GCP-all-pods.png
Binary file not shown.
Binary file removed modules/ROOT/images/care.png
Binary file not shown.
Binary file removed modules/ROOT/images/install-to-admin-console.png
Binary file not shown.
Binary file not shown.
Binary file removed modules/ROOT/images/luna-streaming-configure.png
Binary file not shown.
Binary file not shown.
Binary file removed modules/ROOT/images/luna-streaming-tls-warning.png
Binary file not shown.
Binary file not shown.
24 changes: 24 additions & 0 deletions modules/ROOT/nav.adoc
Original file line number Diff line number Diff line change
@@ -1 +1,25 @@
.Get started
* xref:ROOT:index.adoc[]
* xref:ROOT:faqs.adoc[]

.Install and upgrade
* xref:install-upgrade:quickstart-helm-installs.adoc[]
* xref:install-upgrade:quickstart-server-installs.adoc[]

.Components and extensions
* xref:components:admin-console-tutorial.adoc[]
* xref:components:admin-console-vm.adoc[]
* xref:components:pulsar-monitor.adoc[]
* xref:components:heartbeat-vm.adoc[]
* xref:components:pulsar-beam.adoc[]
* xref:components:pulsar-sql.adoc[]
* xref:components:starlight-for-kafka.adoc[]
* xref:components:starlight-for-rabbitmq.adoc[]

.Operations
* xref:operations:auth.adoc[]
* xref:operations:functions.adoc[]
* xref:operations:io-connectors.adoc[]
* xref:operations:io-elastic-sink.adoc[]
* xref:operations:scale-cluster.adoc[]
* xref:operations:troubleshooting.adoc[]
2 changes: 1 addition & 1 deletion modules/ROOT/pages/faqs.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -113,7 +113,7 @@ There are two packages:
* The `pulsar-sink` functionality of DataStax Apache Pulsar Connector is included with DataStax Luna Streaming. It's built in!
* You can optionally download the DataStax Apache Pulsar Connector tarball from the https://downloads.datastax.com/#pulsar-sink[DataStax Downloads] site, and then use it as its own product with your open-source Apache Pulsar install.

If you're using open-source software (OSS) Apache Pulsar, you can use DataStax Apache Pulsar Connector with the OSS to take advantage of this `pulsar-sink` for Cassandra. See the DataStax Apache Pulsar Connector https://docs.datastax.com/en/pulsar-connector/[documentation].
If you're using open-source software (OSS) Apache Pulsar, you can use DataStax Apache Pulsar Connector with the OSS to take advantage of this `pulsar-sink` for Cassandra. See the xref:pulsar-connector:ROOT:index.adoc[DataStax Apache Pulsar Connector documentation].

== APIs

Expand Down
2 changes: 1 addition & 1 deletion modules/ROOT/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,7 @@ In addition to the distribution of https://pulsar.apache.org/en/versions/[Apache

* Transparent batching of messages

== What's next?
== Next steps

* If you have an existing Kubernetes environment, deploy Luna Streaming with a xref:install-upgrade:quickstart-helm-installs.adoc[Helm Installation].
* If you have a bare metal or a cloud environment, see xref:install-upgrade:quickstart-server-installs.adoc[Server/VM Installation].
Expand Down
22 changes: 0 additions & 22 deletions modules/ROOT/partials/auto-create-credentials.adoc

This file was deleted.

8 changes: 5 additions & 3 deletions modules/ROOT/partials/cleanup-terminal-and-helm.adoc
Original file line number Diff line number Diff line change
@@ -1,11 +1,13 @@
Return to each window running an open process and enter `ctrl-c` to end the process. +
To completely remove all traces of the Helm chart, remove the namespace.
Return to each window running an open process and enter `ctrl-c` to end the process.
To completely remove all traces of the Helm chart, remove the namespace:

[source,shell]
----
kubectl delete namespace datastax-pulsar
----

If you want to keep the data, uninstall only the chart.
If you want to keep the data, uninstall only the chart:

[source,shell]
----
helm --namespace datastax-pulsar uninstall my-pulsar-cluster
Expand Down
10 changes: 0 additions & 10 deletions modules/ROOT/partials/helm-chart-recommendations.adoc

This file was deleted.

19 changes: 13 additions & 6 deletions modules/ROOT/partials/install-helm.adoc
Original file line number Diff line number Diff line change
@@ -1,23 +1,30 @@
. Add the DataStax Helm chart repo to your Helm store.
. Add the DataStax Helm chart repo to your Helm store:
+
[source,shell]
----
include::https://raw.githubusercontent.com/datastaxdevs/luna-streaming-examples/using-s4k-guide/install.sh[tag=helm-repo-add]
helm repo add datastax-pulsar https://datastax.github.io/pulsar-helm-chart
----

. Install the Helm chart using a minimalist values file. +
. Install the Helm chart using a minimalist values file.
This command creates a Helm release named "my-pulsar-cluster" using the DataStax Luna Helm chart, within the K8s namespace "datastax-pulsar".
The minimal cluster creates only the essential components and has no ingress or load balanced services.
+
[source,shell,subs="attributes+"]
----
VALUES_URL="{helmValuesPath}"
include::https://raw.githubusercontent.com/datastaxdevs/luna-streaming-examples/using-s4k-guide/install.sh[tag=helm-install]
helm install \
--namespace datastax-pulsar \
--create-namespace \
--values $VALUES_URL \
--version 3.0.4 \
my-pulsar-cluster \
datastax-pulsar/pulsar
----

. Wait for the broker pod to be in a running state. You might see a few restarts as your components start up.
. Wait for the broker pod to be in a running state.
You might see a few restarts as your components start up.
+
[source,shell,subs="attributes+"]
----
include::https://raw.githubusercontent.com/datastaxdevs/luna-streaming-examples/using-s4k-guide/install.sh[tag=wait-for-broker]
kubectl -n datastax-pulsar wait --for=condition=Ready pod/pulsar-broker-0 --timeout=120s
----
13 changes: 0 additions & 13 deletions modules/ROOT/partials/port-forward-beam.adoc

This file was deleted.

6 changes: 0 additions & 6 deletions modules/ROOT/partials/port-forward-s4k.adoc

This file was deleted.

6 changes: 0 additions & 6 deletions modules/ROOT/partials/port-forward-s4r.adoc

This file was deleted.

5 changes: 0 additions & 5 deletions modules/ROOT/partials/port-forward-sql.adoc

This file was deleted.

5 changes: 3 additions & 2 deletions modules/ROOT/partials/port-forward-web-service.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
In a new terminal, port forward Pulsar’s admin service.
In a new terminal, port forward Pulsar's admin service:

[source,shell]
----
include::https://raw.githubusercontent.com/datastaxdevs/luna-streaming-examples/main/forward-broker-web-service.sh[]
kubectl port-forward -n datastax-pulsar service/pulsar-broker 8080:8080
----
9 changes: 0 additions & 9 deletions modules/components/nav.adoc

This file was deleted.

20 changes: 10 additions & 10 deletions modules/components/pages/admin-console-tutorial.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ The *DataStax Admin Console for Apache Pulsar(R)* is a web-based UI from DataSta

In the *Luna Streaming Pulsar Admin Console*, you can use Pulsar clients to send and receive pub/sub messages.

If you installed the Admin console with the xref:install-upgrade:quickstart-helm-installs.adoc[DataStax Helm chart], access the Admin console via the `pulsar-adminconsole` external load balancer endpoint in your cloud provider:
If you installed the Admin console with the xref:install-upgrade:quickstart-helm-installs.adoc[DataStax Helm chart], access the Admin console with the `pulsar-adminconsole` external load balancer endpoint in your cloud provider:

image::GCP-all-pods.png[GCP Pods]

Expand Down Expand Up @@ -50,7 +50,7 @@ Scroll down to the Consume tab. In this simple example, which verifies that the

image::test-message.png[Send a message using a test client]

In doing so, you published a message to your server, and in the Test Client you're listening to your own topic. +
In doing so, you published a message to your server, and in the Test Client you're listening to your own topic.

Your client is working with the Pulsar server.

Expand All @@ -59,7 +59,7 @@ Your client is working with the Pulsar server.

By default, topics are automatically created when you first publish or subscribe. To manually create a topic or to create a partitioned topic, go to *Topics*.

The first part of a full topic name is your tenant name, followed by the namespace, and then the topic name, as below: +
The first part of a full topic name is your tenant name, followed by the namespace, and then the topic name, as below:

`public/namespace/my-topic`

Expand All @@ -72,8 +72,8 @@ To see detailed information about your topics, go to *Topics*.
[#code-samples]
== Code samples

On the Pulsar Admin Console's *Code Samples* page, there are examples for Java, Python, Golang, Node.js, WebSocket, and HTTP clients. +
Each example shows Producer, Consumer, and Reader code, plus language-specific examples of setting project properties and dependencies. +
On the Pulsar Admin Console's *Code Samples* page, there are examples for Java, Python, Golang, Node.js, WebSocket, and HTTP clients.
Each example shows Producer, Consumer, and Reader code, plus language-specific examples of setting project properties and dependencies.
For example, selecting Java will show you how to connect your Java project to Pulsar by modifying your Maven's `pom.xml` file.

[#connect-to-pulsar]
Expand All @@ -83,11 +83,11 @@ This section describes how to connect Pulsar components to the Admin console.

=== Creating and showing credentials

When connecting clients, you'll need to provide your connect token to identify your account. In the Pulsar APIs, you specify the token when creating the client object. The token is your password to your account, so keep it safe. +
When connecting clients, you'll need to provide your connect token to identify your account. In the Pulsar APIs, you specify the token when creating the client object. The token is your password to your account, so keep it safe.

The code samples automatically add your client token as part of the source code for convenience. However, a more secure practice would be to read the token from an environment variable or a file. +
The code samples automatically add your client token as part of the source code for convenience. However, a more secure practice would be to read the token from an environment variable or a file.

In the Admin console's *Credentials* page, select **Create New token** and follow the instructions. +
In the Admin console's *Credentials* page, select **Create New token** and follow the instructions.

If you previously created a token, use the Credentials page to get its value.

Expand Down Expand Up @@ -155,6 +155,6 @@ You can also follow along with this video from our *Five Minutes About Pulsar* s

video::1IwblLfPiPQ[youtube, list=PL2g2h-wyI4SqeKH16czlcQ5x4Q_z-X7_m]

== What's next
== Next steps

For more on building and running a standalone Pulsar Admin console, see the xref:admin-console-vm.adoc[Admin Console on Server/VM] or the Pulsar Admin console repo https://github.com/datastax/pulsar-admin-console#dev[readme].
For more on building and running a standalone Pulsar Admin console, see the xref:admin-console-vm.adoc[Admin Console on Server/VM] or the Pulsar Admin console repo https://github.com/datastax/pulsar-admin-console#dev[readme].
48 changes: 24 additions & 24 deletions modules/components/pages/admin-console-vm.adoc
Original file line number Diff line number Diff line change
@@ -1,19 +1,19 @@
= Install Pulsar Admin Console on Server/VM
:page-tag: luna-streaming,dev,install,admin,pulsar

*Pulsar Admin Console* is a web-based UI that administrates topics, namespaces, sources, sinks and various aspects of Apache Pulsar™ features. +
*Pulsar Admin Console* is a web-based UI that administrates topics, namespaces, sources, sinks and various aspects of Apache Pulsar(TM) features.

The Admin Console is a VueJS application that runs in a browser. It also includes a web server that serves up the files for the Admin Console as well as providing configuration and authentication services. +
The Admin Console is a VueJS application that runs in a browser. It also includes a web server that serves up the files for the Admin Console as well as providing configuration and authentication services.

This document covers: +
This document covers:

* <<install,*Installation*>> +
* <<install,*Installation*>>

* <<configuration,*Configuration*>> +
* <<configuration,*Configuration*>>

* <<configuration-reference,*Configuration Reference*>> +
* <<configuration-reference,*Configuration Reference*>>

* <<auth-modes,*Auth Modes*>> +
* <<auth-modes,*Auth Modes*>>

[#install]
== Install Pulsar Admin Console
Expand Down Expand Up @@ -58,15 +58,15 @@ Port 6454 is specified in `pulsar-admin-console/config/default.json`. To change
[#configuration]
== Configuration

The `default.json` configuration file contains a set of general configs for the Admin Console, plus a server-specific set under `server_config`. The Admin Console server proxies all requests from the Admin Console to the Pulsar broker (or Pulsar proxy). +
The `default.json` configuration file contains a set of general configs for the Admin Console, plus a server-specific set under `server_config`. The Admin Console server proxies all requests from the Admin Console to the Pulsar broker (or Pulsar proxy).

You can modify the configuration for the Admin Console in `pulsar-admin-console/config/default.json`, or place additional configuration files (for example, `local.json`) in the `/config` subdirectory to override parameters. +
You can modify the configuration for the Admin Console in `pulsar-admin-console/config/default.json`, or place additional configuration files (for example, `local.json`) in the `/config` subdirectory to override parameters.

You need to configure `pulsar_url` to point to one of your brokers or a proxy/loadbalancer in front of the brokers (can be Pulsar proxy). The Admin Console server must be able to directly reach each broker by the IP/hostname that is returned by the Pulsar CLI command `pulsar-admin brokers list <clustername>`. +
You need to configure `pulsar_url` to point to one of your brokers or a proxy/loadbalancer in front of the brokers (can be Pulsar proxy). The Admin Console server must be able to directly reach each broker by the IP/hostname that is returned by the Pulsar CLI command `pulsar-admin brokers list <clustername>`.

[NOTE]
====
The cluster name can be retrieved with the CLI command `pulsar-admin clusters list`. +
The cluster name can be retrieved with the CLI command `pulsar-admin clusters list`.
====

In a geo-replication configuration, use the cluster name for the local cluster. Set this cluster name in the `cluster_name` setting of the configuration file.
Expand Down Expand Up @@ -125,37 +125,37 @@ No login screen is presented. Authentication must be disabled in Pulsar because

=== "auth_mode": "user"

The Admin Console is protected by a login screen. Credentials are configured using the `username` and `password` settings in the `/config/default.json` file. +
The Admin Console is protected by a login screen. Credentials are configured using the `username` and `password` settings in the `/config/default.json` file.
Once authenticated with these credentials, the token for connecting to Pulsar is retrieved from the server (configured using `token_path` or `admin_token`) and used to authenticate with the Pulsar cluster.

=== "auth_mode": "k8"

The Admin Console is protected by a login screen. The credentials are retrieved from a Kubernetes secret. +
The Admin Console is protected by a login screen. The credentials are retrieved from a Kubernetes secret.

The Admin Console server *must* have access to a Kubernetes API, either using a `kubeconfig` file in the environment, or by having local access (for example, by running as a pod in a Kubernetes cluster).

The namespace for the secret is set using `k8s_namespace` in the `/config/default.json` file. +
The secret name must have a prefix of `dashboard-user-` followed by the `username`. +
For example, the admin user would have a secret name of `dashboard-user-admin`. +
The password must be stored in the secret with a key of `password` and a value of the password itself. +
The namespace for the secret is set using `k8s_namespace` in the `/config/default.json` file.
The secret name must have a prefix of `dashboard-user-` followed by the `username`.
For example, the admin user would have a secret name of `dashboard-user-admin`.
The password must be stored in the secret with a key of `password` and a value of the password itself.

Multiple secrets with the prefix can be configured to set up multiple users for the Admin Console. A password can be reset by patching the corresponding Kubernetes secret. +
Multiple secrets with the prefix can be configured to set up multiple users for the Admin Console. A password can be reset by patching the corresponding Kubernetes secret.

Once the user is authenticated using one of the Kubernetes secrets, the token for connecting to Pulsar is retrieved from the server (configured using `token_path` or `admin_token`) and used to authenticate with the Pulsar cluster.

=== "auth_mode": "openidconnect"

In this auth mode, the dashboard will use your login credentials to retrieve a JWT from an authentication provider. +
In this auth mode, the dashboard will use your login credentials to retrieve a JWT from an authentication provider.

In the *DataStax Pulsar Helm Chart*, this is implemented by integrating the Pulsar Admin Console with Keycloak. Upon successful retrieval of the JWT, the Admin Console will use the retrieved JWT as the bearer token when making calls to Pulsar. +
In the *DataStax Pulsar Helm Chart*, this is implemented by integrating the Pulsar Admin Console with Keycloak. Upon successful retrieval of the JWT, the Admin Console will use the retrieved JWT as the bearer token when making calls to Pulsar.

In addition to configuring the `auth_mode`, you must also configure the `oauth_client_id` (see <<configuration-reference,*Configuration reference*>>). This is the client id that the Console will use when authenticating with Keycloak. Note that in Keycloak, it is important that this client exists and that it has the sub claim properly mapped to your desired Pulsar subject. Otherwise, the JWT won't work as desired.

==== *Connecting to an OpenID Connect Auth/Identity Provider*
==== Connecting to an OpenID Connect Auth/Identity Provider

When opening the Admin Console, the first page is the login page. When using the `openidconnect` auth mode, the auth call needs to go to the Provider's server. +
When opening the Admin Console, the first page is the login page. When using the `openidconnect` auth mode, the auth call needs to go to the Provider's server.
In the current design, nginx must be configured to route the call to the provider. The *DataStax Pulsar Helm Chart* does this automatically.

== Next
== Next steps

To install the Admin Console in a cloud environment, see the xref:components:admin-console-tutorial.adoc[Admin Console Tutorial].
To install the Admin Console in a cloud environment, see the xref:components:admin-console-tutorial.adoc[Admin Console Tutorial].
Loading