Skip to content

Commit e027057

Browse files
Update nav format
1 parent fc47cdb commit e027057

19 files changed

+136
-131
lines changed

modules/ROOT/nav.adoc

Lines changed: 59 additions & 65 deletions
Original file line numberDiff line numberDiff line change
@@ -1,65 +1,59 @@
1-
* Processing Data
2-
** Change Data Capture
3-
*** xref:streaming-learning:use-cases-architectures:change-data-capture/index.adoc[Overview]
4-
*** xref:streaming-learning:use-cases-architectures:change-data-capture/table-schema-evolution.adoc[Table schema evolution]
5-
*** xref:streaming-learning:use-cases-architectures:change-data-capture/consuming-change-data.adoc[Consuming change data]
6-
*** xref:streaming-learning:use-cases-architectures:change-data-capture/questions-and-patterns.adoc[Questions and patterns]
7-
8-
** xref:streaming-learning:use-cases-architectures:real-time-data-pipeline/index.adoc[Data Pipelines]
9-
10-
* Migrating to Apache Pulsar
11-
12-
** xref:streaming-learning:use-cases-architectures:starlight/kafka/index.adoc[Starlight for Kafka]
13-
** xref:streaming-learning:use-cases-architectures:starlight/rabbitmq/index.adoc[Starlight for RabbitMQ]
14-
** xref:streaming-learning:use-cases-architectures:starlight/jms/index.adoc[Starlight for JMS]
15-
16-
* APIs & References
17-
18-
** Connectors
19-
20-
*** xref:streaming-learning:pulsar-io:connectors/index.adoc[Connector Overview]
21-
22-
*** Sources
23-
**** xref:streaming-learning:pulsar-io:connectors/sources/data-generator.adoc[]
24-
**** xref:streaming-learning:pulsar-io:connectors/sources/debezium-mongodb.adoc[]
25-
**** xref:streaming-learning:pulsar-io:connectors/sources/debezium-mysql.adoc[]
26-
**** xref:streaming-learning:pulsar-io:connectors/sources/debezium-oracle.adoc[]
27-
**** xref:streaming-learning:pulsar-io:connectors/sources/debezium-postgres.adoc[]
28-
**** xref:streaming-learning:pulsar-io:connectors/sources/debezium-sqlserver.adoc[]
29-
**** xref:streaming-learning:pulsar-io:connectors/sources/kafka.adoc[]
30-
**** xref:streaming-learning:pulsar-io:connectors/sources/kinesis.adoc[]
31-
32-
*** Sinks
33-
**** xref:streaming-learning:pulsar-io:connectors/sinks/astra-db.adoc[]
34-
**** xref:streaming-learning:pulsar-io:connectors/sinks/cloud-storage.adoc[]
35-
**** xref:streaming-learning:pulsar-io:connectors/sinks/elastic-search.adoc[]
36-
**** xref:streaming-learning:pulsar-io:connectors/sinks/google-bigquery.adoc[]
37-
**** xref:streaming-learning:pulsar-io:connectors/sinks/jdbc-postgres.adoc[]
38-
**** xref:streaming-learning:pulsar-io:connectors/sinks/jdbc-mariadb.adoc[]
39-
**** xref:streaming-learning:pulsar-io:connectors/sinks/jdbc-clickhouse.adoc[]
40-
**** xref:streaming-learning:pulsar-io:connectors/sinks/jdbc-sqllite.adoc[]
41-
**** xref:streaming-learning:pulsar-io:connectors/sinks/kafka.adoc[]
42-
**** xref:streaming-learning:pulsar-io:connectors/sinks/kinesis.adoc[]
43-
**** xref:streaming-learning:pulsar-io:connectors/sinks/snowflake.adoc[]
44-
45-
** xref:streaming-learning:functions:astream-functions.adoc[Pulsar Functions]
46-
47-
** Transformation Functions
48-
49-
*** xref:streaming-learning:functions:index.adoc[Overview]
50-
*** xref:streaming-learning:functions:cast.adoc[]
51-
*** xref:streaming-learning:functions:compute.adoc[]
52-
*** xref:streaming-learning:functions:drop.adoc[]
53-
*** xref:streaming-learning:functions:flatten.adoc[]
54-
*** xref:streaming-learning:functions:merge-key-value.adoc[]
55-
*** xref:streaming-learning:functions:unwrap-key-value.adoc[]
56-
*** xref:streaming-learning:functions:deploy-in-sink.adoc[]
57-
*** xref:streaming-learning:functions:drop-fields.adoc[]
58-
59-
** Topic Subscriptions
60-
61-
*** xref:streaming-learning:subscriptions:index.adoc[Overview]
62-
*** xref:streaming-learning:subscriptions:astream-subscriptions-exclusive.adoc[Exclusive]
63-
*** xref:streaming-learning:subscriptions:astream-subscriptions-failover.adoc[Failover]
64-
*** xref:streaming-learning:subscriptions:astream-subscriptions-shared.adoc[Shared]
65-
*** xref:streaming-learning:subscriptions:astream-subscriptions-keyshared.adoc[Key Shared]
1+
.Processing data
2+
* Change Data Capture (CDC)
3+
** xref:use-cases-architectures:change-data-capture/index.adoc[]
4+
** xref:use-cases-architectures:change-data-capture/table-schema-evolution.adoc[]
5+
** xref:use-cases-architectures:change-data-capture/consuming-change-data.adoc[]
6+
** xref:use-cases-architectures:change-data-capture/questions-and-patterns.adoc[]
7+
* Data pipelines
8+
** xref:use-cases-architectures:real-time-data-pipeline/index.adoc[]
9+
** xref:use-cases-architectures:real-time-data-pipeline/01-create-astra-objects.adoc[]
10+
** xref:use-cases-architectures:real-time-data-pipeline/02-create-decodable-objects.adoc[]
11+
** xref:use-cases-architectures:real-time-data-pipeline/03-put-it-all-together.adoc[]
12+
** xref:use-cases-architectures:real-time-data-pipeline/04-debugging-and-clean-up.adoc[]
13+
14+
.Migrating to Apache Pulsar
15+
* xref:use-cases-architectures:starlight/kafka/index.adoc[]
16+
* xref:use-cases-architectures:starlight/rabbitmq/index.adoc[]
17+
* xref:use-cases-architectures:starlight/jms/index.adoc[]
18+
19+
.APIs & References
20+
* Connectors
21+
** xref:pulsar-io:connectors/index.adoc[]
22+
** Sources
23+
*** xref:pulsar-io:connectors/sources/data-generator.adoc[]
24+
*** xref:pulsar-io:connectors/sources/debezium-mongodb.adoc[]
25+
*** xref:pulsar-io:connectors/sources/debezium-mysql.adoc[]
26+
*** xref:pulsar-io:connectors/sources/debezium-oracle.adoc[]
27+
*** xref:pulsar-io:connectors/sources/debezium-postgres.adoc[]
28+
*** xref:pulsar-io:connectors/sources/debezium-sqlserver.adoc[]
29+
*** xref:pulsar-io:connectors/sources/kafka.adoc[]
30+
*** xref:pulsar-io:connectors/sources/kinesis.adoc[]
31+
** Sinks
32+
*** xref:pulsar-io:connectors/sinks/astra-db.adoc[]
33+
*** xref:pulsar-io:connectors/sinks/cloud-storage.adoc[]
34+
*** xref:pulsar-io:connectors/sinks/elastic-search.adoc[]
35+
*** xref:pulsar-io:connectors/sinks/google-bigquery.adoc[]
36+
*** xref:pulsar-io:connectors/sinks/jdbc-postgres.adoc[]
37+
*** xref:pulsar-io:connectors/sinks/jdbc-mariadb.adoc[]
38+
*** xref:pulsar-io:connectors/sinks/jdbc-clickhouse.adoc[]
39+
*** xref:pulsar-io:connectors/sinks/jdbc-sqllite.adoc[]
40+
*** xref:pulsar-io:connectors/sinks/kafka.adoc[]
41+
*** xref:pulsar-io:connectors/sinks/kinesis.adoc[]
42+
*** xref:pulsar-io:connectors/sinks/snowflake.adoc[]
43+
* xref:functions:astream-functions.adoc[]
44+
* Transformation Functions
45+
** xref:functions:index.adoc[]
46+
** xref:functions:cast.adoc[]
47+
** xref:functions:compute.adoc[]
48+
** xref:functions:drop.adoc[]
49+
** xref:functions:flatten.adoc[]
50+
** xref:functions:merge-key-value.adoc[]
51+
** xref:functions:unwrap-key-value.adoc[]
52+
** xref:functions:deploy-in-sink.adoc[]
53+
** xref:functions:drop-fields.adoc[]
54+
* Topic Subscriptions
55+
** xref:subscriptions:index.adoc[]
56+
** xref:subscriptions:astream-subscriptions-exclusive.adoc[]
57+
** xref:subscriptions:astream-subscriptions-failover.adoc[]
58+
** xref:subscriptions:astream-subscriptions-shared.adoc[]
59+
** xref:subscriptions:astream-subscriptions-keyshared.adoc[]

modules/functions/pages/astream-functions.adoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
= Apache Pulsar Functions
2+
:navtitle: Pulsar Functions
23
:page-tag: astra-streaming,dev,develop,pulsar,java,python
34

45
Functions are lightweight compute processes that enable you to process each message received on a topic. You can apply custom logic to that message, transforming or enriching it, and then output it to a different topic.

modules/functions/pages/index.adoc

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
= Transform Functions
2+
:navtitle: Overview
23

34
A Pulsar *transform function* is a low-code implementation of xref:astra-streaming:developing:astream-functions.adoc[Pulsar functions]. +
45
Functions receive data from one or more input topics, apply user-supplied processing, and publish the results to another topic. +

modules/pulsar-io/pages/connectors/index.adoc

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
11
= Connectors
2-
:navtitle: Pulsar IO Connectors
2+
:navtitle: Connector Overview
33
:page-tag: connectors,sinks,sources,astra-streaming,dev,develop,pulsar,go
44

55
{product_name} offers fully-managed Apache Pulsar™ connectors.

modules/subscriptions/pages/astream-subscriptions-exclusive.adoc

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
= Exclusive subscriptions in Pulsar
2+
:navtitle: Exclusive
23
:page-tag: pulsar-subscriptions,quickstart,admin,dev,pulsar
34

45
*Subscriptions* in Pulsar describe which consumers are consuming data from a topic and how they want to consume that data. +
@@ -17,9 +18,9 @@ This example uses the `pulsarConsumer` object in `SimplePulsarConsumer.java` bel
1718
[source,java]
1819
----
1920
pulsarConsumer = pulsarClient.newConsumer(Schema.JSON(DemoBean.class))
20-
.topic("persistent://"
21-
+ conf.getTenantName() + "/"
22-
+ conf.getNamespace() + "/"
21+
.topic("persistent://"
22+
+ conf.getTenantName() + "/"
23+
+ conf.getNamespace() + "/"
2324
+ conf.getTopicName())
2425
.startMessageIdInclusive()
2526
.subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
@@ -31,7 +32,7 @@ pulsarConsumer = pulsarClient.newConsumer(Schema.JSON(DemoBean.class))
3132
====
3233
Pulsar creates an exclusive subscription by default when no `subscriptionType` is declared.
3334
====
34-
35+
3536
. Open the `pulsar-subscription-example` repo in the IDE of your choice and run `SimplePulsarConsumer.java` to begin consuming messages. +
3637
The confirmation message and a cursor appear to indicate the consumer is ready.
3738
+
@@ -42,7 +43,7 @@ The confirmation message and a cursor appear to indicate the consumer is ready.
4243
[pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://<tenant_name>/<namespace>/in][SimplePulsarConsumer] Subscribed to topic on <service_url> -- consumer: 0
4344
----
4445

45-
. In a new terminal window, run `SimplePulsarProducer.java` to begin producing messages.
46+
. In a new terminal window, run `SimplePulsarProducer.java` to begin producing messages.
4647
+
4748
[source,bash]
4849
----
@@ -51,7 +52,7 @@ The confirmation message and a cursor appear to indicate the consumer is ready.
5152
[main] INFO com.datastax.pulsar.SimplePulsarProducer - Message 68306175 sent
5253
----
5354

54-
. The consumer begins consuming the produced messages.
55+
. The consumer begins consuming the produced messages.
5556
+
5657
[source,bash]
5758
----
@@ -60,7 +61,7 @@ The confirmation message and a cursor appear to indicate the consumer is ready.
6061
[main] INFO com.datastax.pulsar.SimplePulsarConsumer - Message received: {"show_id":68306175,"cast":"LeBron James, Anthony Davis, Kyrie Irving, Damian Lillard, Klay Thompson...","country":"United States","date_added":"July 16, 2021","description":"NBA superstar LeBron James teams up with Bugs Bunny and the rest of the Looney Tunes for this long-awaited sequel.","director":"Malcolm D. Lee","duration":"120 min","listed_in":"Animation, Adventure, Comedy","rating":"PG","release_year":2021,"title":"Space Jam: A New Legacy","type":"Movie"}
6162
----
6263

63-
. Open a new terminal window and try to run `SimplePulsarConsumer.java`.
64+
. Open a new terminal window and try to run `SimplePulsarConsumer.java`.
6465
+
6566
[source,bash]
6667
----

modules/subscriptions/pages/astream-subscriptions-failover.adoc

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
= Failover subscriptions in Pulsar
2+
:navtitle: Failover
23
:page-tag: pulsar-subscriptions,quickstart,admin,dev,pulsar
34

45
*Subscriptions* in Pulsar describe which consumers are consuming data from a topic and how they want to consume that data. +
@@ -13,14 +14,14 @@ include::partial$subscription-prereq.adoc[]
1314
[#example]
1415
== Failover subscription example
1516

16-
To try out a Pulsar failover subscription, add `.subscriptionType(SubscriptionType.Failover)` to the `pulsarConsumer` in `SimplePulsarConsumer.java`.
17+
To try out a Pulsar failover subscription, add `.subscriptionType(SubscriptionType.Failover)` to the `pulsarConsumer` in `SimplePulsarConsumer.java`.
1718

1819
[source,java]
1920
----
2021
pulsarConsumer = pulsarClient.newConsumer(Schema.JSON(DemoBean.class))
21-
.topic("persistent://"
22-
+ conf.getTenantName() + "/"
23-
+ conf.getNamespace() + "/"
22+
.topic("persistent://"
23+
+ conf.getTenantName() + "/"
24+
+ conf.getNamespace() + "/"
2425
+ conf.getTopicName())
2526
.startMessageIdInclusive()
2627
.subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
@@ -40,7 +41,7 @@ The confirmation message and a cursor appear to indicate the consumer is ready.
4041
[pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://<tenant_name>/<namespace>/in][SimplePulsarConsumer] Subscribed to topic on <service_url> -- consumer: 0
4142
----
4243

43-
. In a new terminal window, run `SimplePulsarProducer.java` to begin producing messages.
44+
. In a new terminal window, run `SimplePulsarProducer.java` to begin producing messages.
4445
+
4546
[source,bash]
4647
----
@@ -52,7 +53,7 @@ The confirmation message and a cursor appear to indicate the consumer is ready.
5253
[main] INFO com.datastax.pulsar.SimplePulsarProducer - Message 42372149 sent
5354
----
5455

55-
. The primary consumer begins consuming messages.
56+
. The primary consumer begins consuming messages.
5657
+
5758
[source,bash]
5859
----
@@ -75,7 +76,7 @@ The confirmation message and a cursor appear to indicate the consumer is ready.
7576

7677
=== Failover subscription video
7778

78-
Follow along with this video from our *Five Minutes About Pulsar* series to see failover subscriptions in action.
79+
Follow along with this video from our *Five Minutes About Pulsar* series to see failover subscriptions in action.
7980

8081
video::ckB87OLs5eM[youtube, list=PL2g2h-wyI4SqeKH16czlcQ5x4Q_z-X7_m, height=445px,width=100%]
8182

modules/subscriptions/pages/astream-subscriptions-keyshared.adoc

Lines changed: 15 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -1,4 +1,5 @@
11
= Key_Shared subscriptions in Pulsar
2+
:navtitle: Key Shared
23
:page-tag: pulsar-subscriptions,quickstart,admin,dev,pulsar
34

45
*Subscriptions* in Pulsar describe which consumers are consuming data from a topic and how they want to consume that data. +
@@ -8,7 +9,7 @@ Pulsar's xref:astream-subscriptions-shared.adoc[shared subscription] model trade
89
*Keys* are generated with *hashing*, which converts arbitrary values like "topic-name" or JSON blobs into fixed integer values. These hashed values are then assigned to subscribed consumers in one of two ways:
910

1011
* *Auto hash* - uses *consistent hashing* to balance range values across available consumers, without requiring manual setup of hash ranges.
11-
* *Sticky hash* - the client manually assigns consumer range values. All hashes within a configured range go to one consumer.
12+
* *Sticky hash* - the client manually assigns consumer range values. All hashes within a configured range go to one consumer.
1213
1314
This document explains how to use Pulsar's Key_Shared subscription model to manage your topic consumption.
1415

@@ -18,28 +19,28 @@ include::partial$subscription-prereq.adoc[]
1819
== Key_Shared subscription example
1920

2021
To try out a Pulsar Key_Shared subscription, add `.subscriptionType(SubscriptionType.Key_Shared)` to the `pulsarConsumer` in `SimplePulsarConsumer.java`. +
21-
You must also tell Pulsar what `keySharedPolicy` this subscription will use. The example below uses the auto-hashing `keySharedPolicy`.
22+
You must also tell Pulsar what `keySharedPolicy` this subscription will use. The example below uses the auto-hashing `keySharedPolicy`.
2223

2324
[source,java]
2425
----
2526
pulsarConsumer = pulsarClient.newConsumer(Schema.JSON(DemoBean.class))
26-
.topic("persistent://"
27-
+ conf.getTenantName() + "/"
28-
+ conf.getNamespace() + "/"
27+
.topic("persistent://"
28+
+ conf.getTenantName() + "/"
29+
+ conf.getNamespace() + "/"
2930
+ conf.getTopicName())
3031
.startMessageIdInclusive()
3132
.subscriptionInitialPosition(SubscriptionInitialPosition.Earliest)
3233
.subscriptionName("SimplePulsarConsumer")
3334
.subscriptionType(SubscriptionType.Key_Shared)
3435
.keySharedPolicy(KeySharedPolicy.autoSplitHashRange())
35-
.subscribe();
36+
.subscribe();
3637
----
3738

38-
Running multiple consumers with auto-hashing balances the messaging load across all available consumers.
39+
Running multiple consumers with auto-hashing balances the messaging load across all available consumers.
3940

4041
=== Manually set stickyHashRange
4142

42-
You can manually set a hash range with `KeySharedPolicy.stickyHashRange()`.
43+
You can manually set a hash range with `KeySharedPolicy.stickyHashRange()`.
4344

4445
To test out sticky hashed Key_Shared subscriptions, you need to first import some additional classes.
4546

@@ -66,9 +67,9 @@ import org.apache.pulsar.client.api.HashingScheme;
6667
----
6768
pulsarProducer = pulsarClient
6869
.newProducer(Schema.JSON(DemoBean.class))
69-
.topic("persistent://"
70-
+ conf.getTenantName() + "/"
71-
+ conf.getNamespace() + "/"
70+
.topic("persistent://"
71+
+ conf.getTenantName() + "/"
72+
+ conf.getNamespace() + "/"
7273
+ conf.getTopicName())
7374
.batcherBuilder(BatcherBuilder.KEY_BASED)
7475
.hashingScheme(HashingScheme.JavaStringHash)
@@ -93,7 +94,7 @@ The confirmation message and a cursor appear to indicate the consumer is ready.
9394
[pulsar-client-io-1-1] INFO org.apache.pulsar.client.impl.ConsumerImpl - [persistent://<tenant_name>/<namespace>/in][SimplePulsarConsumer] Subscribed to topic on <service_url> -- consumer: 0
9495
----
9596

96-
. In a new terminal window, run `SimplePulsarProducer.java` to begin producing messages.
97+
. In a new terminal window, run `SimplePulsarProducer.java` to begin producing messages.
9798
+
9899
[source,bash]
99100
----
@@ -103,15 +104,15 @@ The confirmation message and a cursor appear to indicate the consumer is ready.
103104
[main] INFO com.datastax.pulsar.SimplePulsarProducer - Message 57467766 sent
104105
----
105106

106-
. The consumer begins receiving messages.
107+
. The consumer begins receiving messages.
107108
+
108109
[source,bash]
109110
----
110111
[main] INFO com.datastax.pulsar.SimplePulsarConsumer - Message received: {"show_id":55794190,"cast":"LeBron James, Anthony Davis, Kyrie Irving, Damian Lillard, Klay Thompson...","country":"United States","date_added":"July 16, 2021","description":"NBA superstar LeBron James teams up with Bugs Bunny and the rest of the Looney Tunes for this long-awaited sequel.","director":"Malcolm D. Lee","duration":"120 min","listed_in":"Animation, Adventure, Comedy","rating":"PG","release_year":2021,"title":"Space Jam: A New Legacy","type":"Movie"}
111112
[main] INFO com.datastax.pulsar.SimplePulsarConsumer - Message received: {"show_id":41791865,"cast":"LeBron James, Anthony Davis, Kyrie Irving, Damian Lillard, Klay Thompson...","country":"United States","date_added":"July 16, 2021","description":"NBA superstar LeBron James teams up with Bugs Bunny and the rest of the Looney Tunes for this long-awaited sequel.","director":"Malcolm D. Lee","duration":"120 min","listed_in":"Animation, Adventure, Comedy","rating":"PG","release_year":2021,"title":"Space Jam: A New Legacy","type":"Movie"}
112113
----
113114

114-
. Open a new terminal window and try to run `SimplePulsarConsumer.java`.
115+
. Open a new terminal window and try to run `SimplePulsarConsumer.java`.
115116
+
116117
[source,bash]
117118
----

0 commit comments

Comments
 (0)