diff --git a/docs-src/pulsar-sink-core/modules/ROOT/nav.adoc b/docs-src/pulsar-sink-core/modules/ROOT/nav.adoc index 45fad28..594d34c 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/nav.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/nav.adoc @@ -1,52 +1,52 @@ * Getting Started ** xref:index.adoc[] -*** xref:pulsarSizing.adoc[] -** link:https://github.com/datastax/release-notes/blob/master/DataStax_Apache_Pulsar_Connector_1.4_Release_Notes.md[Pulsar Connector release notes] -** xref:pulsarInstall.adoc[] -** xref:pulsarQuickStart.adoc[] +*** xref:pulsar-sizing.adoc[] +** link:https://github.com/datastax/release-notes/blob/master/DataStax_Apache_pulsar-_Connector_1.4_Release_Notes.md[pulsar- Connector release notes] +** xref:pulsar-install.adoc[] +** xref:pulsar-quickstart.adoc[] * Guides and examples -** Configuration -*** xref:cfgPulsarConfigTasksMax.adoc[] -*** xref:cfgPulsarColumnWritetimeTimestamp.adoc[] -*** xref:cfgPulsarSettingRowLevelTtl.adoc[] -*** xref:cfgPulsarSettingJavaDriverConfig.adoc[] -*** xref:cfgPulsarMapTopicTable.adoc[] -**** xref:cfgPulsarDisplayingTopic.adoc[] -**** xref:cfgPulsarMapKeyPair.adoc[] -**** xref:cfgPulsarMapJson.adoc[] -***** xref:cfgPulsarStringJson.adoc[] -***** xref:cfgPulsarJsonMessageSchema.adoc[] -**** xref:cfgPulsarMapAvroMessages.adoc[] -**** xref:cfgPulsarRecordHeaderToTable.adoc[] -**** xref:cfgPulsarMapUdt.adoc[] -**** xref:cfgPulsarMapMultipleTables.adoc[] -**** xref:cfgPulsarMultTopicToTables.adoc[] -**** xref:cfgPulsarCqlQuery.adoc[] -**** xref:cfgPulsarNowFunction.adoc[] +** Configuration +*** xref:configure-tasks-max.adoc[] +*** xref:configure-column-writetime-timestamp.adoc[] +*** xref:configure-row-level-ttl.adoc[] +*** xref:configure-java-driver-config.adoc[] +*** xref:configure-map-topic-table.adoc[] +**** xref:configure-displaying-topic.adoc[] +**** xref:configure-map-key-pair.adoc[] +**** xref:configure-map-json.adoc[] +***** xref:configure-string-json.adoc[] +***** xref:configure-json-message-schema.adoc[] +**** xref:configure-map-avro-messages.adoc[] +**** xref:configure-record-header-to-table.adoc[] +**** xref:configure-map-udt.adoc[] +**** xref:configure-map-multiple-tables.adoc[] +**** xref:configure-multiple-topics-to-tables.adoc[] +**** xref:configure-cql-query.adoc[] +**** xref:configure-now-function.adoc[] ** Operations -*** xref:opsPulsarAboutOperations.adoc[] -*** xref:opsPulsarScaling.adoc[] -*** xref:opsPulsarChangeSchema.adoc[] -*** xref:opsPulsarRestart.adoc[] -*** xref:opsPulsarDisplayConfig.adoc[] -*** xref:opsPulsarUpdateConfig.adoc[] -*** xref:opsPulsarDeleteConnector.adoc[] -*** xref:opsPulsarConnectorStatus.adoc[] +*** xref:about-operations.adoc[] +*** xref:scaling.adoc[] +*** xref:change-schema.adoc[] +*** xref:restart.adoc[] +*** xref:display-config.adoc[] +*** xref:update-config.adoc[] +*** xref:delete-connector.adoc[] +*** xref:connector-status.adoc[] ** Security -*** xref:secPulsarInternalLdapAuth.adoc[] -** xref:monPulsarConnectMetrics.adoc[] +*** xref:internal-ldap-auth.adoc[] +** xref:connect-metrics.adoc[] ** Troubleshooting -*** xref:trbPulsarTsRecordFailsToWriteToTable.adoc[] -*** xref:trbPulsarTsLargeBatches.adoc[] -*** xref:trbPulsarTsRecordParsingFails.adoc[] -*** xref:trbPulsarTsLoadBalancing.adoc[] +*** xref:record-fails-to-write-to-table.adoc[] +*** xref:large-batches.adoc[] +*** xref:record-parsing-fails.adoc[] +*** xref:load-balancing.adoc[] * Reference -**** xref:cfgRefPulsarConnector.adoc[] -**** xref:cfgRefPulsarDseConnection.adoc[] -**** xref:cfgRefPulsarDseTable.adoc[] -**** xref:cfgRefPulsarDates.adoc[] -**** xref:cfgRefPulsarAuth.adoc[] -***** xref:cfgRefPulsarAuthLdap.adoc[] -**** xref:cfgRefPulsarSsl.adoc[] -**** xref:cfgPulsarConfigureErrorHandling.adoc[] +**** xref:reference-pulsar-connector.adoc[] +**** xref:reference-pulsar-dse-connection.adoc[] +**** xref:reference-pulsar-dse-table.adoc[] +**** xref:reference-pulsar-dates.adoc[] +**** xref:reference-pulsar-auth.adoc[] +***** xref:reference-pulsar-auth-ldap.adoc[] +**** xref:reference-pulsar-ssl.adoc[] +**** xref:configure-error-handling.adoc[] diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarAboutOperations.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/about-operations.adoc similarity index 93% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarAboutOperations.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/about-operations.adoc index 27fe635..2477c3e 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarAboutOperations.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/about-operations.adoc @@ -1,5 +1,5 @@ = About operating and maintaining the DataStax Connector - +:page-aliases: opsPulsarAboutOperations.adoc :page-tag: pulsar-connector,admin,manage,pulsar To operate and maintain the DataStax Apache Pulsar™ Connector use the Apache Pulsar™ adminstration tool, `pulsar-admin` in concert with the Apache Pulsar™ REST Administration API. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarAddField.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/add-field.adoc similarity index 78% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarAddField.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/add-field.adoc index a09aef9..c0bc6c7 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarAddField.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/add-field.adoc @@ -1,11 +1,11 @@ = Adding a field to a Pulsar Topic - +:page-aliases: opsPulsarAddField.adoc :page-tag: pulsar-connector,dev,develop,pulsar Use these steps to write a new field to an existing xref:index.adoc#supported-databases[supported database] table column. . In the DataStax Apache Pulsar™ Connector configuration file, add the field-to-column definition. -See xref:cfgPulsarMapTopicTable.adoc[]. +See xref:configure-map-topic-table.adoc[]. . Update the configuration with the new settings. -See xref:opsPulsarUpdateConfig.adoc[]. +See xref:update-config.adoc[]. . Start the Pulsar producer that had the schema change. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarAddTableColumn.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/add-table-column.adoc similarity index 90% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarAddTableColumn.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/add-table-column.adoc index 6554be4..fb6127f 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarAddTableColumn.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/add-table-column.adoc @@ -1,5 +1,5 @@ = Adding a column to the database table - +:page-aliases: opsPulsarAddTableColumn.adoc :page-tag: pulsar-connector,dev,develop,pulsar Add a column to the xref:index.adoc#supported-databases[supported database] table to synchronize an existing field from a Pulsar topic. @@ -10,4 +10,4 @@ Add a column to the xref:index.adoc#supported-databases[supported database] tabl IMPORTANT: Data synchronization may fail if the schema change is inconsistent across the entire cluster. . In the DataStax Apache Pulsar™ Connector configuration file, add the field-to-column definition. -See xref:cfgPulsarMapTopicTable.adoc[]. +See xref:configure-map-topic-table.adoc[]. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapTopicTable.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapTopicTable.adoc deleted file mode 100644 index 09a59bb..0000000 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapTopicTable.adoc +++ /dev/null @@ -1,21 +0,0 @@ -= Mapping pulsar topics to database tables - -:page-tag: pulsar-connector,dev,develop,pulsar - -DataStax Apache Pulsar™ Connector has a simple yet powerful syntax for mapping fields from a Pulsar record to columns in a xref:index.adoc#supported-databases[supported database] table. An instance of the DataStax Apache Pulsar™ Connector can capture multiple topics and supports mapping a single topic to one or more tables. - -== DataStax Apache Pulsar™ Connector topic mapping - -DataStax Apache Pulsar™ Connector supports topic-to-table mappings with xref:index.adoc#supported-databases[supported databases]: - -* xref:cfgPulsarDisplayingTopic.adoc[] Display messages to determine the data structure of the topic messages. -* xref:cfgPulsarMapKeyPair.adoc[] Create a topic-table map for Pulsar messages that only contain a key and value in each record. -* xref:cfgPulsarMapJson.adoc[] For JSON fields, map individual fields in the structure to columns. -* xref:cfgPulsarMapAvroMessages.adoc[] Supports mapping individual fields from a Avro format field. -* xref:cfgPulsarRecordHeaderToTable.adoc[] Extract values from Pulsar record header and write to the database table. -* xref:cfgPulsarMapUdt.adoc[] Write complex types directly into User-defined Types (UDT). -* xref:cfgPulsarMapMultipleTables.adoc[] Ingest a single topic into multiple tables using a single connector instance. -* xref:cfgPulsarMultTopicToTables.adoc[] Ingest multiple topics and write to different tables using a single connector instance. -* xref:cfgPulsarSelectiveUpdateMapsAndUdts.adoc[] Selectively update maps and UDTs based on Pulsar fields. -* xref:cfgPulsarCqlQuery.adoc[] Provide CQL queries when new record arrives on the Pulsar topic. -* xref:cfgPulsarNowFunction.adoc[] You can use the now() function in mappings. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/change-schema.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/change-schema.adoc new file mode 100644 index 0000000..e21e33b --- /dev/null +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/change-schema.adoc @@ -0,0 +1,10 @@ += Changing the topic or table schema +:page-aliases: opsPulsarChangeSchema.adoc +:page-tag: pulsar-connector,dev,develop,pulsar + +How to update the DataStax Apache Pulsar™ Connector when schema changes are required. + +* xref:add-field.adoc[] Add a field Pulsar Topic schema. +* xref:remove-field.adoc[] Remove a field from a Pulsar Topic. +* xref:add-table-column.adoc[] Add a column to the database table to synchronize an existing field from a Pulsar topic. +* xref:remove-column.adoc[] Remove a database column from the schema. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarColumnWritetimeTimestamp.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-column-writetime-timestamp.adoc similarity index 96% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarColumnWritetimeTimestamp.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-column-writetime-timestamp.adoc index 32d1196..a949780 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarColumnWritetimeTimestamp.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-column-writetime-timestamp.adoc @@ -1,5 +1,5 @@ = Specify writetime timestamp column - +:page-aliases: cfgPulsarColumnWritetimeTimestamp.adoc :page-tag: pulsar-connector,dev,develop,pulsar In the Apache Pulsar™ topic mapping, you can optionally specify which column should be used to note the timestamp when the Pulsar-produced record was inserted into any of the xref:index.adoc#supported-databases[supported databases]. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarCqlQuery.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-cql-query.adoc similarity index 97% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarCqlQuery.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-cql-query.adoc index b8759ec..12284c3 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarCqlQuery.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-cql-query.adoc @@ -1,5 +1,5 @@ = Provide CQL queries in mappings - +:page-aliases: cfgPulsarCqlQuery.adoc :page-tag: pulsar-connector,dev,develop,pulsar In the Pulsar topic mapping, you can optionally provide a CQL query that should run when the new record arrives on the Pulsar topic. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarDisplayingTopic.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-displaying-topic.adoc similarity index 94% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarDisplayingTopic.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-displaying-topic.adoc index a589d19..4e9cba6 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarDisplayingTopic.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-displaying-topic.adoc @@ -1,5 +1,5 @@ = Determining topic data structure - +:page-aliases: cfgPulsarDisplayingTopic.adoc :page-tag: pulsar-connector,dev,develop,pulsar You can use the Apache Pulsar™ `pulsar-client` command line client to examine the structure of Pulsar messages for a topic. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarConfigureErrorHandling.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-error-handling.adoc similarity index 94% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarConfigureErrorHandling.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-error-handling.adoc index 9178ad9..3ea8455 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarConfigureErrorHandling.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-error-handling.adoc @@ -1,5 +1,5 @@ = Configure error handling - +:page-aliases: cfgPulsarConfigureErrorHandling.adoc :page-tag: pulsar-connector,admin,manage,pulsar The DataStax Apache Pulsar™ Connector provides an optional parameter, `ignoreErrors`. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarSettingJavaDriverConfig.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-java-driver-config.adoc similarity index 95% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarSettingJavaDriverConfig.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-java-driver-config.adoc index d258762..91dacaa 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarSettingJavaDriverConfig.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-java-driver-config.adoc @@ -1,5 +1,5 @@ = Pass Pulsar Connector settings directly to the DataStax Java driver - +:page-aliases: cfgPulsarSettingJavaDriverConfig.adoc :page-tag: pulsar-connector,dev,develop,pulsar,java In your DataStax Apache Pulsar™ Connector configuration file, you can directly pass settings to the DataStax Java driver by using the `datastax-java-driver` prefix. @@ -17,7 +17,7 @@ The following table identifies functionally equivalent DataStax Apache Pulsar Co NOTE: If you define both in your configuration, the Pulsar Connector setting take precedence over the `datastax-java-driver.property-name`. If you do not provide either in your configuration, Pulsar Connector defaults are in effect. -For information about the Java properties, refer to the link:https://docs.datastax.com/en/developer/java-driver-dse/2.3/manual/core/configuration/[DataStax Java driver documentation.] For information about the Pulsar Connector settings, refer to xref:cfgRefPulsarDseConnection.adoc[]. +For information about the Java properties, refer to the link:https://docs.datastax.com/en/developer/java-driver-dse/2.3/manual/core/configuration/[DataStax Java driver documentation.] For information about the Pulsar Connector settings, refer to xref:reference-pulsar-dse-connection.adoc[]. |=== | DataStax Apache Pulsar Connector setting | Using datastax-java-driver prefix diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarJsonMessageSchema.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-json-message-schema.adoc similarity index 91% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarJsonMessageSchema.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-json-message-schema.adoc index c27d609..87b7ba7 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarJsonMessageSchema.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-json-message-schema.adoc @@ -1,5 +1,5 @@ = Mapping JSON messages - +:page-aliases: cfgPulsarJsonMessageSchema.adoc :page-tag: pulsar-connector,dev,develop,pulsar The DataStax Apache Pulsar™ Connector supports mapping JSON messages with or without a schema. @@ -31,7 +31,7 @@ See the https://github.com/datastax/pulsar-examples/tree/master/producers/src/ma . Set up the xref:index.adoc#supported-databases[supported database] table. . Create the keyspace. -Ensure that keyspace is replicated to a datacenter that is set in the DataStax Apache Pulsar Connector xref:cfgRefPulsarDseConnection.adoc#contactPoints[contactPoints] parameter. +Ensure that keyspace is replicated to a datacenter that is set in the DataStax Apache Pulsar Connector xref:reference-pulsar-dse-connection.adoc#contactPoints[contactPoints] parameter. For example, create the `stocks_keyspace`: + [source,language-bash] @@ -67,9 +67,9 @@ nodetool describering -- keyspace\_name . In the DataStax Apache Pulsar™ Connector configuration file: -.. Add the topic name to xref:cfgRefPulsarDseTable.adoc#topics[topics]. -.. Define the topic-to-table map xref:cfgRefPulsarDseTable.adoc#prefix[prefix]. -.. Define the xref:cfgRefPulsarDseTable.adoc#mapping[field-to-column map]. +.. Add the topic name to xref:reference-pulsar-dse-table.adoc#topics[topics]. +.. Define the topic-to-table map xref:reference-pulsar-dse-table.adoc#prefix[prefix]. +.. Define the xref:reference-pulsar-dse-table.adoc#mapping[field-to-column map]. + Example configurations for `stocks_topic` to `stocks_table` using the minimum required settings: + diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapAvroMessages.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-avro-messages.adoc similarity index 97% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapAvroMessages.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-avro-messages.adoc index aa6863c..e6a9254 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapAvroMessages.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-avro-messages.adoc @@ -1,5 +1,5 @@ = Mapping Avro messages - +:page-aliases: cfgPulsarMapAvroMessages.adoc :page-tag: pulsar-connector,dev,develop,pulsar,avro The DataStax Apache Pulsar™ Connector supports mapping individual fields from https://avro.apache.org/docs/1.8.1/gettingstartedjava.html[Avro] formatted key or values. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapJson.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-json.adoc similarity index 50% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapJson.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-json.adoc index 0d299ac..0e9dd81 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapJson.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-json.adoc @@ -1,10 +1,10 @@ = Mapping a message that contain JSON fields - +:page-aliases: cfgPulsarMapJson.adoc :page-tag: pulsar-connector,dev,develop,pulsar,json When the data format for the key or value is JSON, the DataStax Apache Pulsar™ Connector mapping can include individual fields in the JSON structure. NOTE: JSON records in Pulsar can also have a schema associated with them. -* xref:cfgPulsarStringJson.adoc[] When the data format for the Pulsar key or value is JSON, individual fields of that JSON structure can be specified in the connector mapping. -* xref:cfgPulsarJsonMessageSchema.adoc[] Supports mapping JSON messages with or without a schema. +* xref:configure-string-json.adoc[] When the data format for the Pulsar key or value is JSON, individual fields of that JSON structure can be specified in the connector mapping. +* xref:configure-json-message-schema.adoc[] Supports mapping JSON messages with or without a schema. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapKeyPair.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-key-pair.adoc similarity index 84% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapKeyPair.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-key-pair.adoc index a6323b1..d84f6fd 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapKeyPair.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-key-pair.adoc @@ -1,5 +1,5 @@ = Mapping basic messages to table columns - +:page-aliases: cfgPulsarMapKeyPair.adoc :page-tag: pulsar-connector,dev,develop,pulsar When messages are created using a Basic or primitive serializer, the message contains a key-value pair. Map the key and value to table columns. Ensure that the data types of the message field are compatible with the data type of the target column. @@ -17,7 +17,7 @@ Null values are not allowed in PK columns. . Set up the xref:index.adoc#supported-databases[supported database] table. . Create the keyspace. -Ensure that keyspace is replicated to a datacenter that is set in the DataStax Apache Pulsar Connector xref:cfgRefPulsarDseConnection.adoc#contactPoints[contactPoints] parameter. +Ensure that keyspace is replicated to a datacenter that is set in the DataStax Apache Pulsar Connector xref:reference-pulsar-dse-connection.adoc#contactPoints[contactPoints] parameter. For example, create the world_keyspace: + [source,language-bash] @@ -43,9 +43,9 @@ nodetool describering -- keyspace_name ---- . In the DataStax Apache Pulsar™ Connector configuration file: -.. Add the topic name to xref:cfgRefPulsarDseTable.adoc#topics[topics]. -.. Define the topic-to-table map xref:cfgRefPulsarDseTable.adoc#prefix[prefix]. -.. Define the xref:cfgRefPulsarDseTable.adoc#mapping[field-to-column map]. +.. Add the topic name to xref:reference-pulsar-dse-table.adoc#topics[topics]. +.. Define the topic-to-table map xref:reference-pulsar-dse-table.adoc#prefix[prefix]. +.. Define the xref:reference-pulsar-dse-table.adoc#mapping[field-to-column map]. + Example configurations for `world_topic` to `world_table` using the minimum required settings: + diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapMultipleTables.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-multiple-tables.adoc similarity index 98% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapMultipleTables.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-multiple-tables.adoc index e690e44..9c2e0ea 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapMultipleTables.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-multiple-tables.adoc @@ -1,5 +1,5 @@ = Mapping a topic to multiple tables - +:page-aliases: cfgPulsarMapMultipleTables.adoc :page-tag: pulsar-connector,dev,develop,pulsar The DataStax Apache Pulsar™ Connector allows for mapping a single topic to multiple tables for a single connector instance. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-topic-table.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-topic-table.adoc new file mode 100644 index 0000000..7aa9c63 --- /dev/null +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-topic-table.adoc @@ -0,0 +1,21 @@ += Mapping pulsar topics to database tables +:page-aliases: cfgPulsarMapTopicTable.adoc +:page-tag: pulsar-connector,dev,develop,pulsar + +DataStax Apache Pulsar™ Connector has a simple yet powerful syntax for mapping fields from a Pulsar record to columns in a xref:index.adoc#supported-databases[supported database] table. An instance of the DataStax Apache Pulsar™ Connector can capture multiple topics and supports mapping a single topic to one or more tables. + +== DataStax Apache Pulsar™ Connector topic mapping + +DataStax Apache Pulsar™ Connector supports topic-to-table mappings with xref:index.adoc#supported-databases[supported databases]: + +* xref:configure-displaying-topic.adoc[] Display messages to determine the data structure of the topic messages. +* xref:configure-map-key-pair.adoc[] Create a topic-table map for Pulsar messages that only contain a key and value in each record. +* xref:configure-map-json.adoc[] For JSON fields, map individual fields in the structure to columns. +* xref:configure-map-avro-messages.adoc[] Supports mapping individual fields from a Avro format field. +* xref:configure-record-header-to-table.adoc[] Extract values from Pulsar record header and write to the database table. +* xref:configure-map-udt.adoc[] Write complex types directly into User-defined Types (UDT). +* xref:configure-map-multiple-tables.adoc[] Ingest a single topic into multiple tables using a single connector instance. +* xref:configure-multiple-topics-to-tables.adoc[] Ingest multiple topics and write to different tables using a single connector instance. +* xref:configure-selective-update-maps-and-udts.adoc[] Selectively update maps and UDTs based on Pulsar fields. +* xref:configure-cql-query.adoc[] Provide CQL queries when new record arrives on the Pulsar topic. +* xref:configure-now-function.adoc[] You can use the now() function in mappings. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapUdt.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-udt.adoc similarity index 99% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapUdt.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-udt.adoc index af72c18..1d1299e 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMapUdt.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-map-udt.adoc @@ -1,5 +1,5 @@ = Mapping messages to table that has a User Defined Type - +:page-aliases: cfgPulsarMapUdt.adoc :page-tag: pulsar-connector,dev,develop,pulsar The DataStax Apache Pulsar™ Connector processes JSON, Avro, and complex types directly into a link:https://docs.datastax.com/en/dse/6.8/cql/cql/cql_using/useAlterType.html[user-defined type] column in the xref:index.adoc#supported-databases[supported database]. The Pulsar message embedded field names and the UDT field names must match. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMultTopicToTables.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-multiple-topics-to-tables.adoc similarity index 98% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMultTopicToTables.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-multiple-topics-to-tables.adoc index 087e364..e3f03cc 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarMultTopicToTables.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-multiple-topics-to-tables.adoc @@ -1,5 +1,5 @@ = Multiple topics to multiple tables - +:page-aliases: cfgPulsarMultTopicToTables.adoc :page-tag: pulsar-connector,dev,develop,pulsar The DataStax Apache Pulsar™ Connector allows for mapping multiple topics to multiple tables in a single connector instance. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarNowFunction.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-now-function.adoc similarity index 91% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarNowFunction.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-now-function.adoc index e522b47..adcdc48 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarNowFunction.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-now-function.adoc @@ -1,5 +1,5 @@ = The now() function in mappings - +:page-aliases: cfgPulsarNowFunction.adoc :page-tag: pulsar-connector,dev,develop,pulsar In the Apache Pulsar™ topic mapping, you can use the ``now()``function. @@ -38,7 +38,7 @@ In the example above, each occurrence of `now()` is evaluated and return a diffe == Behavior of now() with DELETE records -When the value of a record arrives to DataStax Apache Pulsar Connector instance, it could be a `DELETE`, assuming `deletesEnabled` is `true`; see xref:cfgRefPulsarDseTable.adoc#deletesEnabled[deletesEnabled]. +When the value of a record arrives to DataStax Apache Pulsar Connector instance, it could be a `DELETE`, assuming `deletesEnabled` is `true`; see xref:reference-pulsar-dse-table.adoc#deletesEnabled[deletesEnabled]. For example, consider a case where a user has mapping and used the `now()` function: diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarRecordHeaderToTable.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-record-header-to-table.adoc similarity index 92% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarRecordHeaderToTable.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-record-header-to-table.adoc index d3737f7..3534951 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarRecordHeaderToTable.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-record-header-to-table.adoc @@ -1,5 +1,5 @@ = Extract Pulsar record header values - +:page-aliases: cfgPulsarRecordHeaderToTable.adoc :page-tag: pulsar-connector,dev,develop,pulsar In the Apache Pulsar™ topic mapping, you can extract values from the message properties, and write the values to a xref:index.adoc#supported-databases[supported database table]. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarSettingRowLevelTtl.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-row-level-ttl.adoc similarity index 97% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarSettingRowLevelTtl.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-row-level-ttl.adoc index d323d63..f5317da 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarSettingRowLevelTtl.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-row-level-ttl.adoc @@ -1,5 +1,5 @@ = Setting row-level TTL values from Pulsar fields - +:page-aliases: cfgPulsarSettingRowLevelTtl.adoc :page-tag: pulsar-connector,dev,develop,pulsar In the Apache Pulsar™ topic mapping, you can optionally specify which column should be used as the ttl (time-to-live) of the record being inserted into the database table. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarSelectiveUpdateMapsAndUdts.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-selective-update-maps-and-udts.adoc similarity index 88% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarSelectiveUpdateMapsAndUdts.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-selective-update-maps-and-udts.adoc index 8bada83..75cf3ae 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarSelectiveUpdateMapsAndUdts.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-selective-update-maps-and-udts.adoc @@ -1,5 +1,5 @@ = Selectively update maps and UDTs based on Pulsar fields - +:page-aliases: cfgPulsarSelectiveUpdateMapsAndUdts.adoc :page-tag: pulsar-connector,dev,develop,pulsar In the Apache Pulsar™ topic mapping, you can selectively update maps and User Defined Types (UDTs) based on whether Pulsar fields have present values. That is, any value other than `null`. @@ -41,5 +41,5 @@ UPDATE ks.table set udtColNotFrozen.udtmem1=:udtcol1, udtColNotFrozen.udtmem2=:u When the new record arrives, DataStax Apache Pulsar™ Connector sets only the not null fields; it will not override fields of UDT that have a null value (or are not present) in the Pulsar record, assuming `nullToUnset` is set to `true`. The behavior of the Update versus Override is defined via the `nullToUnset` parameter. -See its description in xref:cfgRefPulsarDseTable.adoc#nullToUnset[nullToUnset]. -For related information about Pulsar maps and UDTs, see xref:cfgPulsarMapUdt.adoc[]. +See its description in xref:reference-pulsar-dse-table.adoc#nullToUnset[nullToUnset]. +For related information about Pulsar maps and UDTs, see xref:configure-map-udt.adoc[]. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarStringJson.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-string-json.adoc similarity index 95% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarStringJson.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-string-json.adoc index da84211..5afa8c5 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarStringJson.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-string-json.adoc @@ -1,5 +1,5 @@ = Mapping a message that contains both basic and JSON fields - +:page-aliases: cfgPulsarStringJson.adoc :page-tag: pulsar-connector,dev,develop,pulsar,json When the data format for the message key or value is JSON, the DataStax Apache Pulsar™ connector mapping can include individual fields in the JSON structure. @@ -62,7 +62,7 @@ Null values are not allowed in PK columns. . Set up the xref:index.adoc#supported-databases[supported database] table. . Create the keyspace. -Ensure that keyspace is replicated to a datacenter that is set in the DataStax Apache Pulsar Connector xref:cfgRefPulsarDseConnection.adoc#contactPoints[contactPoints] parameter. +Ensure that keyspace is replicated to a datacenter that is set in the DataStax Apache Pulsar Connector xref:reference-pulsar-dse-connection.adoc#contactPoints[contactPoints] parameter. For example, create the `stocks_keyspace`: + [source,language-bash] diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarConfigTasksMax.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-tasks-max.adoc similarity index 60% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarConfigTasksMax.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/configure-tasks-max.adoc index 1e97145..c4978d9 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgPulsarConfigTasksMax.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/configure-tasks-max.adoc @@ -1,5 +1,5 @@ = Configuring parallelism - +:page-aliases: cfgPulsarConfigTasksMax.adoc :page-tag: pulsar-connector,admin,manage,pulsar -For information on configuring parallelism for the DataStax Apache Pulsar™ Connector, see xref:opsPulsarScaling.adoc[]. +For information on configuring parallelism for the DataStax Apache Pulsar™ Connector, see xref:scaling.adoc[]. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/monPulsarConnectMetrics.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/connect-metrics.adoc similarity index 99% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/monPulsarConnectMetrics.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/connect-metrics.adoc index e387772..9577be6 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/monPulsarConnectMetrics.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/connect-metrics.adoc @@ -1,5 +1,5 @@ = DataStax Apache Pulsar™ Connector metrics - +:page-aliases: monPulsarConnectMetrics.adoc :page-tag: pulsar-connector,admin,manage,pulsar You can view DataStax Apache Pulsar™ Connector metrics in raw Prometheus format at the Apache Pulsar metrics endpoint: diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarConnectorStatus.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/connector-status.adoc similarity index 95% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarConnectorStatus.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/connector-status.adoc index 031617b..bc3d974 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarConnectorStatus.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/connector-status.adoc @@ -1,5 +1,5 @@ = Getting the DataStax Connector status - +:page-aliases: opsPulsarConnectorStatus.adoc :page-tag: pulsar-connector,dev,develop,admin,manage,pulsar Use the Apache Pulsar™ administration tool `sinks status` command to display the status of the DataStax Apache Pulsar™ Connector. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarDeleteConnector.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/delete-connector.adoc similarity index 98% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarDeleteConnector.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/delete-connector.adoc index a0e25f8..2c5d83a 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarDeleteConnector.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/delete-connector.adoc @@ -1,5 +1,5 @@ = Deleting the DataStax Apache Pulsar™ Connector - +:page-aliases: opsPulsarDeleteConnector.adoc :page-tag: pulsar-connector,admin,manage,pulsar To completely remove the DataStax Apache Pulsar™ Connector: diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarDisplayConfig.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/display-config.adoc similarity index 98% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarDisplayConfig.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/display-config.adoc index c956298..3141862 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarDisplayConfig.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/display-config.adoc @@ -1,5 +1,5 @@ = Displaying the DataStax Apache Pulsar™ Connector configuration - +:page-aliases: opsPulsarDisplayConfig.adoc :page-tag: pulsar-connector,dev,develop,admin,manage,pulsar Use the Apache Pulsar™ administration tool sinks command to retrieve detailed information on a Pulsar sink: diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/index.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/index.adoc index e2f0fd1..7c52cd1 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/index.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/index.adoc @@ -34,20 +34,20 @@ Ingest data from Pulsar topics with records in the following data structures: == Getting started -* xref:pulsarQuickStart.adoc[] A quick introduction connecting Pulsar to DataStax Enterprise using the DataStax Pulsar Connector and sending simple key/value messages using the Pulsar client utility. +* xref:pulsar-quickstart.adoc[] A quick introduction connecting Pulsar to DataStax Enterprise using the DataStax Pulsar Connector and sending simple key/value messages using the Pulsar client utility. == Advanced message mapping topics If you're already familiar with Apache Pulsar™ and DSE/Astra/Cassandra® see the following advanced message mapping topics: -* xref:cfgPulsarDisplayingTopic.adoc[] Display messages to determine the data structure of the topic messages. -* xref:cfgPulsarMapKeyPair.adoc[] Create a topic-table map for Pulsar messages that only contain a key and value in each record. -* xref:cfgPulsarMapJson.adoc[] For JSON fields, map individual fields in the structure to columns. -* xref:cfgPulsarMapAvroMessages.adoc[] Supports mapping individual fields from a Avro format field. -* xref:cfgPulsarRecordHeaderToTable.adoc[] Extract values from Pulsar record header and write to the database table. -* xref:cfgPulsarMapUdt.adoc[] Write complex types directly into User-defined Types (UDT). -* xref:cfgPulsarMapMultipleTables.adoc[] Ingest a single topic into multiple tables using a single connector instance. -* xref:cfgPulsarMultTopicToTables.adoc[] Ingest multiple topics and write to different tables using a single connector instance. -* xref:cfgPulsarSelectiveUpdateMapsAndUdts.adoc[] Selectively update maps and UDTs based on Pulsar fields. -* xref:cfgPulsarCqlQuery.adoc[] Provide CQL queries when new record arrives on the Pulsar topic. -* xref:cfgPulsarNowFunction.adoc[] You can use the now() function in mappings. +* xref:configure-displaying-topic.adoc[] Display messages to determine the data structure of the topic messages. +* xref:configure-map-key-pair.adoc[] Create a topic-table map for Pulsar messages that only contain a key and value in each record. +* xref:configure-map-json.adoc[] For JSON fields, map individual fields in the structure to columns. +* xref:configure-map-avro-messages.adoc[] Supports mapping individual fields from a Avro format field. +* xref:configure-record-header-to-table.adoc[] Extract values from Pulsar record header and write to the database table. +* xref:configure-map-udt.adoc[] Write complex types directly into User-defined Types (UDT). +* xref:configure-map-multiple-tables.adoc[] Ingest a single topic into multiple tables using a single connector instance. +* xref:configure-multiple-topics-to-tables.adoc[] Ingest multiple topics and write to different tables using a single connector instance. +* xref:configure-selective-update-maps-and-udts.adoc[] Selectively update maps and UDTs based on Pulsar fields. +* xref:configure-cql-query.adoc[] Provide CQL queries when new record arrives on the Pulsar topic. +* xref:configure-now-function.adoc[] You can use the now() function in mappings. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/secPulsarInternalLdapAuth.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/internal-ldap-auth.adoc similarity index 95% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/secPulsarInternalLdapAuth.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/internal-ldap-auth.adoc index 1979624..bcf6087 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/secPulsarInternalLdapAuth.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/internal-ldap-auth.adoc @@ -1,5 +1,5 @@ = Using internal or LDAP authentication - +:page-aliases: secPulsarInternalLdapAuth.adoc :page-tag: pulsar-connector,security,secure,pulsar,ldap Authenticate the DataStax Apache Pulsar™ Connector session using internal or LDAP authentication. @@ -19,4 +19,4 @@ For details, refer to link:https://docs.datastax.com/en/security/6.8/security/se . Internal and LDAP schemes can also be used for role management. Refer to the link:https://docs.datastax.com/en/dse/6.8/dse-admin/datastax_enterprise/config/configDseYaml.html#configDseYaml__role_management_options[role management] options. . Set the relevant auth parameters in dse.yaml for `auth.provider`, `auth.username` (the login role name or LDAP username), and `auth.password`. -See xref:cfgRefPulsarAuthLdap.adoc#ldap-parameters[parameters]. +See xref:reference-pulsar-auth-ldap.adoc#ldap-parameters[parameters]. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/trbPulsarTsLargeBatches.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/large-batches.adoc similarity index 93% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/trbPulsarTsLargeBatches.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/large-batches.adoc index df0ba76..cc83431 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/trbPulsarTsLargeBatches.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/large-batches.adoc @@ -1,5 +1,5 @@ = Writing fails because of mutation size - +:page-aliases: trbPulsarTsLargeBatches.adoc :page-tag: pulsar-connector,planner,plan,pulsar The DataStax Apache Pulsar™ Connector collects records to write to the DataStax Enterprise (DSE) database in CQL link:https://docs.datastax.com/en/dse/6.8/cql/cql/cql_reference/cql_commands/cqlBatch.html[BATCH] commands. @@ -29,7 +29,7 @@ If you set `max_mutation_size_in_kb` explicitly, also set `commitlog_segment_siz `2 * max_mutation_size_in_kb / 1024` -You can decrease the batch size by lowering the number of records collected in each batch (xref:cfgRefPulsarDseConnection.adoc#maxNumberOfRecordsInBatch[maxNumberOfRecordsInBatch]). +You can decrease the batch size by lowering the number of records collected in each batch (xref:reference-pulsar-dse-connection.adoc#maxNumberOfRecordsInBatch[maxNumberOfRecordsInBatch]). The default is 32. If you cannot decrease the size of your batches, test whether increases to `max_mutation_size_in_kb` and `commitlog_segment_size_in_mb` result in batches completing successfully, without consuming too much RAM on the partition's node. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/trbPulsarTsLoadBalancing.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/load-balancing.adoc similarity index 74% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/trbPulsarTsLoadBalancing.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/load-balancing.adoc index 18807ee..187cdeb 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/trbPulsarTsLoadBalancing.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/load-balancing.adoc @@ -1,8 +1,8 @@ = Loading balancing datacenter is not specified - +:page-aliases: trbPulsarTsLoadBalancing.adoc :page-tag: pulsar-connector,admin,manage,pulsar -The following error message occurs when the load balancing datacenter (xref:cfgRefPulsarDseConnection.adoc#loadBalancing[loadBalancing.localDc]) is not set in the configuration file. +The following error message occurs when the load balancing datacenter (xref:reference-pulsar-dse-connection.adoc#loadBalancing[loadBalancing.localDc]) is not set in the configuration file. [source,no-highlight] ---- @@ -16,5 +16,5 @@ org.apache.pulsar.common.config.ConfigException: Invalid value [127.0.0.2] for The connector requires that you set the `loadBalancing.localDc`. DataStax Enterprise datacenter names are case-sensitive. -Check the datacenter name in the cluster by running link:https://docs.datastax.com/en/dse/6.7/dse-admin/datastax_enterprise/tools/nodetool/toolsStatus.html[nodetool status] and verify that xref:cfgRefPulsarDseConnection.adoc#loadBalancing[loadBalancing.localDc] exactly matches. +Check the datacenter name in the cluster by running link:https://docs.datastax.com/en/dse/6.7/dse-admin/datastax_enterprise/tools/nodetool/toolsStatus.html[nodetool status] and verify that xref:reference-pulsar-dse-connection.adoc#loadBalancing[loadBalancing.localDc] exactly matches. Datacenter names are case sensitive. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarChangeSchema.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarChangeSchema.adoc deleted file mode 100644 index 1986802..0000000 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarChangeSchema.adoc +++ /dev/null @@ -1,10 +0,0 @@ -= Changing the topic or table schema - -:page-tag: pulsar-connector,dev,develop,pulsar - -How to update the DataStax Apache Pulsar™ Connector when schema changes are required. - -* xref:opsPulsarAddField.adoc[] Add a field Pulsar Topic schema. -* xref:opsPulsarRemoveField.adoc[] Remove a field from a Pulsar Topic. -* xref:opsPulsarAddTableColumn.adoc[] Add a column to the database table to synchronize an existing field from a Pulsar topic. -* xref:opsPulsarRemoveColumn.adoc[] Remove a database column from the schema. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/pulsarInstall.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/pulsar-install.adoc similarity index 98% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/pulsarInstall.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/pulsar-install.adoc index b3ffd62..98a5ab6 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/pulsarInstall.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/pulsar-install.adoc @@ -1,5 +1,5 @@ = Installing DataStax Apache Pulsar™ Connector - +:page-aliases: pulsarInstall.adoc :page-tag: pulsar-connector,admin,dev,install,pulsar Install DataStax Apache Pulsar™ Connector from the DataStax distribution tar file using an account that has write access to the Pulsar configuration directory. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/pulsarQuickStart.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/pulsar-quickstart.adoc similarity index 82% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/pulsarQuickStart.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/pulsar-quickstart.adoc index b3a6059..326b6b8 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/pulsarQuickStart.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/pulsar-quickstart.adoc @@ -1,5 +1,5 @@ = Pulsar Connector single instance quick start - +:page-aliases: pulsarQuickStart.adoc :page-tag: pulsar-connector,dev,admin,planner,quickstart,pulsar This quick start will walk you through the simplest possible configuration connecting a standalone Apache Pulsar™ instance to a single standalone DataStax Enterprise (DSE) 6.8 node. @@ -214,14 +214,14 @@ cqlsh> SELECT * FROM pulsar_qs.pulsar_kv; Now that you've run through a simple end-to-end configuration, you can start experimenting with more complicated Pulsar mappings and DSE schemas. For more details, see the following topics: -* xref:cfgPulsarDisplayingTopic.adoc[] Display messages to determine the data structure of the topic messages. -* xref:cfgPulsarMapKeyPair.adoc[] Create a topic-table map for Pulsar messages that only contain a key and value in each record. -* xref:cfgPulsarMapJson.adoc[] For JSON fields, map individual fields in the structure to columns. -* xref:cfgPulsarMapAvroMessages.adoc[] Supports mapping individual fields from a Avro format field. -* xref:cfgPulsarRecordHeaderToTable.adoc[] Extract values from Pulsar record header and write to the database table. -* xref:cfgPulsarMapUdt.adoc[] Write complex types directly into User-defined Types (UDT). -* xref:cfgPulsarMapMultipleTables.adoc[] Ingest a single topic into multiple tables using a single connector instance. -* xref:cfgPulsarMultTopicToTables.adoc[] Ingest multiple topics and write to different tables using a single connector instance. -* xref:cfgPulsarSelectiveUpdateMapsAndUdts.adoc[] Selectively update maps and UDTs based on Pulsar fields. -* xref:cfgPulsarCqlQuery.adoc[] Provide CQL queries when new record arrives on the Pulsar topic. -* xref:cfgPulsarNowFunction.adoc[] You can use the now() function in mappings. +* xref:configure-displaying-topic.adoc[] Display messages to determine the data structure of the topic messages. +* xref:configure-map-key-pair.adoc[] Create a topic-table map for Pulsar messages that only contain a key and value in each record. +* xref:configure-map-json.adoc[] For JSON fields, map individual fields in the structure to columns. +* xref:configure-map-avro-messages.adoc[] Supports mapping individual fields from a Avro format field. +* xref:configure-record-header-to-table.adoc[] Extract values from Pulsar record header and write to the database table. +* xref:configure-map-udt.adoc[] Write complex types directly into User-defined Types (UDT). +* xref:configure-map-multiple-tables.adoc[] Ingest a single topic into multiple tables using a single connector instance. +* xref:configure-multiple-topics-to-tables.adoc[] Ingest multiple topics and write to different tables using a single connector instance. +* xref:configure-selective-update-maps-and-udts.adoc[] Selectively update maps and UDTs based on Pulsar fields. +* xref:configure-cql-query.adoc[] Provide CQL queries when new record arrives on the Pulsar topic. +* xref:configure-now-function.adoc[] You can use the now() function in mappings. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/pulsarSizing.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/pulsar-sizing.adoc similarity index 97% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/pulsarSizing.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/pulsar-sizing.adoc index b0f70a4..47f08ed 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/pulsarSizing.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/pulsar-sizing.adoc @@ -1,5 +1,5 @@ = System requirements - +:page-aliases: pulsarSizing.adoc :page-tag: pulsar-connector,planner,plan,pulsar The system requirements for DataStax Apache Pulsar™ Connector depends on the workload and network capacity. The factors include characteristics of the Pulsar topic and the cluster data models and volume. DataStax recommends testing with realistic data flows before committing to an instance type for the connector. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/trbPulsarTsRecordFailsToWriteToTable.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/record-fails-to-write-to-table.adoc similarity index 93% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/trbPulsarTsRecordFailsToWriteToTable.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/record-fails-to-write-to-table.adoc index bdc3f5d..c649baf 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/trbPulsarTsRecordFailsToWriteToTable.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/record-fails-to-write-to-table.adoc @@ -1,5 +1,5 @@ = Record fails to write - +:page-aliases: trbPulsarTsRecordFailsToWriteToTable.adoc :page-tag: pulsar-connector,dev,develop,admin,manage,pulsar Records never appear in the DataStax Enterprise (DSE) database table. The cause can be issues with node availability, transient errors that prevent writes due to consistency level, a schema change where a mapped column was removed from the target table, and so forth. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/trbPulsarTsRecordParsingFails.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/record-parsing-fails.adoc similarity index 96% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/trbPulsarTsRecordParsingFails.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/record-parsing-fails.adoc index c2f5180..742d590 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/trbPulsarTsRecordParsingFails.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/record-parsing-fails.adoc @@ -1,5 +1,5 @@ = Data parsing fails - +:page-aliases: trbPulsarTsRecordParsingFails.adoc :page-tag: pulsar-connector,dev,develop,admin,manage,pulsar When the data from an Apache Pulsar™ field is not compatible with the data type of the target xref:index.adoc#supported-databases[supported database] table column, the data conversion fails and a message is logged in the Pulsar Connect log. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarAuthLdap.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-auth-ldap.adoc similarity index 94% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarAuthLdap.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-auth-ldap.adoc index 33a643a..9fbfb31 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarAuthLdap.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-auth-ldap.adoc @@ -1,5 +1,5 @@ = Internal or LDAP authentication - +:page-aliases: cfgRefPulsarAuthLdap.adoc :page-tag: pulsar-connector,security,secure,pulsar When the cluster has internal or LDAP authentication enabled, configure the user name and password settings for DataStax Apache Pulsar™ Connector. @@ -21,7 +21,7 @@ provider:: Select the type of authentication provider configured for the DataSta - **DSE** - Internal or LDAP authentication. - **GSSAPI** - Supports SASL authentication to DSE clusters using the GSSAPI mechanism \(Kerberos authentication\) + -NOTE: When using GSSAPI, the Pulsar Connect process requires that the Kerberos configuration file `krb5.conf` location is provided in the `java.security.krb5.conf` system property at startup. See xref:secPulsarInternalLdapAuth.adoc[]. +NOTE: When using GSSAPI, the Pulsar Connect process requires that the Kerberos configuration file `krb5.conf` location is provided in the `java.security.krb5.conf` system property at startup. See xref:internal-ldap-auth.adoc[]. + Default: `None` diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarAuth.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-auth.adoc similarity index 80% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarAuth.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-auth.adoc index 97edb9b..cc7387f 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarAuth.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-auth.adoc @@ -1,5 +1,5 @@ = Using the DataStax Apache Pulsar™ Connector with DataStax Enterprise authentication - +:page-aliases: cfgRefPulsarAuth.adoc :page-tag: pulsar-connector,security,secure,pulsar When authentication is enabled, provide credentials for internal authentication or LDAP. @@ -10,4 +10,4 @@ TIP: To verify that authentication is configured, check the following parameters * link:https://docs.datastax.com/en/dse/6.8/dse-admin/datastax_enterprise/config/configCassandra_yaml.html#configCassandra_yaml__authenticator[authenticator] of the `cassandra.yaml` * link:https://docs.datastax.com/en/dse/6.8/dse-admin/datastax_enterprise/config/configDseYaml.html#configDseYaml__authentication_options[authentication_options] of the `dse.yaml` -* xref:cfgRefPulsarAuthLdap.adoc[] When the cluster has internal or LDAP authentication enabled, you should configure the username and password settings for the Pulsar Connector. +* xref:reference-pulsar-auth-ldap.adoc[] When the cluster has internal or LDAP authentication enabled, you should configure the username and password settings for the Pulsar Connector. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarConnector.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-connector.adoc similarity index 97% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarConnector.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-connector.adoc index b292f8d..7942c4f 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarConnector.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-connector.adoc @@ -1,5 +1,5 @@ = DataStax Apache Pulsar™ Connector details - +:page-aliases: cfgRefPulsarConnector.adoc :page-tag: pulsar-connector,dev,develop,pulsar Set the common DataStax Apache Pulsar™ Connector parameters. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarDates.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-dates.adoc similarity index 98% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarDates.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-dates.adoc index 3124aa8..aa4a6e6 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarDates.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-dates.adoc @@ -1,5 +1,5 @@ = Converting date and times for a topic - +:page-aliases: cfgRefPulsarDates.adoc :page-tag: pulsar-connector,dev,develop,pulsar Use the following syntax to configure date and time conversion parameters for each Pulsar topic. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarDseConnection.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-dse-connection.adoc similarity index 97% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarDseConnection.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-dse-connection.adoc index 0dfff8e..840b7a3 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarDseConnection.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-dse-connection.adoc @@ -1,5 +1,5 @@ = DataStax connection - +:page-aliases: cfgRefPulsarDseConnection.adoc :page-tag: pulsar-connector,admin,dev,connect,pulsar Configure the connection settings from Apache Pulsar™ to the cluster. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarDseTable.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-dse-table.adoc similarity index 97% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarDseTable.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-dse-table.adoc index be8208b..05c3838 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarDseTable.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-dse-table.adoc @@ -1,5 +1,5 @@ = Pulsar topic-to-table settings - +:page-aliases: cfgRefPulsarDseTable.adoc :page-tag: pulsar-connector,dev,develop,pulsar Capture Apache Pulsar™ topics in the xref:index.adoc#supported-databases[supported database] by specifying a target keyspace, table, and then map the topic fields to table columns in the name of the parameter. @@ -35,7 +35,7 @@ where: [#mapping] mapping:: Required, field-to-column mapping. -See xref:ROOT:cfgPulsarMapTopicTable.adoc[]. +See xref:ROOT:configure-map-topic-table.adoc[]. [#consistencyLevel] consistencyLevel:: Query consistency level. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarSsl.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-ssl.adoc similarity index 98% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarSsl.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-ssl.adoc index 4624874..9c1623c 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/cfgRefPulsarSsl.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/reference-pulsar-ssl.adoc @@ -1,5 +1,5 @@ = SSL encrypted connection - +:page-aliases: cfgRefPulsarSsl.adoc :page-tag: pulsar-connector,security,secure,pulsar When the cluster has client encryption enabled, configure the SSL keys and certificates for the DataStax Apache Pulsar™ Connector. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarRemoveColumn.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/remove-column.adoc similarity index 90% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarRemoveColumn.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/remove-column.adoc index bab06ee..98f8b6c 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarRemoveColumn.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/remove-column.adoc @@ -1,10 +1,10 @@ = Removing a Table Column from the mapping specification - +:page-aliases: opsPulsarRemoveColumn.adoc :page-tag: pulsar-connector,dev,develop,pulsar Remove a xref:index.adoc#supported-databases[supported database] table column. -. Remove the field to column mapping from the configuration. See xref:cfgPulsarMapTopicTable.adoc[]. +. Remove the field to column mapping from the configuration. See xref:configure-map-topic-table.adoc[]. . On a database node, use the link:https://docs.datastax.com/en/dse/6.8/cql/cql/cql_reference/cql_commands/cqlAlterTable.html[ALTER TABLE] to remove the column from the table. + TIP: You can only remove non-PRIMARY KEY columns. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarRemoveField.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/remove-field.adoc similarity index 86% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarRemoveField.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/remove-field.adoc index cf531b0..1e27934 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarRemoveField.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/remove-field.adoc @@ -1,5 +1,5 @@ = Removing a field map - +:page-aliases: opsPulsarRemoveField.adoc :page-tag: pulsar-connector,dev,develop,pulsar DataStax Apache Pulsar™ Connector processing fails if any mapped Pulsar topic fields are missing from the record. @@ -13,6 +13,6 @@ A NULL is not allowed in a PRIMARY KEY column, therefore, each `PRIMARY KEY` col + NOTE: If the field is a PK column, map a different field to the column. -. Remove the field to column mapping from the configuration. See xref:cfgPulsarMapTopicTable.adoc[]. -. Update the configuration with the new settings. See xref:opsPulsarUpdateConfig.adoc[]. +. Remove the field to column mapping from the configuration. See xref:configure-map-topic-table.adoc[]. +. Update the configuration with the new settings. See xref:update-config.adoc[]. . Start the Pulsar producer that had the schema change. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarRestart.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/restart.adoc similarity index 93% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarRestart.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/restart.adoc index 2012fd3..25314b8 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarRestart.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/restart.adoc @@ -1,5 +1,5 @@ = Restarting the DataStax Apache Pulsar™ Connector - +:page-aliases: opsPulsarRestart.adoc :page-tag: pulsar-connector,admin,manage,pulsar To restart a DataStax Apache Pulsar™ Connector that was previously running, use the `pulsar-admin sinks` command: diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarScaling.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/scaling.adoc similarity index 97% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarScaling.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/scaling.adoc index 2eb121b..523d3ec 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarScaling.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/scaling.adoc @@ -1,5 +1,5 @@ = Scaling the DataStax Apache Pulsar™ Connector - +:page-aliases: opsPulsarScaling.adoc :page-tag: pulsar-connector,admin,manage,pulsar Use the Apache Pulsar™ administration tool to increase or decrease the number of workers to run for a given sink using the parallelism factor. You can specify the parallelism factor during creation of a Pulsar sink, and you can modify it after the fact as well. The default parallelism factor is `1`. diff --git a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarUpdateConfig.adoc b/docs-src/pulsar-sink-core/modules/ROOT/pages/update-config.adoc similarity index 98% rename from docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarUpdateConfig.adoc rename to docs-src/pulsar-sink-core/modules/ROOT/pages/update-config.adoc index 0cff116..bc80b14 100644 --- a/docs-src/pulsar-sink-core/modules/ROOT/pages/opsPulsarUpdateConfig.adoc +++ b/docs-src/pulsar-sink-core/modules/ROOT/pages/update-config.adoc @@ -1,5 +1,5 @@ = Updating the DataStax Apache Pulsar™ Connector configuration - +:page-aliases: opsPulsarUpdateConfig.adoc :page-tag: pulsar-connector,admin,manage,pulsar To update the configuration of an active DataStax Apache Pulsar™ Connector sink, use the Apache Pulsar™ `pulsar-admin sinks update` command: