Skip to content

Commit 74ec533

Browse files
eric-schneidereric-schneider
committed
Move CDC instructions to DB page
1 parent 8e3576c commit 74ec533

File tree

2 files changed

+153
-145
lines changed

2 files changed

+153
-145
lines changed

‎docs-src/astra-cli-core/modules/ROOT/pages/astra-streaming-cli.adoc

Lines changed: 2 additions & 141 deletions
Original file line numberDiff line numberDiff line change
@@ -638,10 +638,7 @@ default(pulsar-aws-useast2.streaming.datastax.com)>
638638

639639
Type `exit` and press kbd:[Return] to exit the Pulsar shell.
640640

641-
[NOTE]
642-
====
643641
The first time you use the `astra streaming pulsar-shell` commands, the {product} downloads and installs the `pulsar-shell` utility to the {product} installation directory (`~/.astra/lunastreaming-shell-*`).
644-
====
645642

646643
=== `streaming pulsar-shell` options
647644

@@ -947,144 +944,8 @@ The directory must already exist before you run the command.
947944
If the directory doesn't exist, the {product} will report the following error: `INVALID_ARGUMENT: Destination folder has not been found`.
948945
====
949946

950-
== Change Data Capture
951-
952-
[IMPORTANT]
953-
====
954-
CDC for {astra_db} incurs billed charges based on your {astra_stream} usage.
955-
For more information, see https://www.datastax.com/pricing/astra-streaming[{astra_stream} pricing] and https://www.datastax.com/products/datastax-astra/cdc-for-astra-db[CDC metering rates].
956-
====
947+
== Change Data Capture (CDC)
957948

958949
CDC for {astra_db} automatically captures changes in real time, de-duplicates the changes, and then streams the clean set of changed data into xref:astra-streaming:ROOT:index.adoc[{astra_stream}] where it can be processed by client applications or sent to downstream systems.
959950

960-
The following procedure provides some examples of how to use the {product} to configure and use CDC for {astra_db}:
961-
962-
. Create an {astra_db} {db-serverless} database:
963-
+
964-
[source,bash]
965-
----
966-
astra db create demo_cdc
967-
----
968-
+
969-
.Result
970-
[%collapsible]
971-
====
972-
[source,console]
973-
----
974-
975-
----
976-
====
977-
978-
. Create a tenant in same region as your database:
979-
+
980-
[source,bash]
981-
----
982-
astra streaming create demo-cdc -c gcp -r useast1
983-
----
984-
+
985-
.Result
986-
[%collapsible]
987-
====
988-
[source,console]
989-
----
990-
https://api.astra.datastax.com/v2/streaming/tenants/demo-cdc
991-
[OK] Tenant 'demo-cdc' has being created.
992-
----
993-
====
994-
995-
. Start the CQL shell in interactive mode:
996-
+
997-
[source,bash]
998-
----
999-
astra db cqlsh demo_cdc -k default_keyspace
1000-
----
1001-
+
1002-
.Result
1003-
[%collapsible]
1004-
====
1005-
[source,console]
1006-
----
1007-
[INFO] Cqlsh is starting, please wait for connection establishment...
1008-
Connected to cndb at 127.0.0.1:9042.
1009-
[cqlsh 6.8.0 | Cassandra 4.0.0.6816 | CQL spec 3.4.5 | Native protocol v4]
1010-
Use HELP for help.
1011-
token@cqlsh:default_keyspace>
1012-
----
1013-
====
1014-
1015-
. Create a table.
1016-
+
1017-
[source,cql]
1018-
----
1019-
CREATE TABLE IF NOT EXISTS demo (foo text PRIMARY KEY, bar text);
1020-
CREATE TABLE IF NOT EXISTS table2 (foo text PRIMARY KEY, bar text);
1021-
INSERT INTO demo(foo,bar) VALUES('1','item1');
1022-
INSERT INTO demo(foo,bar) VALUES('1','item2');
1023-
quit;
1024-
----
1025-
+
1026-
.Result
1027-
[%collapsible]
1028-
====
1029-
[source,console]
1030-
----
1031-
token@cqlsh:default_keyspace> CREATE TABLE IF NOT EXISTS demo (foo text PRIMARY KEY, bar text);
1032-
token@cqlsh:default_keyspace> CREATE TABLE IF NOT EXISTS table2 (foo text PRIMARY KEY, bar text);
1033-
token@cqlsh:default_keyspace> INSERT INTO demo(foo,bar) VALUES('1','item1');
1034-
token@cqlsh:default_keyspace> INSERT INTO demo(foo,bar) VALUES('1','item2');
1035-
token@cqlsh:default_keyspace> quit;
1036-
----
1037-
====
1038-
1039-
. Create CDC.
1040-
+
1041-
[source,bash]
1042-
----
1043-
astra db create-cdc demo_cdc \
1044-
-k demo_cdc \
1045-
--table demo \
1046-
--tenant demo-cdc
1047-
1048-
astra db cdc create demo_cdc --table demo_cdc.test
1049-
----
1050-
+
1051-
.Result
1052-
[%collapsible]
1053-
====
1054-
[source,console]
1055-
----
1056-
1057-
----
1058-
====
1059-
1060-
. List CDC (from Streaming)
1061-
+
1062-
[source,bash]
1063-
----
1064-
astra streaming list-cdc demo-cdc
1065-
----
1066-
+
1067-
.Result
1068-
[%collapsible]
1069-
====
1070-
[source,console]
1071-
----
1072-
1073-
----
1074-
====
1075-
1076-
. List CDC (from DB)
1077-
+
1078-
[source,bash]
1079-
----
1080-
astra db list-cdc demo_cdc
1081-
----
1082-
+
1083-
.Result
1084-
[%collapsible]
1085-
====
1086-
[source,console]
1087-
----
1088-
1089-
----
1090-
====
951+
For more information about using the {product} to configure CDC, see xref:ROOT:managing.adoc#cdc[Configure Change Data Capture].

‎docs-src/astra-cli-core/modules/ROOT/pages/managing.adoc

Lines changed: 151 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1607,6 +1607,7 @@ include::ROOT:partial$regions-change-admonition.adoc[]
16071607

16081608
// TODO: Add examples for `-cf`/`--config-file`, `-conf/`--config`, `--env`, `--token`, and `-v`/`--verbose` options.
16091609

1610+
[#cqlsh]
16101611
== Use CQL shell (cqlsh)
16111612

16121613
The CQL shell (`cqlsh`) is an interface that you can use to issue xref:astra@cql:ROOT:index.adoc[{cass-short} Query Language (CQL)] commands to your {astra_db} database.
@@ -1641,12 +1642,12 @@ Use HELP for help.
16411642

16421643
Type `exit` and press kbd:[Return] to exit the CQL shell.
16431644

1644-
[NOTE]
1645-
======
16461645
The first time you use `astra db cqlsh`, the {product} downloads and installs the `cqlsh` utility to the {product} installation directory (`~/.astra/cqlsh-astra`).
16471646
The {product} also downloads the {scb-long} for each database you connect to and stores the {scb} zip files in the `~/.astra/scb` directory.
16481647

1649-
If you encounter a `ModuleNotFoundError` error, check that you have a supported version of Python available in your terminal:
1648+
[NOTE]
1649+
======
1650+
If the {product} reports `ModuleNotFoundError`, check that you have a supported version of Python available in your terminal:
16501651
16511652
[source,bash]
16521653
----
@@ -2933,4 +2934,150 @@ SYNOPSIS
29332934
29342935
Where * indicates the default command(s)
29352936
See 'astra help role <command>' for more information on a specific command.
2936-
----
2937+
----
2938+
2939+
[#cdc]
2940+
== Configure Change Data Capture (CDC)
2941+
2942+
[IMPORTANT]
2943+
====
2944+
CDC for {astra_db} incurs billed charges based on your {astra_stream} usage.
2945+
For more information, see https://www.datastax.com/pricing/astra-streaming[{astra_stream} pricing] and https://www.datastax.com/products/datastax-astra/cdc-for-astra-db[CDC metering rates].
2946+
====
2947+
2948+
CDC for {astra_db} automatically captures changes in real time, de-duplicates the changes, and then streams the clean set of changed data into xref:astra-streaming:ROOT:index.adoc[{astra_stream}] where it can be processed by client applications or sent to downstream systems.
2949+
2950+
The following procedure describes how to use the {product} to enable CDC on a {db-serverless} database:
2951+
2952+
. Create an {astra_db} {db-serverless} database:
2953+
+
2954+
[source,bash]
2955+
----
2956+
astra db create cdc_demo_db -c aws -r us-east-1 -k cdc_demo_keyspace
2957+
----
2958+
+
2959+
.Result
2960+
[%collapsible]
2961+
====
2962+
[source,console]
2963+
----
2964+
REGION OK
2965+
[INFO] Database 'cdc_demo_db' does not exist. Creating database 'cdc_demo_db' with keyspace 'cdc_demo_keyspace'
2966+
get CLoud provider
2967+
[INFO] Database 'cdc_demo_db' and keyspace 'cdc_demo_keyspace' are being created.
2968+
[INFO] Database 'cdc_demo_db' has status 'PENDING' waiting to be 'ACTIVE' ...
2969+
[INFO] Database 'cdc_demo_db' has status 'ACTIVE' (took 112117 millis)
2970+
[OK] Database 'cdc_demo_db' is ready.
2971+
----
2972+
====
2973+
2974+
. Create an {astra_stream} tenant in the same region as your database:
2975+
+
2976+
[source,bash]
2977+
----
2978+
astra streaming create cdc-demo-tenant -c aws -r useast1
2979+
----
2980+
+
2981+
.Result
2982+
[%collapsible]
2983+
====
2984+
[source,console]
2985+
----
2986+
https://api.astra.datastax.com/v2/streaming/tenants/cdc-demo-tenant
2987+
[OK] Tenant 'cdc-demo-tenant' has being created.
2988+
----
2989+
====
2990+
2991+
. Use the xref:ROOT:managing.adoc#cqlsh[CQL shell] to create a table with a primary key column in your database.
2992+
+
2993+
[source,bash]
2994+
----
2995+
astra db cqlsh cdc_demo_db -e "CREATE TABLE IF NOT EXISTS cdc_demo_keyspace.cdc_demo_table (key text PRIMARY KEY, c1 text);"
2996+
----
2997+
+
2998+
.Result
2999+
[%collapsible]
3000+
====
3001+
[source,console]
3002+
----
3003+
[INFO] Cqlsh is starting, please wait for connection establishment...
3004+
----
3005+
====
3006+
3007+
. Confirm table creation:
3008+
+
3009+
[source,bash]
3010+
----
3011+
astra db cqlsh cdc_demo_db -e "SELECT * FROM cdc_demo_keyspace.cdc_demo_table"
3012+
----
3013+
+
3014+
.Result
3015+
[%collapsible]
3016+
====
3017+
[source,console]
3018+
----
3019+
[INFO] Cqlsh is starting, please wait for connection establishment...
3020+
3021+
key | c1
3022+
-----+----
3023+
3024+
(0 rows)
3025+
----
3026+
====
3027+
3028+
. Create a CDC connector.
3029+
+
3030+
[source,bash]
3031+
----
3032+
astra db create-cdc cdc_demo_db -k cdc_demo_keyspace --table cdc_demo_table --tenant cdc-demo-tenant
3033+
----
3034+
+
3035+
.Result
3036+
[%collapsible]
3037+
====
3038+
[source,console]
3039+
----
3040+
3041+
----
3042+
====
3043+
// TODO: The command is not worrking as expected. Must investigate and figure out why it is reporting the following error: [ERROR] INVALID_ARGUMENT: Error Code=422(422) Invalid information provided to create DB: 422 Unprocessable Entity: databaseId, keyspace, tableName, and orgId are mandatory fields
3044+
3045+
. Confirm CDC details for the database and tenant.
3046+
+
3047+
[source,bash]
3048+
----
3049+
astra db list-cdc cdc_demo_db
3050+
----
3051+
+
3052+
.Result
3053+
[%collapsible]
3054+
====
3055+
[source,console]
3056+
----
3057+
+-----------------------+-------------------+----------------+----------------+--------------------+----------------+----------------+
3058+
| id | keyspace | table | tenant | cluster | namespace | Status |
3059+
+-----------------------+-------------------+----------------+----------------+--------------------+----------------+----------------+
3060+
| 57a3024f-cdcdemotable | cdc_demo_keyspace | cdc_demo_table | cdc-demo-tenant| pulsar-aws-useast1 | astracdc | Running |
3061+
+-----------------------+-------------------+----------------+----------------+--------------------+----------------+----------------+
3062+
----
3063+
====
3064+
+
3065+
[source,bash]
3066+
----
3067+
astra streaming list-cdc cdc-demo-tenant
3068+
----
3069+
+
3070+
.Result
3071+
[%collapsible]
3072+
====
3073+
[source,console]
3074+
----
3075+
+--------------------+----------------+----------------+-------------------+----------------+----------------+
3076+
| cluster | namespace | database | keyspace | table | status |
3077+
+--------------------+----------------+----------------+-------------------+----------------+----------------+
3078+
| pulsar-aws-useast1 | astracdc | cdc_demo_db | cdc_demo_keyspace | cdc_demo_table | running |
3079+
+--------------------+----------------+----------------+-------------------+----------------+----------------+
3080+
----
3081+
====
3082+
3083+
After you enable CDC on your {db-serverless} database, you can xref:astra-db-serverless:databases:change-data-capture.adoc#connect-a-sink[connect a sink].

0 commit comments

Comments
 (0)