Skip to content

Commit faf3b2b

Browse files
eric-schneidereric-schneider
committed
More CDC cleanup
1 parent 74ec533 commit faf3b2b

File tree

2 files changed

+316
-10
lines changed

2 files changed

+316
-10
lines changed

‎docs-src/astra-cli-core/modules/ROOT/pages/astra-streaming-cli.adoc

Lines changed: 64 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -949,3 +949,67 @@ If the directory doesn't exist, the {product} will report the following error: `
949949
CDC for {astra_db} automatically captures changes in real time, de-duplicates the changes, and then streams the clean set of changed data into xref:astra-streaming:ROOT:index.adoc[{astra_stream}] where it can be processed by client applications or sent to downstream systems.
950950

951951
For more information about using the {product} to configure CDC, see xref:ROOT:managing.adoc#cdc[Configure Change Data Capture].
952+
953+
Use the `streaming list-cdc` command to list the CDC connections associated with a tenant:
954+
955+
[source,bash,subs="+quotes"]
956+
----
957+
astra streaming list-cdc **TENANT_NAME**
958+
----
959+
960+
.Result
961+
[%collapsible]
962+
====
963+
[source,console]
964+
----
965+
+--------------------+----------------+----------------+-------------------+----------------+----------------+
966+
| cluster | namespace | database | keyspace | table | status |
967+
+--------------------+----------------+----------------+-------------------+----------------+----------------+
968+
| pulsar-aws-useast1 | astracdc | cdc_demo_db | cdc_demo_keyspace | cdc_demo_table | running |
969+
+--------------------+----------------+----------------+-------------------+----------------+----------------+
970+
----
971+
====
972+
973+
=== `streaming list-cdc` options
974+
975+
[source,console]
976+
----
977+
NAME
978+
astra streaming list-cdc - List CDC available on this tenant
979+
980+
SYNOPSIS
981+
astra streaming list-cdc [ {-cf | --config-file} <CONFIG_FILE> ]
982+
[ {-conf | --config} <CONFIG_SECTION> ] [ --env <Environment> ]
983+
[ --no-color ] [ {-o | --output} <FORMAT> ]
984+
[ --token <AUTH_TOKEN> ] [ {-v | --verbose} ] [--] <TENANT>
985+
986+
OPTIONS
987+
-cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
988+
Configuration file (default = ~/.astrarc)
989+
990+
-conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
991+
Section in configuration file (default = ~/.astrarc)
992+
993+
--env <Environment>
994+
Astra Environment to use
995+
996+
--no-color
997+
Remove all colors in output
998+
999+
-o <FORMAT>, --output <FORMAT>
1000+
Output format, valid values are: human,json,csv
1001+
1002+
--token <AUTH_TOKEN>
1003+
Key to use authenticate each call.
1004+
1005+
-v, --verbose
1006+
Verbose mode with log in console
1007+
1008+
--
1009+
This option can be used to separate command-line options from the
1010+
list of arguments (useful when arguments might be mistaken for
1011+
command-line options)
1012+
1013+
<TENANT>
1014+
Tenant identifier
1015+
----

‎docs-src/astra-cli-core/modules/ROOT/pages/managing.adoc

Lines changed: 252 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -2255,7 +2255,7 @@ OPTIONS
22552255

22562256
=== End-to-end example
22572257

2258-
The following procedure provides an end-to-end example of how to use the {product}'s built-in DSBulk support to load data into an {astra_db} database, get information about the data, and unload the data into CSV files:
2258+
The following end-to-end example shows how to use the {product}'s built-in DSBulk support to load data into an {astra_db} database, get information about the data, and unload the data into CSV files:
22592259

22602260
. (Optional) This procedure assumes that you have a database and keyspace both named `demo`:
22612261
+
@@ -2937,7 +2937,7 @@ SYNOPSIS
29372937
----
29382938

29392939
[#cdc]
2940-
== Configure Change Data Capture (CDC)
2940+
== Change Data Capture (CDC)
29412941

29422942
[IMPORTANT]
29432943
====
@@ -2947,13 +2947,243 @@ For more information, see https://www.datastax.com/pricing/astra-streaming[{astr
29472947

29482948
CDC for {astra_db} automatically captures changes in real time, de-duplicates the changes, and then streams the clean set of changed data into xref:astra-streaming:ROOT:index.adoc[{astra_stream}] where it can be processed by client applications or sent to downstream systems.
29492949

2950-
The following procedure describes how to use the {product} to enable CDC on a {db-serverless} database:
2950+
Use the `astra db create-cdc` command to create a CDC connection for a table in a {db-serverless} database:
2951+
2952+
[source,bash,subs="+quotes"]
2953+
----
2954+
astra db create-cdc **DATABASE_NAME** \
2955+
-k **KEYSPACE_NAME** \
2956+
--table **TABLE_NAME** \
2957+
--tenant **TENANT_NAME**
2958+
----
2959+
2960+
////
2961+
// TODO: The command is not working as expected. Must investigate and figure out why it is reporting the following error: [ERROR] INVALID_ARGUMENT: Error Code=422(422) Invalid information provided to create DB: 422 Unprocessable Entity: databaseId, keyspace, tableName, and orgId are mandatory fields
2962+
.Result
2963+
[%collapsible]
2964+
====
2965+
[source,console]
2966+
----
2967+
2968+
----
2969+
====
2970+
////
2971+
2972+
Use the `astra db list-cdc` command to list the CDC connections for a specifc database:
2973+
2974+
[source,bash,subs="+quotes"]
2975+
----
2976+
astra db list-cdc **DATABASE_NAME**
2977+
----
2978+
2979+
.Result
2980+
[%collapsible]
2981+
====
2982+
[source,console]
2983+
----
2984+
+-----------------------+-------------------+----------------+----------------+--------------------+----------------+----------------+
2985+
| id | keyspace | table | tenant | cluster | namespace | Status |
2986+
+-----------------------+-------------------+----------------+----------------+--------------------+----------------+----------------+
2987+
| 57a3024f-cdcdemotable | cdc_demo_keyspace | cdc_demo_table | cdc-demo-tenant| pulsar-aws-useast1 | astracdc | Running |
2988+
+-----------------------+-------------------+----------------+----------------+--------------------+----------------+----------------+
2989+
----
2990+
====
2991+
2992+
Use the `astra db delete-cdc` command to delete a CDC connection for a specific table:
2993+
2994+
[source,bash,subs="+quotes"]
2995+
----
2996+
astra db delete-cdc **DATABASE_NAME** \
2997+
-k **KEYSPACE_NAME** \
2998+
--table **TABLE_NAME** \
2999+
--tenant **TENANT_NAME**
3000+
----
3001+
3002+
////
3003+
// TODO: The command is not working as expected. Must investigate and figure out why it is reporting the following error: [ERROR] INVALID_ARGUMENT: Error Code=422(422) Invalid information provided to create DB: 422 Unprocessable Entity: databaseId, keyspace, tableName, and orgId are mandatory fields
3004+
.Result
3005+
[%collapsible]
3006+
====
3007+
[source,console]
3008+
----
3009+
3010+
----
3011+
====
3012+
////
3013+
3014+
=== `db create-cdc` options
3015+
3016+
[source,console]
3017+
----
3018+
NAME
3019+
astra db create-cdc - Create a CDC to pulsar
3020+
3021+
SYNOPSIS
3022+
astra db create-cdc [ {-cf | --config-file} <CONFIG_FILE> ]
3023+
[ {-conf | --config} <CONFIG_SECTION> ] [ --env <Environment> ]
3024+
[ {-k | --keyspace} <KEYSPACE> ] [ --no-color ]
3025+
[ {-o | --output} <FORMAT> ]
3026+
[ -p, --partition <PARTITION TOKEN> ] [ --table <TABLE> ]
3027+
[ --tenant <TENANT> ] [ --token <AUTH_TOKEN> ]
3028+
[ {-v | --verbose} ] [--] <DB>
3029+
3030+
OPTIONS
3031+
-cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
3032+
Configuration file (default = ~/.astrarc)
3033+
3034+
-conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
3035+
Section in configuration file (default = ~/.astrarc)
3036+
3037+
--env <Environment>
3038+
Astra Environment to use
3039+
3040+
-k <KEYSPACE>, --keyspace <KEYSPACE>
3041+
Keyspace name
3042+
3043+
--no-color
3044+
Remove all colors in output
3045+
3046+
-o <FORMAT>, --output <FORMAT>
3047+
Output format, valid values are: human,json,csv
3048+
3049+
-p, --partition <PARTITION TOKEN>
3050+
Paritions in the topics
3051+
3052+
--table <TABLE>
3053+
Table name
3054+
3055+
--tenant <TENANT>
3056+
Tenant name
3057+
3058+
--token <AUTH_TOKEN>
3059+
Key to use authenticate each call.
3060+
3061+
-v, --verbose
3062+
Verbose mode with log in console
3063+
3064+
--
3065+
This option can be used to separate command-line options from the
3066+
list of arguments (useful when arguments might be mistaken for
3067+
command-line options)
3068+
3069+
<DB>
3070+
Database name (if unique) or Database identifier
3071+
----
3072+
3073+
=== `db list-cdc` options
3074+
3075+
[source,console]
3076+
----
3077+
NAME
3078+
astra db list-cdc - List CDC available on this databse
3079+
3080+
SYNOPSIS
3081+
astra db list-cdc [ {-cf | --config-file} <CONFIG_FILE> ]
3082+
[ {-conf | --config} <CONFIG_SECTION> ] [ --env <Environment> ]
3083+
[ --no-color ] [ {-o | --output} <FORMAT> ]
3084+
[ --token <AUTH_TOKEN> ] [ {-v | --verbose} ] [--] <DB>
3085+
3086+
OPTIONS
3087+
-cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
3088+
Configuration file (default = ~/.astrarc)
3089+
3090+
-conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
3091+
Section in configuration file (default = ~/.astrarc)
3092+
3093+
--env <Environment>
3094+
Astra Environment to use
3095+
3096+
--no-color
3097+
Remove all colors in output
3098+
3099+
-o <FORMAT>, --output <FORMAT>
3100+
Output format, valid values are: human,json,csv
3101+
3102+
--token <AUTH_TOKEN>
3103+
Key to use authenticate each call.
3104+
3105+
-v, --verbose
3106+
Verbose mode with log in console
3107+
3108+
--
3109+
This option can be used to separate command-line options from the
3110+
list of arguments (useful when arguments might be mistaken for
3111+
command-line options)
3112+
3113+
<DB>
3114+
Database name (if unique) or Database identifier
3115+
----
3116+
3117+
=== `db delete-cdc` options
3118+
3119+
[source,console]
3120+
----
3121+
NAME
3122+
astra db delete-cdc - Delete a CDC connection
3123+
3124+
SYNOPSIS
3125+
astra db delete-cdc [ {-cf | --config-file} <CONFIG_FILE> ]
3126+
[ {-conf | --config} <CONFIG_SECTION> ] [ --env <Environment> ]
3127+
[ -id <CDC_ID> ] [ {-k | --keyspace} <KEYSPACE> ]
3128+
[ --no-color ] [ {-o | --output} <FORMAT> ] [ --table <TABLE> ]
3129+
[ --tenant <TENANT> ] [ --token <AUTH_TOKEN> ]
3130+
[ {-v | --verbose} ] [--] <DB>
3131+
3132+
OPTIONS
3133+
-cf <CONFIG_FILE>, --config-file <CONFIG_FILE>
3134+
Configuration file (default = ~/.astrarc)
3135+
3136+
-conf <CONFIG_SECTION>, --config <CONFIG_SECTION>
3137+
Section in configuration file (default = ~/.astrarc)
3138+
3139+
--env <Environment>
3140+
Astra Environment to use
3141+
3142+
-id <CDC_ID>
3143+
Identifier of the cdc when providing
3144+
3145+
-k <KEYSPACE>, --keyspace <KEYSPACE>
3146+
Keyspace name
3147+
3148+
--no-color
3149+
Remove all colors in output
3150+
3151+
-o <FORMAT>, --output <FORMAT>
3152+
Output format, valid values are: human,json,csv
3153+
3154+
--table <TABLE>
3155+
Table name
3156+
3157+
--tenant <TENANT>
3158+
Tenant name
3159+
3160+
--token <AUTH_TOKEN>
3161+
Key to use authenticate each call.
3162+
3163+
-v, --verbose
3164+
Verbose mode with log in console
3165+
3166+
--
3167+
This option can be used to separate command-line options from the
3168+
list of arguments (useful when arguments might be mistaken for
3169+
command-line options)
3170+
3171+
<DB>
3172+
Database name (if unique) or Database identifier
3173+
----
3174+
3175+
=== End-to-end example
3176+
3177+
The following end-to-end example shows how to use the {product} to create a CDC connection for a table in a {db-serverless} database:
29513178

29523179
. Create an {astra_db} {db-serverless} database:
29533180
+
29543181
[source,bash]
29553182
----
2956-
astra db create cdc_demo_db -c aws -r us-east-1 -k cdc_demo_keyspace
3183+
astra db create cdc_demo_db \
3184+
-c aws \
3185+
-r us-east-1 \
3186+
-k cdc_demo_keyspace
29573187
----
29583188
+
29593189
.Result
@@ -2975,7 +3205,9 @@ get CLoud provider
29753205
+
29763206
[source,bash]
29773207
----
2978-
astra streaming create cdc-demo-tenant -c aws -r useast1
3208+
astra streaming create cdc-demo-tenant \
3209+
-c aws \
3210+
-r useast1
29793211
----
29803212
+
29813213
.Result
@@ -2992,7 +3224,11 @@ https://api.astra.datastax.com/v2/streaming/tenants/cdc-demo-tenant
29923224
+
29933225
[source,bash]
29943226
----
2995-
astra db cqlsh cdc_demo_db -e "CREATE TABLE IF NOT EXISTS cdc_demo_keyspace.cdc_demo_table (key text PRIMARY KEY, c1 text);"
3227+
astra db cqlsh cdc_demo_db -e \
3228+
"CREATE TABLE IF NOT EXISTS cdc_demo_keyspace.cdc_demo_table ( \
3229+
key text PRIMARY KEY, \
3230+
c1 text \
3231+
);"
29963232
----
29973233
+
29983234
.Result
@@ -3008,7 +3244,8 @@ astra db cqlsh cdc_demo_db -e "CREATE TABLE IF NOT EXISTS cdc_demo_keyspace.cdc_
30083244
+
30093245
[source,bash]
30103246
----
3011-
astra db cqlsh cdc_demo_db -e "SELECT * FROM cdc_demo_keyspace.cdc_demo_table"
3247+
astra db cqlsh cdc_demo_db -e \
3248+
"SELECT * FROM cdc_demo_keyspace.cdc_demo_table"
30123249
----
30133250
+
30143251
.Result
@@ -3025,13 +3262,18 @@ astra db cqlsh cdc_demo_db -e "SELECT * FROM cdc_demo_keyspace.cdc_demo_table"
30253262
----
30263263
====
30273264

3028-
. Create a CDC connector.
3265+
. Create a CDC connection.
30293266
+
30303267
[source,bash]
30313268
----
3032-
astra db create-cdc cdc_demo_db -k cdc_demo_keyspace --table cdc_demo_table --tenant cdc-demo-tenant
3269+
astra db create-cdc cdc_demo_db \
3270+
-k cdc_demo_keyspace \
3271+
--table cdc_demo_table \
3272+
--tenant cdc-demo-tenant
30333273
----
30343274
+
3275+
////
3276+
// TODO: The command is not working as expected. Must investigate and figure out why it is reporting the following error: [ERROR] INVALID_ARGUMENT: Error Code=422(422) Invalid information provided to create DB: 422 Unprocessable Entity: databaseId, keyspace, tableName, and orgId are mandatory fields
30353277
.Result
30363278
[%collapsible]
30373279
====
@@ -3040,7 +3282,7 @@ astra db create-cdc cdc_demo_db -k cdc_demo_keyspace --table cdc_demo_table --te
30403282
30413283
----
30423284
====
3043-
// TODO: The command is not worrking as expected. Must investigate and figure out why it is reporting the following error: [ERROR] INVALID_ARGUMENT: Error Code=422(422) Invalid information provided to create DB: 422 Unprocessable Entity: databaseId, keyspace, tableName, and orgId are mandatory fields
3285+
////
30443286

30453287
. Confirm CDC details for the database and tenant.
30463288
+

0 commit comments

Comments
 (0)