Skip to content

Commit 0fcb9c0

Browse files
committed
test: improve SST progress test with realistic data volume and timing
- Load 2M documents before JOIN to ensure meaningful SST transfer - Check SST variables immediately after JOIN starts (no delay) - Use specific regex patterns instead of catch-all patterns - Simplify test comments for better readability - Fix manticore-load usage with proper --load parameter
1 parent 587fd47 commit 0fcb9c0

File tree

1 file changed

+22
-31
lines changed

1 file changed

+22
-31
lines changed

test/clt-tests/replication/test-sst-progress.rec

Lines changed: 22 additions & 31 deletions
Original file line numberDiff line numberDiff line change
@@ -1,19 +1,5 @@
11
––– comment –––
2-
Test for SST (State Snapshot Transfer) progress monitoring during table replication
3-
Issue: https://github.com/manticoresoftware/effyis/issues/390
4-
5-
Problem: When adding a large table to cluster via ALTER CLUSTER ADD, there was no way
6-
to track replication progress. Users couldn't determine how much was copied or remaining.
7-
8-
Solution: Added SST progress variables to SHOW STATUS:
9-
- cluster_<name>_sst_total: Overall progress (0-100)
10-
- cluster_<name>_sst_stage: Current stage (e.g., "send files")
11-
- cluster_<name>_sst_stage_total: Progress of current stage (0-100)
12-
- cluster_<name>_sst_tables: Total tables being transferred
13-
- cluster_<name>_sst_table: Current table being transferred (e.g., "3 (products)")
14-
- cluster_<name>_node_state: Node state (donor/joiner/synced)
15-
16-
This test validates that SST progress variables exist in SHOW STATUS output.
2+
Test SST progress monitoring variables during replication (effyis#390)
173
––– input –––
184
apt-get update -y > /dev/null; echo $?
195
––– output –––
@@ -55,9 +41,9 @@ mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_sst_test_status'\G"
5541
Counter: cluster_sst_test_status
5642
Value: primary
5743
––– comment –––
58-
Create simple table and add to cluster
44+
Create table and add to cluster
5945
––– input –––
60-
mysql -h0 -P1306 -e "CREATE TABLE test_table (id bigint, title text)"
46+
mysql -h0 -P1306 -e "CREATE TABLE test_table (id bigint, title text, content text)"
6147
––– output –––
6248
––– input –––
6349
mysql -h0 -P1306 -e "ALTER CLUSTER sst_test ADD test_table"
@@ -84,58 +70,63 @@ if timeout 10 grep -qm1 '\[BUDDY\] started' <(tail -n 1000 -f /var/log/manticore
8470
––– output –––
8571
Buddy started!
8672
––– comment –––
87-
Join node 2 to cluster - this triggers SST
73+
Load 2M documents into the table before replication
8874
––– input –––
89-
mysql -h0 -P2306 -e "JOIN CLUSTER sst_test AT '127.0.0.1:1312'"
75+
manticore-load --host=127.0.0.1 --port=1306 --quiet --threads=4 --total=2000000 --load="INSERT INTO sst_test:test_table (id, title, content) VALUES (<increment>, '<text/10/50>', '<text/20/100>')" --batch-size=5000
9076
––– output –––
77+
#!/.*/!#
9178
––– comment –––
92-
Verify SST progress variables exist on donor node
93-
The main goal is to check that these variables are available, not their specific values
79+
Verify data is loaded
9480
––– input –––
95-
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_sst_test_node_state'\G"
81+
mysql -h0 -P1306 -NB -e "SELECT COUNT(*) FROM sst_test:test_table"
82+
––– output –––
83+
#!/[0-9]+/!#
84+
––– comment –––
85+
Join node 2 - triggers SST with 2M documents
86+
––– input –––
87+
mysql -h0 -P2306 -e "JOIN CLUSTER sst_test AT '127.0.0.1:1312'" > /dev/null 2>&1 & mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_sst_test_node_state'\G"
9688
––– output –––
9789
*************************** 1. row ***************************
9890
Counter: cluster_sst_test_node_state
99-
Value: #!/.+/!#
91+
Value: #!/(donor|joiner|synced)/!#
10092
––– comment –––
10193
Verify all 5 SST progress variables exist
10294
––– input –––
10395
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_sst_test_sst_%'\G" | grep "Counter:" | wc -l
10496
––– output –––
10597
5
10698
––– comment –––
107-
Check that each individual SST variable exists
108-
Values will be "-" if SST completed, or numbers/stage names if still active
99+
Check individual SST variables during transfer
109100
––– input –––
110101
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_sst_test_sst_total'\G"
111102
––– output –––
112103
*************************** 1. row ***************************
113104
Counter: cluster_sst_test_sst_total
114-
Value: #!/.*/!#
105+
Value: #!/([0-9]{1,2}|100|-)/!#
115106
––– input –––
116107
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_sst_test_sst_stage'\G"
117108
––– output –––
118109
*************************** 1. row ***************************
119110
Counter: cluster_sst_test_sst_stage
120-
Value: #!/.*/!#
111+
Value: #!/([a-z ]+|-)/!#
121112
––– input –––
122113
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_sst_test_sst_stage_total'\G"
123114
––– output –––
124115
*************************** 1. row ***************************
125116
Counter: cluster_sst_test_sst_stage_total
126-
Value: #!/.*/!#
117+
Value: #!/([0-9]{1,2}|100|-)/!#
127118
––– input –––
128119
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_sst_test_sst_tables'\G"
129120
––– output –––
130121
*************************** 1. row ***************************
131122
Counter: cluster_sst_test_sst_tables
132-
Value: #!/.*/!#
123+
Value: #!/([0-9]+|-)/!#
133124
––– input –––
134125
mysql -h0 -P1306 -e "SHOW STATUS LIKE 'cluster_sst_test_sst_table'\G"
135126
––– output –––
136127
*************************** 1. row ***************************
137128
Counter: cluster_sst_test_sst_table
138-
Value: #!/.*/!#
129+
Value: #!/([0-9]+ \([a-z_]+\)|-)/!#
139130
––– comment –––
140131
Verify both nodes are synced
141132
––– input –––
@@ -158,7 +149,7 @@ mysql -h0 -P2306 -e "SHOW STATUS LIKE 'cluster_sst_test_indexes'\G"
158149
Counter: cluster_sst_test_indexes
159150
Value: test_table
160151
––– comment –––
161-
Cleanup: Remove watchdog from base config to avoid affecting other tests
152+
Cleanup: Remove watchdog from base config
162153
––– input –––
163154
sed -i '/watchdog = 0/d' test/clt-tests/base/searchd-with-flexible-ports.conf; echo $?
164155
––– output –––

0 commit comments

Comments
 (0)