Skip to content

Commit 262a4a1

Browse files
committed
tests: fix test_storage_s3_queue/test_0.py::test_streaming_to_many_views flakiness
The problem is that streaming is done in background and it can be started when there are less MVs created yet, but in the mean time the test will create missing MVs and create files in S3. Here is an example of server logs for such failure [1]: 2025.04.17 16:08:20.523134 [ 493 ] {} <Debug> StorageS3Queue (default.streaming_to_many_views_ordered): Started streaming to 9 attached views 2025.04.17 16:08:20.523665 [ 13 ] {4e2228b2-48d9-42c8-bafb-9965d9c74625} <Debug> executeQuery: (from 172.16.1.1:58662) (query 2, line 5) CREATE MATERIALIZED VIEW streaming_to_many_views_ordered_10_mv TO streaming_to_many_views_ordered_10_dst AS SELECT *, _path FROM streaming_to_many_views_ordered; (stage: Complete) ... 2025.04.17 16:08:20.572373 [ 744 ] {} <Test> StorageObjectStorageQueue(/clickhouse/test_streaming_to_many_views_ordered_ddxwkh): Path: streaming_to_many_views_ordered_data/a_0.csv, node_name: 5331496251994711888, max_loading_retries: 2, processed_path: /clickhouse/test_streaming_to_many_views_ordered_ddxwkh/buckets/0/processed, processing_path: /clickhouse/test_streaming_to_many_views_ordered_ddxwkh/processing/5331496251994711888, failed_path: /clickhouse/test_streaming_to_many_views_ordered_ddxwkh/failed/5331496251994711888 [1]: https://s3.amazonaws.com/clickhouse-test-reports/json.html?REF=master&sha=79015e3ac01920269c1789b176fcc739ae9bbaca&name_0=MasterCI&name_1=Integration%20tests%20%28tsan%2C%201%2F6%29 Sadly, but there is no way to sync this things without extra sleep.
1 parent 1291c77 commit 262a4a1

File tree

1 file changed

+3
-0
lines changed
  • tests/integration/test_storage_s3_queue

1 file changed

+3
-0
lines changed

tests/integration/test_storage_s3_queue/test_0.py

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -441,6 +441,9 @@ def test_streaming_to_many_views(started_cluster, mode):
441441
expect_rows_num = [0]
442442
files = []
443443

444+
# ensure that streaming from S3 will be started only once all MVs has been created
445+
time.sleep(5)
446+
444447
def generate_files(files_num=20, row_num=100, file_prefix = "a"):
445448
files.extend(
446449
[

0 commit comments

Comments
 (0)