Skip to content

Commit a1389f4

Browse files
authored
Merge pull request #86 from ClickHouse/downloader
Minor update of download script
2 parents 863bf47 + 2d5b57f commit a1389f4

File tree

1 file changed

+4
-4
lines changed

1 file changed

+4
-4
lines changed

download_data.sh

Lines changed: 4 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -10,18 +10,18 @@ read -p "Enter the number corresponding to your choice: " choice
1010
case $choice in
1111
2)
1212
# Download 10m dataset: files 0001 to 0010
13-
wget --timestamping --directory-prefix ~/data/bluesky --input-file <(seq --format "https://clickhouse-public-datasets.s3.amazonaws.com/bluesky/file_%04g.json.gz" 1 10)
13+
wget --continue --timestamping --progress=dot:giga --directory-prefix ~/data/bluesky --input-file <(seq --format "https://clickhouse-public-datasets.s3.amazonaws.com/bluesky/file_%04g.json.gz" 1 10)
1414
;;
1515
3)
1616
# Download 100m dataset: files 0001 to 0100
17-
wget --timestamping --directory-prefix ~/data/bluesky --input-file <(seq --format "https://clickhouse-public-datasets.s3.amazonaws.com/bluesky/file_%04g.json.gz" 1 100)
17+
wget --continue --timestamping --progress=dot:giga --directory-prefix ~/data/bluesky --input-file <(seq --format "https://clickhouse-public-datasets.s3.amazonaws.com/bluesky/file_%04g.json.gz" 1 100)
1818
;;
1919
4)
2020
# Download 1000m dataset: files 0001 to 1000
21-
wget --timestamping --directory-prefix ~/data/bluesky --input-file <(seq --format "https://clickhouse-public-datasets.s3.amazonaws.com/bluesky/file_%04g.json.gz" 1 1000)
21+
wget --continue --timestamping --progress=dot:giga --directory-prefix ~/data/bluesky --input-file <(seq --format "https://clickhouse-public-datasets.s3.amazonaws.com/bluesky/file_%04g.json.gz" 1 1000)
2222
;;
2323
*)
2424
# Download 1m dataset: single file
25-
wget --timestamping --directory-prefix ~/data/bluesky "https://clickhouse-public-datasets.s3.amazonaws.com/bluesky/file_0001.json.gz"
25+
wget --continue --timestamping --progress=dot:giga --directory-prefix ~/data/bluesky "https://clickhouse-public-datasets.s3.amazonaws.com/bluesky/file_0001.json.gz"
2626
;;
2727
esac

0 commit comments

Comments
 (0)