Skip to content

chore(⏲): Use fast download mirrors #1061

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 6 commits into from
Apr 24, 2025
Merged

chore(⏲): Use fast download mirrors #1061

Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
6 commits
Select commit Hold shift + click to select a range
552cb1d
chore: Use fast download mirrors
NickLarsenNZ Apr 23, 2025
fdebec5
chore: Show download progress bar
NickLarsenNZ Apr 23, 2025
05e9f08
chore: Improve gpg key warning message
NickLarsenNZ Apr 23, 2025
3391d6d
chore: Show upload progress bar
NickLarsenNZ Apr 23, 2025
baa7f56
Apply suggestions from code review
NickLarsenNZ Apr 24, 2025
87f29f6
chore: Make output more consistent
NickLarsenNZ Apr 24, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .scripts/upload_new_async-profiler_version.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -43,10 +43,10 @@ for arch in "${ARCHITECTURES[@]}"; do
file=async-profiler-$VERSION-linux-$arch.tar.gz

echo "Downloading $file from github.com"
curl --fail -LOs "https://github.com/async-profiler/async-profiler/releases/download/v$VERSION/$file"
curl --fail -LO --progress-bar "https://github.com/async-profiler/async-profiler/releases/download/v$VERSION/$file"

echo "Uploading $file to Nexus"
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" \
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" \
--upload-file "$file" \
'https://repo.stackable.tech/repository/packages/async-profiler/'
done
Expand Down
4 changes: 2 additions & 2 deletions .scripts/upload_new_inotify-tools_version.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -49,10 +49,10 @@ for arch in "${ARCHITECTURES[@]}"; do
file=inotify-tools-$VERSION.$arch.rpm

echo "Downloading $file from dl.fedoraproject.org"
curl --fail -LOs "https://dl.fedoraproject.org/pub/epel/$EPEL_VERSION/Everything/$arch/Packages/i/$file"
curl --fail -LO --progress-bar "https://dl.fedoraproject.org/pub/epel/$EPEL_VERSION/Everything/$arch/Packages/i/$file"

echo "Uploading $file to Nexus"
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" \
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" \
--upload-file "$file" \
'https://repo.stackable.tech/repository/packages/inotify-tools/'
done
Expand Down
8 changes: 4 additions & 4 deletions .scripts/upload_new_jmx_exporter_version.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -35,15 +35,15 @@ JAR_FILE="jmx_prometheus_javaagent-$VERSION.jar"
SUM_FILE="$JAR_FILE.sha256"

echo "Downloading JMX Exporter"
curl --fail -LOs "https://github.com/prometheus/jmx_exporter/releases/download/$VERSION/$JAR_FILE"
curl --fail -LOs "https://github.com/prometheus/jmx_exporter/releases/download/$VERSION/$SUM_FILE"
curl --fail -LO --progress-bar "https://github.com/prometheus/jmx_exporter/releases/download/$VERSION/$JAR_FILE"
curl --fail -LO --progress-bar "https://github.com/prometheus/jmx_exporter/releases/download/$VERSION/$SUM_FILE"

# Check that sha256 sum matches before uploading
sha256sum --check --status "$SUM_FILE" && echo "SHA256 Sum matches"

echo "Uploading to Nexus"
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$JAR_FILE" 'https://repo.stackable.tech/repository/packages/jmx-exporter/'
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$SUM_FILE" 'https://repo.stackable.tech/repository/packages/jmx-exporter/'
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$JAR_FILE" 'https://repo.stackable.tech/repository/packages/jmx-exporter/'
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$SUM_FILE" 'https://repo.stackable.tech/repository/packages/jmx-exporter/'

echo "Successfully uploaded new version of the JMX Exporter ($VERSION) Jar to Nexus"
echo "https://repo.stackable.tech/service/rest/repository/browse/packages/jmx-exporter/"
Expand Down
2 changes: 1 addition & 1 deletion .scripts/upload_new_kcat_version.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ echo "Downloading kcat"
curl --fail -Ls -o "kcat-$VERSION.tar.gz" "https://github.com/edenhill/kcat/archive/refs/tags/$VERSION.tar.gz"

echo "Uploading to Nexus"
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "kcat-$VERSION.tar.gz" 'https://repo.stackable.tech/repository/packages/kcat/'
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "kcat-$VERSION.tar.gz" 'https://repo.stackable.tech/repository/packages/kcat/'


echo "Successfully uploaded new version of kcat ($VERSION) to Nexus"
Expand Down
2 changes: 1 addition & 1 deletion .scripts/upload_new_protoc_version.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ for arch in "${ARCHITECTURES[@]}"; do
FILE_NAME=$(basename "$DOWNLOAD_URL")

echo "Uploading protoc to Nexus"
if ! curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$FILE_NAME" 'https://repo.stackable.tech/repository/packages/protoc/'; then
if ! curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$FILE_NAME" 'https://repo.stackable.tech/repository/packages/protoc/'; then
echo "Failed to upload protoc to Nexus"
exit 1
fi
Expand Down
2 changes: 1 addition & 1 deletion .scripts/upload_new_tini_version.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ for arch in "${ARCHITECTURES[@]}"; do
curl --fail -Ls -o "tini-$VERSION-$arch" "https://github.com/krallin/tini/releases/download/v$VERSION/tini-$arch"

echo "Uploading to Nexus"
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "tini-$VERSION-$arch" 'https://repo.stackable.tech/repository/packages/tini/'
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "tini-$VERSION-$arch" 'https://repo.stackable.tech/repository/packages/tini/'
done

echo "Successfully uploaded new version of TINI ($VERSION) to Nexus"
Expand Down
23 changes: 14 additions & 9 deletions druid/upload_new_druid_version.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ set -euo pipefail
VERSION=${1:?"Missing version number argument (arg 1)"}
NEXUS_USER=${2:?"Missing Nexus username argument (arg 2)"}

# We prefer fast downloads...
BASE_DOWNLOAD_URL="https://dlcdn.apache.org/druid"
# However, if the version is not available, use the slow archive instead:
# BASE_DOWNLOAD_URL="https://archive.apache.org/dist/druid"

read -r -s -p "Nexus Password: " NEXUS_PASSWORD
echo ""

Expand Down Expand Up @@ -33,10 +38,10 @@ cd "$WORK_DIR" || exit

src_file="apache-druid-${VERSION}-src.tar.gz"

echo "Downloading Druid (this can take a while, it is intentionally downloading from a slow mirror that contains all old versions)"
curl --fail -LOs "https://archive.apache.org/dist/druid/${VERSION}/${src_file}"
curl --fail -LOs "https://archive.apache.org/dist/druid/${VERSION}/${src_file}.asc"
curl --fail -LOs "https://archive.apache.org/dist/druid/${VERSION}/${src_file}.sha512"
echo "Downloading Druid (if this fails, try switching the BASE_DOWNLOAD_URL to the archive)"
curl --fail -LO --progress-bar "${BASE_DOWNLOAD_URL}/${VERSION}/${src_file}"
curl --fail -LO --progress-bar "${BASE_DOWNLOAD_URL}/${VERSION}/${src_file}.asc"
curl --fail -LO --progress-bar "${BASE_DOWNLOAD_URL}/${VERSION}/${src_file}.sha512"

# It is probably redundant to check both the checksum and the signature but it's cheap and why not
echo "Validating SHA512 Checksum"
Expand All @@ -46,18 +51,18 @@ if ! (sha512sum "${src_file}" | cut -d " " -f 1 | diff -Z - "${src_file}.sha512"
fi

echo "Validating signature"
echo '--> NOTE: Make sure you have downloaded and added the KEYS file (https://www.apache.org/dist/druid/KEYS) to GPG: https://www.apache.org/info/verification.html (e.g. by using "curl https://archive.apache.org/dist/druid/KEYS | gpg --import")'

if ! (gpg --verify "${src_file}.asc" "${src_file}" 2> /dev/null); then
echo "ERROR: The signature could not be verified"
echo "--> Make sure you have imported the KEYS file (${BASE_DOWNLOAD_URL}/KEYS) into GPG: https://www.apache.org/info/verification.html"
echo "--> e.g. \"curl ${BASE_DOWNLOAD_URL}/KEYS | gpg --import\""
exit 1
fi

echo "Uploading everything to Nexus"
EXIT_STATUS=0
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "${src_file}" 'https://repo.stackable.tech/repository/packages/druid/' || EXIT_STATUS=$?
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "${src_file}.asc" 'https://repo.stackable.tech/repository/packages/druid/' || EXIT_STATUS=$?
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "${src_file}.sha512" 'https://repo.stackable.tech/repository/packages/druid/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "${src_file}" 'https://repo.stackable.tech/repository/packages/druid/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "${src_file}.asc" 'https://repo.stackable.tech/repository/packages/druid/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "${src_file}.sha512" 'https://repo.stackable.tech/repository/packages/druid/' || EXIT_STATUS=$?

if [ $EXIT_STATUS -ne 0 ]; then
echo "ERROR: Upload failed"
Expand Down
16 changes: 8 additions & 8 deletions hadoop/upload_new_hadoop_version.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -58,22 +58,22 @@ if ! (sha512sum --tag "$bin_file" | diff - "$bin_file.sha512" && sha512sum --tag
fi

echo "Validating signatures"
echo "--> NOTE: Make sure you have downloaded and added the KEYS file (${BASE_DOWNLOAD_URL}/KEYS) to GPG: https://www.apache.org/info/verification.html (e.g. by using \"curl ${BASE_DOWNLOAD_URL}/KEYS | gpg --import\")"

if ! (gpg --verify "$bin_file.asc" "$bin_file" 2> /dev/null && gpg --verify "$src_file.asc" "$src_file" 2> /dev/null); then
echo "ERROR: One of the signatures could not be verified"
echo "--> Make sure you have imported the KEYS file (${BASE_DOWNLOAD_URL}/KEYS) into GPG: https://www.apache.org/info/verification.html"
echo "--> e.g. \"curl ${BASE_DOWNLOAD_URL}/KEYS | gpg --import\""
exit 1
fi

echo "Uploading everything to Nexus"
EXIT_STATUS=0
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$bin_file" 'https://repo.stackable.tech/repository/packages/hadoop/' || EXIT_STATUS=$?
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$bin_file.asc" 'https://repo.stackable.tech/repository/packages/hadoop/' || EXIT_STATUS=$?
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$bin_file.sha512" 'https://repo.stackable.tech/repository/packages/hadoop/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$bin_file" 'https://repo.stackable.tech/repository/packages/hadoop/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$bin_file.asc" 'https://repo.stackable.tech/repository/packages/hadoop/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$bin_file.sha512" 'https://repo.stackable.tech/repository/packages/hadoop/' || EXIT_STATUS=$?

curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file" 'https://repo.stackable.tech/repository/packages/hadoop/' || EXIT_STATUS=$?
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file.asc" 'https://repo.stackable.tech/repository/packages/hadoop/' || EXIT_STATUS=$?
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file.sha512" 'https://repo.stackable.tech/repository/packages/hadoop/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file" 'https://repo.stackable.tech/repository/packages/hadoop/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file.asc" 'https://repo.stackable.tech/repository/packages/hadoop/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file.sha512" 'https://repo.stackable.tech/repository/packages/hadoop/' || EXIT_STATUS=$?

if [ $EXIT_STATUS -ne 0 ]; then
echo "ERROR: Upload failed"
Expand Down
22 changes: 14 additions & 8 deletions hbase/upload_new_hbase-operator-tools_version.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ set -euo pipefail
VERSION=${1:?"Missing version number argument (arg 1)"}
NEXUS_USER=${2:?"Missing Nexus username argument (arg 2)"}

# We prefer fast downloads...
BASE_DOWNLOAD_URL="https://dlcdn.apache.org/hbase"
# However, if the version is not available, use the slow archive instead:
# BASE_DOWNLOAD_URL="https://archive.apache.org/dist/hbase"

read -r -s -p "Nexus Password: " NEXUS_PASSWORD
echo ""

Expand Down Expand Up @@ -33,10 +38,10 @@ cd "$WORK_DIR" || exit

src_file=hbase-operator-tools-$VERSION-src.tar.gz

echo "Downloading hbase-operator-tools (this can take a while, it is intentionally downloading from a slow mirror that contains all old versions)"
curl --fail -LOs "https://archive.apache.org/dist/hbase/hbase-operator-tools-$VERSION/$src_file"
curl --fail -LOs "https://archive.apache.org/dist/hbase/hbase-operator-tools-$VERSION/$src_file.asc"
curl --fail -LOs "https://archive.apache.org/dist/hbase/hbase-operator-tools-$VERSION/$src_file.sha512"
echo "Downloading hbase-operator-tools (if this fails, try switching the BASE_DOWNLOAD_URL to the archive)"
curl --fail -LO --progress-bar "${BASE_DOWNLOAD_URL}/hbase-operator-tools-$VERSION/$src_file"
curl --fail -LO --progress-bar "${BASE_DOWNLOAD_URL}/hbase-operator-tools-$VERSION/$src_file.asc"
curl --fail -LO --progress-bar "${BASE_DOWNLOAD_URL}/hbase-operator-tools-$VERSION/$src_file.sha512"


# It is probably redundant to check both the checksum and the signature but it's cheap and why not
Expand All @@ -47,19 +52,20 @@ if ! (gpg --print-md SHA512 "$src_file" | diff - "$src_file.sha512"); then
fi

echo "Validating signatures"
echo '--> NOTE: Make sure you have downloaded and added the KEYS file (https://downloads.apache.org/hbase/KEYS) to GPG: https://www.apache.org/info/verification.html (e.g. by using "curl https://downloads.apache.org/hbase/KEYS | gpg --import")'
if ! (gpg --verify "$src_file.asc" "$src_file" 2> /dev/null); then
echo "ERROR: One of the signatures could not be verified"
echo "--> Make sure you have imported the KEYS file (${BASE_DOWNLOAD_URL}/KEYS) into GPG: https://www.apache.org/info/verification.html"
echo "--> e.g. \"curl ${BASE_DOWNLOAD_URL}/KEYS | gpg --import\""
exit 1
fi



echo "Uploading everything to Nexus"
EXIT_STATUS=0
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file" 'https://repo.stackable.tech/repository/packages/hbase-operator-tools/' || EXIT_STATUS=$?
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file.asc" 'https://repo.stackable.tech/repository/packages/hbase-operator-tools/' || EXIT_STATUS=$?
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file.sha512" 'https://repo.stackable.tech/repository/packages/hbase-operator-tools/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file" 'https://repo.stackable.tech/repository/packages/hbase-operator-tools/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file.asc" 'https://repo.stackable.tech/repository/packages/hbase-operator-tools/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file.sha512" 'https://repo.stackable.tech/repository/packages/hbase-operator-tools/' || EXIT_STATUS=$?


if [ $EXIT_STATUS -ne 0 ]; then
Expand Down
23 changes: 14 additions & 9 deletions hbase/upload_new_hbase_version.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ set -euo pipefail
VERSION=${1:?"Missing version number argument (arg 1)"}
NEXUS_USER=${2:?"Missing Nexus username argument (arg 2)"}

# We prefer fast downloads...
BASE_DOWNLOAD_URL="https://dlcdn.apache.org/hbase"
# However, if the version is not available, use the slow archive instead:
# BASE_DOWNLOAD_URL="https://archive.apache.org/dist/hbase"

read -r -s -p "Nexus Password: " NEXUS_PASSWORD
echo ""

Expand Down Expand Up @@ -33,11 +38,10 @@ cd "$WORK_DIR" || exit

src_file=hbase-$VERSION-src.tar.gz

echo "Downloading HBase source (this can take a while, it is intentionally downloading from a slow mirror that contains all old versions)"
curl --fail -LOs "https://archive.apache.org/dist/hbase/$VERSION/$src_file"
curl --fail -LOs "https://archive.apache.org/dist/hbase/$VERSION/$src_file.asc"
curl --fail -LOs "https://archive.apache.org/dist/hbase/$VERSION/$src_file.sha512"

echo "Downloading HBase source (if this fails, try switching the BASE_DOWNLOAD_URL to the archive)"
curl --fail -LO --progress-bar "${BASE_DOWNLOAD_URL}/$VERSION/$src_file"
curl --fail -LO --progress-bar "${BASE_DOWNLOAD_URL}/$VERSION/$src_file.asc"
curl --fail -LO --progress-bar "${BASE_DOWNLOAD_URL}/$VERSION/$src_file.sha512"

# It is probably redundant to check both the checksum and the signature but it's cheap and why not
echo "Validating SHA512 Checksums"
Expand All @@ -47,17 +51,18 @@ if ! (gpg --print-md SHA512 "$src_file" | diff - "$src_file.sha512"); then
fi

echo "Validating signatures"
echo '--> NOTE: Make sure you have downloaded and added the KEYS file (https://downloads.apache.org/hbase/KEYS) to GPG: https://www.apache.org/info/verification.html (e.g. by using "curl https://downloads.apache.org/hbase/KEYS | gpg --import")'
if ! (gpg --verify "$src_file.asc" "$src_file" 2> /dev/null); then
echo "ERROR: One of the signatures could not be verified"
echo "--> Make sure you have imported the KEYS file (${BASE_DOWNLOAD_URL}/KEYS) into GPG: https://www.apache.org/info/verification.html"
echo "--> e.g. \"curl ${BASE_DOWNLOAD_URL}/KEYS | gpg --import\""
exit 1
fi

echo "Uploading everything to Nexus"
EXIT_STATUS=0
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file" 'https://repo.stackable.tech/repository/packages/hbase/' || EXIT_STATUS=$?
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file.asc" 'https://repo.stackable.tech/repository/packages/hbase/' || EXIT_STATUS=$?
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file.sha512" 'https://repo.stackable.tech/repository/packages/hbase/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file" 'https://repo.stackable.tech/repository/packages/hbase/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file.asc" 'https://repo.stackable.tech/repository/packages/hbase/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file.sha512" 'https://repo.stackable.tech/repository/packages/hbase/' || EXIT_STATUS=$?

if [ $EXIT_STATUS -ne 0 ]; then
echo "ERROR: Upload failed"
Expand Down
22 changes: 14 additions & 8 deletions hbase/upload_new_phoenix_version.sh
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -5,6 +5,11 @@ set -euo pipefail
VERSION=${1:?"Missing version number argument (arg 1)"}
NEXUS_USER=${2:?"Missing Nexus username argument (arg 2)"}

# We prefer fast downloads...
BASE_DOWNLOAD_URL="https://dlcdn.apache.org/phoenix"
# However, if the version is not available, use the slow archive instead:
# BASE_DOWNLOAD_URL="https://archive.apache.org/dist/phoenix"

read -r -s -p "Nexus Password: " NEXUS_PASSWORD
echo ""

Expand Down Expand Up @@ -33,10 +38,10 @@ cd "$WORK_DIR" || exit

src_file=phoenix-$VERSION-src.tar.gz

echo "Downloading phoenix (this can take a while, it is intentionally downloading from a slow mirror that contains all old versions)"
curl --fail -LOs "https://archive.apache.org/dist/phoenix/phoenix-$VERSION/$src_file"
curl --fail -LOs "https://archive.apache.org/dist/phoenix/phoenix-$VERSION/$src_file.asc"
curl --fail -LOs "https://archive.apache.org/dist/phoenix/phoenix-$VERSION/$src_file.sha512"
echo "Downloading phoenix (if this fails, try switching the BASE_DOWNLOAD_URL to the archive)"
curl --fail -LO --progress-bar "${BASE_DOWNLOAD_URL}/phoenix-$VERSION/$src_file"
curl --fail -LO --progress-bar "${BASE_DOWNLOAD_URL}/phoenix-$VERSION/$src_file.asc"
curl --fail -LO --progress-bar "${BASE_DOWNLOAD_URL}/phoenix-$VERSION/$src_file.sha512"


# It is probably redundant to check both the checksum and the signature but it's cheap and why not
Expand All @@ -47,19 +52,20 @@ if ! (gpg --print-md SHA512 "$src_file" | diff - "$src_file.sha512"); then
fi

echo "Validating signatures"
echo '--> NOTE: Make sure you have downloaded and added the KEYS file (https://downloads.apache.org/phoenix/KEYS) to GPG: https://www.apache.org/info/verification.html (e.g. by using "curl https://downloads.apache.org/phoenix/KEYS | gpg --import")'
if ! (gpg --verify "$src_file.asc" "$src_file" 2> /dev/null); then
echo "ERROR: One of the signatures could not be verified"
echo "--> Make sure you have imported the KEYS file (${BASE_DOWNLOAD_URL}/KEYS) into GPG: https://www.apache.org/info/verification.html"
echo "--> e.g. \"curl ${BASE_DOWNLOAD_URL}/KEYS | gpg --import\""
exit 1
fi



echo "Uploading everything to Nexus"
EXIT_STATUS=0
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file" 'https://repo.stackable.tech/repository/packages/phoenix/' || EXIT_STATUS=$?
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file.asc" 'https://repo.stackable.tech/repository/packages/phoenix/' || EXIT_STATUS=$?
curl --fail -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file.sha512" 'https://repo.stackable.tech/repository/packages/phoenix/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file" 'https://repo.stackable.tech/repository/packages/phoenix/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file.asc" 'https://repo.stackable.tech/repository/packages/phoenix/' || EXIT_STATUS=$?
curl --fail -o /dev/null --progress-bar -u "$NEXUS_USER:$NEXUS_PASSWORD" --upload-file "$src_file.sha512" 'https://repo.stackable.tech/repository/packages/phoenix/' || EXIT_STATUS=$?


if [ $EXIT_STATUS -ne 0 ]; then
Expand Down
Loading