Skip to content

Commit 41e0732

Browse files
stephan48ph3-der-loewe
authored andcommitted
Feature: Build Dist Archives via CI and Upload them to OBS
1 parent c78f01e commit 41e0732

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

60 files changed

+5577
-5
lines changed

.gitignore

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -60,3 +60,7 @@ config.h.in~
6060

6161
# Ignore vim swap files
6262
.*.swp
63+
64+
65+
# CI: OSC temp stuff
66+
/osc_tmp

.gitlab-ci.yml

Lines changed: 51 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,5 @@
1-
# Build Icecast
2-
build:
1+
# Build Icecast - test
2+
make_dist:
33
variables:
44
GIT_SUBMODULE_STRATEGY: recursive
55

@@ -12,8 +12,11 @@ build:
1212

1313
before_script:
1414
- apk update
15+
- cat /etc/os*
1516
- apk add musl-dev git make gcc automake autoconf libtool
1617
- apk add curl-dev libogg-dev libvorbis-dev libxslt-dev libxml2-dev
18+
# gzip required because busybox gzip does not understand best / zip for make dist
19+
- apk add gzip zip tar
1720
# Required for tests
1821
#- apk add curl ffmpeg
1922
# Create user to run tests
@@ -22,7 +25,52 @@ build:
2225
script:
2326
- ./autogen.sh
2427
- ./configure || cat config.log
25-
- make
28+
- make dist
29+
- ls -la
2630
# Tests
2731
#- su -c "./tests/admin-tests.sh" icecast
2832

33+
artifacts:
34+
paths:
35+
- icecast-*.tar.gz
36+
- icecast-*.zip
37+
expire_in: 1 week
38+
39+
upload_dist:
40+
tags:
41+
- docker
42+
- linux
43+
44+
only:
45+
- master
46+
- devel
47+
48+
stage: deploy
49+
50+
dependencies:
51+
- make_dist
52+
53+
before_script:
54+
- ./ci/osc/prepare-osc-tools.sh
55+
56+
script:
57+
- ./ci/osc/handle-osc-upload.sh
58+
59+
upload_dist_release:
60+
tags:
61+
- docker
62+
- linux
63+
64+
only:
65+
- tags
66+
67+
stage: deploy
68+
69+
dependencies:
70+
- make_dist
71+
72+
before_script:
73+
- ./ci/osc/prepare-osc-tools.sh
74+
75+
script:
76+
- ./ci/osc/handle-osc-upload.sh release

ci/README.md

Lines changed: 49 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,49 @@
1+
# Icecast CI for automated OBS Uploads
2+
3+
This directory contains scripting triggered by .gitlab-ci.yml to automatically upload releases and nightlies to build.opensuse.org.
4+
The Pipeline triggers are set so that releases are pushed only on tags and nightlies on master/devel commits into respective OBS projects.
5+
These are defined in the ci/osc/ for releases and nightlies in their respective config files.
6+
7+
# How to release
8+
9+
To make a new release please call the magic version changer as following from the repo root:
10+
11+
```
12+
export ICECAST_BETA_VERSION=3
13+
ci/create-changelog-and-set-versions.sh "$ICECAST_BETA_VERSION" "2.4.99.$ICECAST_BETA_VERSION" "2.5 beta$ICECAST_BETA_VERSION" "25-beta-$ICECAST_BETA_VERSION" "2.5-beta$ICECAST_BETA_VERSION" "_VERSION_ARCHIVE_" "2.4.99.$ICECAST_BETA_VERSION" "now" "Stephan Jauernick <[email protected]>" "Preparing for 2.5 beta$ICECAST_BETA_VERSION aka 2.4.99.$ICECAST_BETA_VERSION" "icecast" "mingw32-icecast" "mingw32-icecast-installer"
14+
```
15+
16+
Please adapt the "ICECAST_BETA_VERSION", the Author, the Date(now; please enter a valid ISO8601 Date if needed) and the Message as needed.
17+
18+
This script/mechanism will update all the version references and then show you a git status/diff. Please check all changes and commit them as needed.
19+
20+
After tagging and uploading the release will be picked up by gitlabs CI and a release will be pushed to OBS.
21+
22+
# Nightlies
23+
24+
A nightly will be build on each change to master/devel - these will be marked with a git version + build date in the version and a "correct" changelog entry where relevant.
25+
26+
# Required Repo Configuration in Gitlab
27+
28+
- master and devel need to be set to protected with Maintainer only push/merge permissions
29+
- all tags need to be set to protected with Maintainer only permissions
30+
- There needs to be a "File" Variable configured as follows:
31+
32+
Name: OSC_RC
33+
Protected only: yes
34+
Content:
35+
```
36+
[general]
37+
apiurl = https://api.opensuse.org
38+
39+
[https://api.opensuse.org]
40+
user = OBS USER
41+
pass = OBS PASSWORD
42+
```
43+
44+
The referenced user needs to have Maintainer access to the OBS projects referenced in the build configuration files.
45+
46+
# How to improve
47+
48+
Make version update script more robust... currently it will probably explode when we do a final 2.5 release.
49+
Rework OBS spec and debian files to modern standards.
Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,68 @@
1+
#!/bin/bash -xe
2+
3+
BETA_VERSION=${1:?Missing Beta Version, Use 2}; shift
4+
SHORT_VERSION=${1:?Missing Short Version, Use 2.4.99.2}; shift
5+
STRANGE_VERSION=${1:?Missing Strange Version, Use '2.5 beta2'}; shift
6+
HTML_VERSION=${1:?Missing HTML Version, Use '25-beta-2'}; shift
7+
WIN32_VERSION=${1:?Missing Win32 Version, Use '2.5-beta2'}; shift
8+
ARCHIVE_VERSION=${1:?Missing Archive Version, Use '2.4.99.2' for ci or _VERSION_ARCHIVE_ for release}; shift
9+
CI_VERSION=${1:?Missing CI Version - Use 2.4.99.2+bla}; shift
10+
DATE=`date --date=${1:?ISO DATE or now} --iso-8601=seconds`; shift
11+
AUTHOR=${1:?Mail Address}; shift
12+
TEXT=${1:?Release Text}; shift
13+
ICECAST_PROJECT=${1:?Icecast OSC Project Name}; shift
14+
W32_ICECAST_PROJECT=${1:?Icecast W32 OSC Project Name}; shift
15+
W32_ICECAST_INSTALLER_PROJECT=${1:?Icecast W32 Installer OSC Project Name}; shift
16+
17+
OSC_BASE_DIR=osc_tmp
18+
19+
# upon release we modify the templates - in ci we modiy temporary files
20+
if [ "$ARCHIVE_VERSION" = "_VERSION_ARCHIVE_" ]; then
21+
OSC_BASE_DIR=ci/osc
22+
fi
23+
24+
pushd $OSC_BASE_DIR
25+
26+
sed -i "1s#^#icecast2 ($CI_VERSION-1) UNRELEASED; urgency=medium\n\n * $TEXT\n\n -- $AUTHOR `date --date=$DATE +"%a, %d %b %Y %H:%M:%S %z"`\n\n#" $ICECAST_PROJECT/debian/changelog
27+
28+
for i in "$ICECAST_PROJECT/$ICECAST_PROJECT.spec" "$W32_ICECAST_INSTALLER_PROJECT/$W32_ICECAST_INSTALLER_PROJECT.spec" "$W32_ICECAST_PROJECT/$W32_ICECAST_PROJECT.spec"; do
29+
sed -i "s/_VERSION_ARCHIVE_/$ARCHIVE_VERSION/; s/^Version:\(\s*\)[^\s]*$/Version:\1$CI_VERSION/; s#^%changelog.*\$#\0\n* `date --date=$DATE +"%a %b %d %Y"` $AUTHOR - $CI_VERSION-1\n- $TEXT\n\n#" "$i";
30+
done
31+
32+
sed -i "s/\(icecast_win32_\).*\(.exe\)/\1$WIN32_VERSION\2/" $W32_ICECAST_INSTALLER_PROJECT/$W32_ICECAST_INSTALLER_PROJECT.spec
33+
34+
popd
35+
36+
# we only update the changelog for releases - until i figure out if we want to run the magic script pre CI
37+
if [ "$ARCHIVE_VERSION" = "_VERSION_ARCHIVE_" ]; then
38+
sed -i "1s#^#`date --date=$DATE +"%Y-%m-%d %H:%M:%S"` $AUTHOR\n\n * $TEXT\n\n#" ChangeLog
39+
fi
40+
41+
sed -i "1s#\[[.0-9]*\]#[$SHORT_VERSION]#" configure.ac
42+
43+
sed -i "s/Icecast .* Documentation/Icecast $STRANGE_VERSION Documentation/; s/icecast-.*-documentation/icecast-$HTML_VERSION-documentation/" doc/index.html
44+
45+
sed -i "s/\(\"DisplayVersion\" \"\).*\(\"\)$/\1$STRANGE_VERSION\2/" win32/icecast.nsis
46+
sed -i "s/\(OutFile \"icecast_win32_\).*\(.exe\"\)$/\1$WIN32_VERSION\2/" win32/icecast.nsis
47+
48+
sed -i "s/^\(export ICECAST_VERSION=\).*$/\1$SHORT_VERSION/; s/\(export ICECAST_BETA_VERSION=\).*$/\1$BETA_VERSION/" ci/osc/*-config.sh
49+
50+
# we only do this for release builds
51+
if [ "$ARCHIVE_VERSION" = "_VERSION_ARCHIVE_" ]; then
52+
sed -i "s/^\(export RELEASE_AUTHOR=\"\).*\(\"\)$/\1$AUTHOR\2/; s/\(export RELEASE_DATETIME=\).*$/\1$DATE/" ci/osc/release-config.sh
53+
fi
54+
55+
if [ "$ARCHIVE_VERSION" != "_VERSION_ARCHIVE_" ]; then
56+
if ! git diff --quiet; then
57+
echo "git detected differences after ci driven create changelog run, this should not happen - please check";
58+
git status
59+
git --no-pager diff
60+
exit 1;
61+
else
62+
echo "no repo diffs detected, this is good as CI should not change the repo(only temp files)!"
63+
fi
64+
else
65+
echo "applied changes to versions, please verify and commit them for a new release"
66+
git status
67+
git --no-pager diff
68+
fi

ci/fix-dsc.sh

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,21 @@
1+
#!/bin/bash -xe
2+
3+
BASE=`dirname $0`
4+
TARGET=icecast.dsc
5+
TARBASE=icecast2_${ICECAST_CI_VERSION:?Missing ICECAST_CI_VERSION}
6+
sed "s/^Version: .*$/Version: $ICECAST_CI_VERSION-1/" $BASE/$TARGET.templ > $TARGET
7+
8+
function helper {
9+
HEADER=$1
10+
FUNCTION=$2
11+
12+
echo "$HEADER" >> $TARGET
13+
for FILE in $TARBASE.orig.tar.gz $TARBASE-1.debian.tar.gz; do
14+
echo -e " `$FUNCTION $FILE | awk '{ print \$1 }'` `stat -c '%s' $FILE` $FILE" >> $TARGET
15+
done
16+
}
17+
18+
19+
helper "Checksums-Sha1:" "sha1sum"
20+
helper "Checksums-Sha256:" "sha256sum"
21+
helper "Files:" "md5sum"

ci/icecast.dsc.templ

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
Format: 3.0 (quilt)
2+
Source: icecast2
3+
Binary: icecast2
4+
Architecture: any
5+
Version: 2.4.99.2-1
6+
Maintainer: Thomas B. Ruecker <thomas@ruecker.fi>
7+
Uploaders: Thomas B. Ruecker <thomas@ruecker.fi>
8+
Homepage: http://www.icecast.org/
9+
Standards-Version: 3.9.5
10+
Build-Depends: cdbs, autotools-dev, devscripts, libtool, automake1.11, autoconf, debhelper, dh-buildinfo, libogg-dev (>> 1.0.0), libvorbis-dev (>> 1.0.0), libxslt1-dev | libxslt-dev, libxml2-dev, libcurl4-gnutls-dev, libtheora-dev (>= 0.0.0.alpha7), libspeex-dev, libssl-dev, po-debconf, pkg-config, dh-autoreconf, automake
11+
Package-List:
12+
icecast2 deb sound optional arch=any

ci/osc/handle-osc-upload.sh

Lines changed: 85 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,85 @@
1+
#!/bin/bash -xe
2+
3+
SCRIPT_DIR=`dirname $0`
4+
SCRIPT_DIR=`realpath $SCRIPT_DIR`
5+
CONFIG=${1:-nightly}
6+
7+
. $SCRIPT_DIR/$CONFIG-config.sh
8+
9+
: "${OSC_RC:?Variable OSC_RC not set or empty}"
10+
: "${CI_PIPELINE_URL:?Variable CI_PIPELINE_URL not set or empty}"
11+
12+
pwd
13+
14+
ls -la
15+
16+
rm -rf osc_tmp
17+
mkdir -p osc_tmp
18+
cd osc_tmp
19+
20+
export HOME=`pwd`
21+
export SOURCE=`pwd`/..
22+
export OSC_CMD="osc-wrapper.py --config=$OSC_RC"
23+
24+
# checkout into a dkirectory named like the project - avoiding having it in a subdir called OBS_BASE
25+
for i in "$ICECAST_PROJECT" "$W32_ICECAST_INSTALLER_PROJECT" "$W32_ICECAST_PROJECT"; do
26+
$OSC_CMD checkout -o "$i" $OBS_BASE "$i"
27+
rm -vrf "$i"/*
28+
done
29+
30+
# no comment needed
31+
for i in "$ICECAST_PROJECT" "$W32_ICECAST_PROJECT"; do
32+
cp $SOURCE/icecast-$ICECAST_VERSION.tar.gz "$i"/icecast2_$ICECAST_CI_VERSION.orig.tar.gz
33+
done
34+
35+
# we copy the spec for these projects - for the icecast project the spec is globeed
36+
for i in "$W32_ICECAST_INSTALLER_PROJECT" "$W32_ICECAST_PROJECT"; do
37+
cp -a $SCRIPT_DIR/$i/$i.spec $i/
38+
done
39+
40+
# this is more complex because we have more files.
41+
cp -a $SCRIPT_DIR/$ICECAST_PROJECT/icecast* $ICECAST_PROJECT/
42+
cp -a $SCRIPT_DIR/$ICECAST_PROJECT/debian $ICECAST_PROJECT/
43+
44+
if [ "$DISABLE_CHANGELOG" == "0" ]; then
45+
pushd $SOURCE
46+
$SCRIPT_DIR/../create-changelog-and-set-versions.sh "$ICECAST_BETA_VERSION" "$ICECAST_VERSION" "2.5 beta$ICECAST_BETA_VERSION" "25-beta-$ICECAST_BETA_VERSION" "2.5-beta$ICECAST_BETA_VERSION" "$ICECAST_VERSION" "$ICECAST_CI_VERSION" "$RELEASE_DATETIME" "$RELEASE_AUTHOR" "CI Build - $CI_PIPELINE_URL" "$ICECAST_PROJECT" "$W32_ICECAST_INSTALLER_PROJECT" "$W32_ICECAST_PROJECT"
47+
popd
48+
else
49+
for i in "$ICECAST_PROJECT/$ICECAST_PROJECT.spec" "$W32_ICECAST_INSTALLER_PROJECT/$W32_ICECAST_INSTALLER_PROJECT.spec" "$W32_ICECAST_PROJECT/$W32_ICECAST_PROJECT.spec"; do
50+
sed -i "s/_VERSION_ARCHIVE_/$ICECAST_VERSION/;" "$i";
51+
done
52+
fi
53+
54+
tar -C $ICECAST_PROJECT -cvzf $ICECAST_PROJECT/icecast2_$ICECAST_CI_VERSION-1.debian.tar.gz debian/
55+
56+
# remove debian/ so it does not end up in the archive
57+
rm -rf $ICECAST_PROJECT/debian
58+
59+
# we fix the dsc to have the correct hashsums so dpkg-buildpackage and co do not complain
60+
61+
pushd $ICECAST_PROJECT
62+
63+
$SCRIPT_DIR/../fix-dsc.sh
64+
65+
popd
66+
67+
# we use addremove to detect changes and commit them to the server
68+
for i in "$ICECAST_PROJECT" "$W32_ICECAST_INSTALLER_PROJECT" "$W32_ICECAST_PROJECT"; do
69+
pushd $i
70+
71+
$OSC_CMD addremove
72+
$OSC_CMD diff
73+
$OSC_CMD commit -m "Commit via $CI_PIPELINE_URL"
74+
75+
popd
76+
done
77+
78+
# we cleanup because the OSC_RC should not remain on disk
79+
if [ "$NOCLEANUP" != "1" ]; then
80+
shred -vzf $OSC_RC .osc_cookiejar
81+
cd ..
82+
echo > $OSC_RC
83+
rm -rf "osc_tmp"
84+
fi
85+
Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,30 @@
1+
icecast2 for Debian
2+
-------------------
3+
4+
In the Debian packaging the configuration files have been symlinked from
5+
the upstream location below /usr/share to /etc. This is needed to
6+
satisfy FHS (/usr/share are for static content only).
7+
If running icecast2 in a chroot environment, beware that the symlinks to
8+
/etc will break. A possible (untestet!) solution might be to manually
9+
put the configuration files back below /usr/share - and revert the hack
10+
again before updating the package!
11+
12+
-- Jonas Smedegaard <[email protected]> Thu, 20 May 2004 21:04:27 +0200
13+
14+
15+
It is recommended to run icecast under a dedicated user account, which only
16+
has access to write the log files. The Debian package creates such an
17+
account, named 'icecast2', and uses it by default, but you are free to
18+
reconfigure it and remove the account.
19+
20+
Edit /etc/default/icecast2 to change the init-script configuration.
21+
22+
It is possible (but discouraged for security reasons) to bind to a
23+
privileged port (like standard web port 80). Edit /etc/init.d/icecast2
24+
to not change userid and instead set the correct userid and group in
25+
/etc/icecast2/icecast2.xml. Beware that this way you rely on the
26+
icecast2 binary to properly drop privileges (instead of the much more
27+
thoroughly audited start-stop-daemon). Thanks to Jürgen A. Erhard
28+
<[email protected]> for the tip.
29+
30+
-- Keegan Quinn <[email protected]>

0 commit comments

Comments
 (0)