diff --git a/ci/arm/README.md b/ci/arm/README.md new file mode 100644 index 0000000000..86e09ad89e --- /dev/null +++ b/ci/arm/README.md @@ -0,0 +1,117 @@ +# ARM CI Directory + +## Folder overview + +This directory contains scripts and configuration for building Scopy for ARM platforms (both 32-bit armhf and 64-bit aarch64). The build process uses cross-compilation toolchains and creates AppImage packages for ARM devices like Raspberry Pi. More information about this build can be found here [Creating an Installer for other architectures](https://analogdevicesinc.github.io/scopy/user_guide/build_instructions/linuxBuild.html#creating-an-installer-for-other-architectures). + +### Scripts + +#### `arm_build_config.sh` - Configuration file that sets up build environment variables + +#### `arm_build_process.sh` - Main build orchestration script + +#### `create_sysroot.sh` - Creates a sysroot from ADI Kuiper Linux image, the sysroot is used during cross-compilation + +#### `build_qt.sh` - Cross-compiles Qt 5.15.16 for ARM + +#### `create_docker_image.sh` - Creates Docker images for ARM builds + +#### `docker/Dockerfile` - Base image configuration for ARM build environment + +#### `copy-deps.sh` - Copies runtime dependencies into AppImage directory + +#### `inside_chroot_arm64.sh` / `inside_chroot_armhf.sh` - Scripts that run inside chroot environment + +#### `local_build_scopy_for_kuiper.sh` - Local build script specifically for Kuiper Linux + +### Supporting Files + +#### `cmake_toolchain.cmake` - Contains the toolchain used for cross-compiling + +#### `qt_patch_arm32/arm64.patch` - A patch to some Qt files to be able to cross-compile it for arm32/64 + +#### `runtime-armhf/aarch64` - This is the runtime used inside the AppImage + +## Build Process + +### Building Scopy using the Docker Image + +`Prerequisites:` + +- [Docker](https://docs.docker.com/desktop/setup/install/windows-install/) +- the rest of the dependencies are installed using the scripts + +This is by far the easiest method of building an armhf or arm64 Scopy AppImage. + +A temporary Docker volume is created to link the local environment with the Docker container. The compilation process takes place inside the container using +the pre-configured filesystem and all the changes inside the volume will reflect to the locally stored files. + +The next steps will describe how to build Scopy and create a armhf(arm32) AppImage, the process is the same for arm64(aarch64), just change the argument. + +1. **Install the packages needed for emulating the required architecture** + + ```bash + ci/arm/create_sysroot.sh arm32 install_packages install_qemu + ``` + +2. **Pull the Docker image** + + ```bash + docker pull cristianbindea/scopy2-armhf-appimage:latest + ``` + +3. **Run the image, while creating a docker volume** + + ```bash + docker run -it --mount type=bind,source=path/to/scopy/repository,target=/home/runner/scopy cristianbindea/scopy2-armhf-appimage:latest + ``` + +4. **Using the Docker environment you can compile and package the application with one command:** + + ```bash + /home/runner/scopy/ci/arm/arm_build_process.sh arm32 run_workflow + ``` + +`TIP:` Inspect the arm_build_process.sh file. You can call any function inside that file using **$ ./arm_build_process.sh architecture function**. +This way you can only build Scopy, you can create an AppDir folder or even run the whole workflow and create the AppImage. + +Finally, after the development is done use this to clean the system + + ```bash + docker container ls -a # get the container id + docker container rm -v (container id) + docker image rm cristianbindea/scopy2-armhf-appimage:latest + ``` + +### Building the Docker Image + +To build the Docker image, just run the script and select the required architecture. + + ```bash + ci/arm/create_docker_image.sh arm32 run_workflow + # or + ci/arm/create_docker_image.sh arm64 run_workflow + ``` + +### Building locally from sources + +In order to build Scopy locally you will need a device with the armhf or arm64 architecture, for example a Raspberry Pi. + +To build the app directly on the Raspberry Pi, just run: + + ```bash + ci\arm\local_build_scopy_for_kuiper.sh install_apt clone buid_deps build_scopy + ``` + +This script is made for [Kuiper Linux](https://wiki.analog.com/resources/tools-software/linux-software/kuiper-linux), but you can adapt it for another distro by updating the packages installed via apt. + +## CI Integration + +- **GitHub Actions**: `.github/workflows/appimage-armhf.yml` +- **GitHub Actions**: `.github/workflows/appimage-arm64.yml` + +## Notes + +- Uses ADI Kuiper Linux as the base system +- Requires significant disk space (~15GB) for sysroot and build artifacts +- Build time can be several hours depending on hardware diff --git a/ci/arm/arm_build_config.sh b/ci/arm/arm_build_config.sh index 86a5775e0a..3e6b88426e 100755 --- a/ci/arm/arm_build_config.sh +++ b/ci/arm/arm_build_config.sh @@ -1,5 +1,15 @@ #!/bin/bash +# ARM Cross-Compilation Configuration Script +# ========================================== +# Sets up all environment variables and paths needed for ARM cross-compilation +# Usage: source arm_build_config.sh [arm32|arm64] +# Note: This script must be sourced, not executed directly + + +# Validate and set architecture-specific variables +# arm32: 32-bit ARM (armhf) +# arm64: 64-bit ARM (aarch64) if [ "$1" == "arm64" ];then echo "Building for aarch64" TOOLCHAIN_HOST="aarch64-linux-gnu" @@ -17,6 +27,9 @@ else exit fi + +# These branches/tags define which version of each dependency to build +# Update these when you need to use different versions of dependencies LIBSERIALPORT_BRANCH=master LIBIIO_VERSION=v0.26 LIBAD9361_BRANCH=main @@ -36,77 +49,82 @@ ECM_BRANCH=kf5 KARCHIVE_BRANCH=kf5 GENALYZER_BRANCH=main +# This environment variable tells the C++ code to use relative paths for libraries +# When set to 1, the application looks for libraries in the AppImage bundle +# instead of system locations, making the package portable export APPIMAGE=1 +# SRC_SCRIPT: Directory containing this script (ci/arm) +# STAGING_AREA: Main working directory for builds SRC_SCRIPT=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) [ $CI_SCRIPT == "ON" ] && STAGING_AREA=$HOME/scopy/ci/arm/staging || STAGING_AREA=$SRC_SCRIPT/staging -SYSROOT=$STAGING_AREA/sysroot + +SYSROOT=$STAGING_AREA/sysroot # root filesystem extracted from Kuiper Linux image SYSROOT_TAR=$STAGING_AREA/sysroot.tar.gz -TOOLCHAIN=$STAGING_AREA/cross-pi-gcc +TOOLCHAIN=$STAGING_AREA/cross-pi-gcc # Cross-compiler location TOOLCHAIN_BIN=$TOOLCHAIN/bin TOOLCHAIN_FILE=$SRC_SCRIPT/cmake_toolchain.cmake -QT_LOCATION=$SYSROOT/usr/lib/$TOOLCHAIN_HOST/qt5 -if [ $TOOLCHAIN_HOST == "aarch64-linux-gnu" ]; then - QT_BUILD_DEVICE=linux-rasp-pi4-v3d-g++ -elif [ $TOOLCHAIN_HOST == "arm-linux-gnueabihf" ]; then - QT_BUILD_DEVICE=linux-rasp-pi3-vc4-g++ -fi +QT_LOCATION=$SYSROOT/usr/lib/$TOOLCHAIN_HOST/qt5 +QT_BUILD_LOCATION=$QT_LOCATION # The location where Qt will be installed in the system +QT_SYSTEM_LOCATION=/usr/lib/$TOOLCHAIN_HOST/qt5 # The Qt location relative to the sysroot folder CMAKE_BIN=$STAGING_AREA/cmake/bin/cmake QMAKE_BIN=$QT_LOCATION/bin/qmake -JOBS=-j14 -APP_DIR=$SRC_SCRIPT/scopy.AppDir -APP_IMAGE=$SRC_SCRIPT/Scopy.AppImage -APP_RUN=$SRC_SCRIPT/../general/AppRun -APP_DESKTOP=$SRC_SCRIPT/../general/scopy.desktop +JOBS=-j14 # Parallel build configuration + +CMAKE_OPTS=(\ + -DCMAKE_INSTALL_PREFIX="$SYSROOT" \ + -DCMAKE_TOOLCHAIN_FILE="$TOOLCHAIN_FILE" \ + -DCMAKE_BUILD_TYPE=RelWithDebInfo \ + ) + +CMAKE="$CMAKE_BIN ${CMAKE_OPTS[*]}" # Complete CMake command with all options + +CROSS_COMPILER=$TOOLCHAIN + +APP_DIR=$SRC_SCRIPT/scopy.AppDir # Temporary directory structure that becomes the AppImage +APP_IMAGE=$SRC_SCRIPT/Scopy.AppImage # Final AppImage executable file +APP_RUN=$SRC_SCRIPT/../general/AppRun # Entry point script that launches the application +APP_DESKTOP=$SRC_SCRIPT/../general/scopy.desktop # Desktop entry file for Linux desktop integration APP_SQUASHFS=$SRC_SCRIPT/scopy.squashfs -# Runetimes downloaded from https://github.com/AppImage/type2-runtime/releases/tag/20251108 8 Dec 2025 +# AppImage runtime for ARM +# Downloaded from https://github.com/AppImage/AppImageKit/releases/continuous +# The runtime is a small executable that mounts and runs the AppImage RUNTIME_ARM=$SRC_SCRIPT/runtime-$ARCHITECTURE - -# The exports below ensure these variables are available to the toolchain file. +# These exports make variables available to cmake_toolchain.cmake +# The toolchain file needs these to properly configure cross-compilation export CMAKE_SYSROOT="$SYSROOT" export QT_LOCATION="$QT_LOCATION" export STAGING_AREA="$STAGING_AREA" export CMAKE_SYSTEM_PROCESSOR="$CMAKE_SYSTEM_PROCESSOR" export CMAKE_LIBRARY_ARCHITECTURE="$CMAKE_LIBRARY_ARCHITECTURE" -CMAKE_OPTS=(\ - -DCMAKE_INSTALL_PREFIX="$SYSROOT" \ - -DCMAKE_TOOLCHAIN_FILE="$TOOLCHAIN_FILE" \ - -DCMAKE_BUILD_TYPE=RelWithDebInfo \ - ) - -CMAKE="$CMAKE_BIN ${CMAKE_OPTS[*]}" - -QT_BUILD_LOCATION=$QT_LOCATION # the location where Qt will be installed in the system -QT_SYSTEM_LOCATION=/usr/lib/$TOOLCHAIN_HOST/qt5 # the Qt location relative to the sysroot folder -CROSS_COMPILER=$TOOLCHAIN +# CMake 3.29 +CMAKE_DOWNLOAD_LINK=https://github.com/Kitware/CMake/releases/download/v3.29.0-rc2/cmake-3.29.0-rc2-linux-x86_64.tar.gz +# Qt 5.15.16 LTS source code +QT_DOWNLOAD_LINK=https://download.qt.io/archive/qt/5.15/5.15.16/single/qt-everywhere-opensource-src-5.15.16.tar.xz +# Script to fix absolute symlinks in sysroot +SYSROOT_RELATIVE_LINKS=https://raw.githubusercontent.com/abhiTronix/rpi_rootfs/master/scripts/sysroot-relativelinks.py +# Sets download URLs and Qt device specs based on target architecture if [ $TOOLCHAIN_HOST == "aarch64-linux-gnu" ]; then + QT_BUILD_DEVICE=linux-rasp-pi4-v3d-g++ CROSSCOMPILER_DOWNLOAD_LINK=https://sourceforge.net/projects/raspberry-pi-cross-compilers/files/Bonus%20Raspberry%20Pi%20GCC%2064-Bit%20Toolchains/Raspberry%20Pi%20GCC%2064-Bit%20Cross-Compiler%20Toolchains/Bookworm/GCC%2012.2.0/cross-gcc-12.2.0-pi_64.tar.gz + KUIPER_DOWNLOAD_LINK=https://github.com/analogdevicesinc/adi-kuiper-gen/releases/download/v2.0.0/image_2025-04-03-ADI-Kuiper-Linux-arm64.zip + IMAGE_NAME="2025-04-03-ADI-Kuiper-Linux-arm64" + elif [ $TOOLCHAIN_HOST == "arm-linux-gnueabihf" ]; then + QT_BUILD_DEVICE=linux-rasp-pi3-vc4-g++ # bookwork #CROSSCOMPILER_DOWNLOAD_LINK=https://sourceforge.net/projects/raspberry-pi-cross-compilers/files/Raspberry%20Pi%20GCC%20Cross-Compiler%20Toolchains/Bookworm/GCC%2012.2.0/Raspberry%20Pi%202%2C%203/cross-gcc-12.2.0-pi_2-3.tar.gz # bullseye, with armv8 flags #CROSSCOMPILER_DOWNLOAD_LINK=https://sourceforge.net/projects/raspberry-pi-cross-compilers/files/Raspberry%20Pi%20GCC%20Cross-Compiler%20Toolchains/Bullseye/GCC%2010.2.0/Raspberry%20Pi%203A%2B%2C%203B%2B%2C%204%2C%205/cross-gcc-10.2.0-pi_3%2B.tar.gz # compiler with armv7 flags CROSSCOMPILER_DOWNLOAD_LINK=https://sourceforge.net/projects/raspberry-pi-cross-compilers/files/Raspberry%20Pi%20GCC%20Cross-Compiler%20Toolchains/Bullseye/GCC%2010.2.0/Raspberry%20Pi%202%2C%203/cross-gcc-10.2.0-pi_2-3.tar.gz -fi - -CMAKE_DOWNLOAD_LINK=https://github.com/Kitware/CMake/releases/download/v3.29.0-rc2/cmake-3.29.0-rc2-linux-x86_64.tar.gz - -if [ $TOOLCHAIN_HOST == "aarch64-linux-gnu" ]; then - KUIPER_DOWNLOAD_LINK=https://github.com/analogdevicesinc/adi-kuiper-gen/releases/download/v2.0.0/image_2025-04-03-ADI-Kuiper-Linux-arm64.zip - IMAGE_NAME="2025-04-03-ADI-Kuiper-Linux-arm64" -elif [ $TOOLCHAIN_HOST == "arm-linux-gnueabihf" ]; then KUIPER_DOWNLOAD_LINK=https://swdownloads.analog.com/cse/kuiper/image_2023-12-13-ADI-Kuiper-full.zip IMAGE_NAME="2023-12-13-ADI-Kuiper-full" fi - -QT_DOWNLOAD_LINK=https://download.qt.io/archive/qt/5.15/5.15.16/single/qt-everywhere-opensource-src-5.15.16.tar.xz - -SYSROOT_RELATIVE_LINKS=https://raw.githubusercontent.com/abhiTronix/rpi_rootfs/master/scripts/sysroot-relativelinks.py diff --git a/ci/arm/arm_build_process.sh b/ci/arm/arm_build_process.sh index 4cd5cd1982..11b806cd38 100755 --- a/ci/arm/arm_build_process.sh +++ b/ci/arm/arm_build_process.sh @@ -1,10 +1,30 @@ #!/bin/bash +# ARM Cross-Compilation Build Process Script +# ========================================== +# Orchestrates the complete build process for ARM platforms +# Usage: ./arm_build_process.sh [arm32|arm64] [function_name ...] +# +# Examples: +# ./arm_build_process.sh arm64 run_workflow # Full CI build +# ./arm_build_process.sh arm32 build_deps # Build dependencies only +# ./arm_build_process.sh arm64 generate_appimage # Build Scopy and create AppImage +# +# Available functions: +# - install_packages: Install host build dependencies +# - build_deps: Build all Scopy dependencies from source +# - run_workflow: Complete CI workflow (recommended for CI) +# - get_tools: Download/setup build tools only +# - generate_appimage: Build Scopy and package as AppImage +# - dev_setup: Show instructions for local development + set -ex + SRC_DIR=$(git rev-parse --show-toplevel 2>/dev/null ) || \ -SRC_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && cd ../../ && pwd ) -SRC_SCRIPT=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) +SRC_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && cd ../../ && pwd ) # Repository root directory +SRC_SCRIPT=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) # Directory containing this script (ci/arm) +# Build status tracking file BUILD_STATUS_FILE=$SRC_SCRIPT/build-status source $SRC_SCRIPT/arm_build_config.sh $1 @@ -15,19 +35,32 @@ echo -- USING QT: $QT_LOCATION echo -- USING QMAKE: $QMAKE_BIN echo -- SYSROOT: $SYSROOT +# Generic CMake build function +# Used by all dependency build functions to handle CMake-based projects +# +# Process: +# 1. Creates clean build directory +# 2. Runs CMake with cross-compilation options +# 3. Builds with parallel jobs +# 4. Records build info to status file build_with_cmake() { BUILD_FOLDER=$PWD/build rm -rf $BUILD_FOLDER mkdir -p $BUILD_FOLDER cd $BUILD_FOLDER + # Evaluate CMAKE command with any additional options eval "$CMAKE $CURRENT_BUILD_CMAKE_OPTS ../" make $JOBS + # Clear options for next build CURRENT_BUILD_CMAKE_OPTS="" + # Log build information for tracking echo "$(basename -a "$(git config --get remote.origin.url)") - $(git rev-parse --abbrev-ref HEAD) - $(git rev-parse --short HEAD)" \ >> $BUILD_STATUS_FILE } +# Configure autotools build options for cross-compilation +# Sets up environment variables and configure options for autotools-based projects set_config_opts() { CPP="${TOOLCHAIN_BIN}/${TOOLCHAIN_HOST}-cpp" CC="${TOOLCHAIN_BIN}/${TOOLCHAIN_HOST}-gcc -v" @@ -41,16 +74,19 @@ set_config_opts() { CONFIG_OPTS+=("--prefix=${SYSROOT}") CONFIG_OPTS+=("--host=${TOOLCHAIN_HOST}") CONFIG_OPTS+=("--with-sysroot=${SYSROOT}") + CONFIG_OPTS+=("PKG_CONFIG_DIR=") CONFIG_OPTS+=("PKG_CONFIG_LIBDIR=${SYSROOT}/usr/lib/${TOOLCHAIN_HOST}/pkgconfig:${SYSROOT}/usr/share/pkgconfig:${SYSROOT}/usr/lib/${TOOLCHAIN_HOST}/pkgconfig:${SYSROOT}/usr/local/lib/pkgconfig") CONFIG_OPTS+=("PKG_CONFIG_SYSROOT=${SYSROOT}") CONFIG_OPTS+=("PKG_CONFIG_SYSROOT_DIR=${SYSROOT}") if [ "$TOOLCHAIN_HOST" == "aarch64-linux-gnu" ]; then + # 64-bit ARM uses system pkg-config wrapper CONFIG_OPTS+=("PKG_CONFIG_PATH=${SYSROOT}/usr/lib/${TOOLCHAIN_HOST}/pkgconfig") CONFIG_OPTS+=("PKG_CONFIG=/usr/bin/${TOOLCHAIN_HOST}-pkg-config" ) CFLAGS="-march=armv8-a" elif [ "$TOOLCHAIN_HOST" == "arm-linux-gnueabihf" ]; then + # 32-bit ARM uses pkg-config from sysroot CONFIG_OPTS+=("PKG_CONFIG_PATH=${SYSROOT}/usr/bin/arm-linux-gnueabihf-pkg-config") CONFIG_OPTS+=("PKG_CONFIG=${SYSROOT}/usr/bin/${TOOLCHAIN_HOST}-pkg-config" ) CFLAGS="-march=armv7-a" @@ -58,6 +94,7 @@ set_config_opts() { CFLAGS="${CFLAGS} -I${SYSROOT}/include -I${SYSROOT}/include/${TOOLCHAIN_HOST} -I${SYSROOT}/usr/include -I${SYSROOT}/usr/include/${TOOLCHAIN_HOST} -I${TOOLCHAIN}/include- -fPIC" CPPFLAGS="-fexceptions ${CFLAGS}" + # -rpath=XORIGIN: Look for shared libraries relative to binary location LDFLAGS="--sysroot=${SYSROOT} -Wl,-rpath=XORIGIN -L${SYSROOT}/lib -L${SYSROOT}/usr/lib -L${SYSROOT}/usr/lib/${TOOLCHAIN_HOST} -L${SYSROOT}/usr/lib/${TOOLCHAIN_HOST}" CONFIG_OPTS+=("PKG_CONFIG_ALLOW_CROSS=1") @@ -86,7 +123,7 @@ download_cmake() { pushd ${STAGING_AREA} if [ ! -d cmake ];then wget ${CMAKE_DOWNLOAD_LINK} - # unzip and rename + # Extract and rename to 'cmake' directory tar -xf cmake*.tar.gz && rm cmake*.tar.gz && mv cmake* cmake else echo "Cmake already downloaded" @@ -99,7 +136,7 @@ download_crosscompiler(){ pushd ${STAGING_AREA} if [ ! -d cross-pi-gcc ];then wget --progress=dot:giga ${CROSSCOMPILER_DOWNLOAD_LINK} - # unzip and rename + # Extract and rename to 'cross-pi-gcc' directory tar -xf cross-gcc-*.tar.gz && rm cross-gcc-*.tar.gz && mv cross-pi-* cross-pi-gcc else echo "Crosscompiler already downloaded" @@ -111,6 +148,8 @@ clone() { echo "#######CLONE#######" mkdir -p $STAGING_AREA pushd $STAGING_AREA + # Pattern: [ -d 'directory' ] || git clone ... + # Only clones if directory doesn't exist [ -d 'libserialport' ] || git clone --recursive https://github.com/sigrokproject/libserialport -b $LIBSERIALPORT_BRANCH libserialport [ -d 'libiio' ] || git clone --recursive https://github.com/analogdevicesinc/libiio.git -b $LIBIIO_VERSION libiio [ -d 'libad9361' ] || git clone --recursive https://github.com/analogdevicesinc/libad9361-iio.git -b $LIBAD9361_BRANCH libad9361 @@ -132,13 +171,17 @@ clone() { build_libserialport(){ echo "### Building libserialport - branch $LIBSERIALPORT_BRANCH" + # Load cross-compilation settings for autotools set_config_opts pushd $STAGING_AREA/libserialport git clean -xdf ./autogen.sh ./configure "${CONFIG_OPTS[@]}" make $JOBS + # Fix RPATH to use $ORIGIN (relative to library location) + # This makes the library portable in AppImage patchelf --force-rpath --set-rpath \$ORIGIN $STAGING_AREA/libserialport/.libs/libserialport.so + # Install to sysroot sudo make install echo "$(basename -a "$(git config --get remote.origin.url)") - $(git rev-parse --abbrev-ref HEAD) - $(git rev-parse --short HEAD)" \ >> $BUILD_STATUS_FILE @@ -209,6 +252,8 @@ build_gnuradio() { echo "### Building gnuradio - branch $GNURADIO_BRANCH" pushd $STAGING_AREA/gnuradio + # Configure minimal GNU Radio build + # Only enable components needed by Scopy CURRENT_BUILD_CMAKE_OPTS="\ -DENABLE_DEFAULT=OFF \ -DENABLE_GNURADIO_RUNTIME=ON \ @@ -222,6 +267,10 @@ build_gnuradio() { # This is not needed anymore, (don't know why) but it was used as a workaround to execute the python binary, from the sysroot, in the host machine # it was needed because, for some reason, the gnuradio build process needed to execute the binary at build time + + # Special Python handling for 32-bit ARM + # GNU Radio build needs to run Python during build + # This wrapper allows x86 host to execute ARM Python binary # if [ $TOOLCHAIN_HOST == "arm-linux-gnueabihf" ]; then # PYTHON_WRAPPER="$STAGING_AREA/python-wrapper.sh" # echo '#!/bin/bash' > $PYTHON_WRAPPER @@ -266,10 +315,15 @@ build_qwt() { echo "### Building qwt - branch $QWT_BRANCH" pushd $STAGING_AREA/qwt git clean -xdf + # Fix install prefix - remove version suffix from path sed -i 's|/usr/local/qwt-$$QWT_VERSION-ma|/usr/local|g' qwtconfig.pri + # INCLUDEPATH: Where to find headers + # LIBS: Where to find libraries (both generic and arch-specific paths) $QMAKE_BIN INCLUDEPATH=$SYSROOT/include LIBS=-L$SYSROOT/lib LIBS+=-L$SYSROOT/lib/$TOOLCHAIN_HOST qwt.pro make $JOBS + # Fix RPATH for library portability patchelf --force-rpath --set-rpath \$ORIGIN $STAGING_AREA/qwt/lib/libqwt.so + # Install to sysroot setting the INSTALL_ROOT sudo make INSTALL_ROOT=$SYSROOT install echo "$(basename -a "$(git config --get remote.origin.url)") - $(git rev-parse --abbrev-ref HEAD) - $(git rev-parse --short HEAD)" \ @@ -368,6 +422,15 @@ build_scopy() { popd } +# AppDir/ +# ├── AppRun (entry point script) +# ├── scopy.desktop (desktop integration) +# ├── scopy.png (application icon) +# └── usr/ +# ├── bin/ (executables: scopy, iio-emu) +# ├── lib/ (all required libraries) +# └── share/ (resources, icons, desktop files) + create_appdir(){ BUILD_FOLDER=$SRC_DIR/build EMU_BUILD_FOLDER=$STAGING_AREA/iio-emu/build @@ -385,24 +448,28 @@ create_appdir(){ mkdir -p $APP_DIR/usr/share/applications mkdir -p $APP_DIR/usr/share/icons/hicolor/512x512 - cp $APP_RUN $APP_DIR - cp $APP_DESKTOP $APP_DIR - cp $SRC_DIR/gui/res/scopy.png $APP_DIR - cp $SRC_DIR/gui/res/scopy.png $APP_DIR/usr/share/icons/hicolor/512x512 - cp $APP_DESKTOP $APP_DIR/usr/share/applications + # Copy AppImage metadata files + cp $APP_RUN $APP_DIR # Entry point script + cp $APP_DESKTOP $APP_DIR # Desktop file for app menu + cp $SRC_DIR/gui/res/scopy.png $APP_DIR # Icon for file manager + cp $SRC_DIR/gui/res/scopy.png $APP_DIR/usr/share/icons/hicolor/512x512 # Icon for desktop + cp $APP_DESKTOP $APP_DIR/usr/share/applications # Desktop file for system cp $EMU_BUILD_FOLDER/iio-emu $APP_DIR/usr/bin + # Workaround for 32-bit ARM Qt libraries conflict + # Remove system Qt libraries that interfere with bundled ones if [ $TOOLCHAIN_HOST == "arm-linux-gnueabihf" ]; then sudo rm -rfv ${SYSROOT}/usr/lib/arm-linux-gnueabihf/libQt5* fi + # copy-deps.sh recursively finds and copies all library dependencies $COPY_DEPS --lib-dir ${SYSROOT}:${BUILD_FOLDER} --output-dir $APP_DIR/usr/lib $APP_DIR/usr/bin/scopy $COPY_DEPS --lib-dir ${SYSROOT}:${BUILD_FOLDER} --output-dir $APP_DIR/usr/lib $APP_DIR/usr/bin/iio-emu $COPY_DEPS --lib-dir ${SYSROOT}:${BUILD_FOLDER} --output-dir $APP_DIR/usr/lib "$(find $APP_DIR/usr -type f -name "libscopy*.so")" + cp -r $QT_LOCATION/plugins $APP_DIR/usr - # search for the python version linked by cmake and copy inside the appimage the same version FOUND_PYTHON_VERSION=$(grep 'PYTHON_VERSION' $SRC_DIR/build/CMakeCache.txt | awk -F= '{print $2}' | grep -o 'python[0-9]\+\.[0-9]\+') python_path=${SYSROOT}/usr/lib/$FOUND_PYTHON_VERSION cp -r $python_path $APP_DIR/usr/lib @@ -410,17 +477,29 @@ create_appdir(){ cp -r $SYSROOT/share/libsigrokdecode/decoders $APP_DIR/usr/lib cp $SYSROOT/lib/libgenalyzer.so* $APP_DIR/usr/lib - cp $QT_LOCATION/lib/libQt5XcbQpa.so* $APP_DIR/usr/lib - cp $QT_LOCATION/lib/libQt5EglFSDeviceIntegration.so* $APP_DIR/usr/lib - cp $QT_LOCATION/lib/libQt5DBus.so* $APP_DIR/usr/lib - cp $QT_LOCATION/lib/libQt5OpenGL.so* $APP_DIR/usr/lib - cp $SYSROOT/lib/${TOOLCHAIN_HOST}/libGLESv2.so* $APP_DIR/usr/lib - cp $SYSROOT/lib/${TOOLCHAIN_HOST}/libbsd.so* $APP_DIR/usr/lib - cp $SYSROOT/lib/${TOOLCHAIN_HOST}/libXdmcp.so* $APP_DIR/usr/lib - cp $SYSROOT/usr/lib/${TOOLCHAIN_HOST}/libXau.so* $APP_DIR/usr/lib - cp $SYSROOT/usr/lib/${TOOLCHAIN_HOST}/libffi.so* $APP_DIR/usr/lib + cp $QT_LOCATION/lib/libQt5XcbQpa.so* $APP_DIR/usr/lib # X11 platform + cp $QT_LOCATION/lib/libQt5EglFSDeviceIntegration.so* $APP_DIR/usr/lib # EGL/OpenGL ES + cp $QT_LOCATION/lib/libQt5DBus.so* $APP_DIR/usr/lib # D-Bus support + cp $QT_LOCATION/lib/libQt5OpenGL.so* $APP_DIR/usr/lib # OpenGL support + + cp $SYSROOT/lib/${TOOLCHAIN_HOST}/libGLESv2.so* $APP_DIR/usr/lib # OpenGL ES 2.0 + cp $SYSROOT/lib/${TOOLCHAIN_HOST}/libbsd.so* $APP_DIR/usr/lib # BSD compatibility + cp $SYSROOT/lib/${TOOLCHAIN_HOST}/libXdmcp.so* $APP_DIR/usr/lib # X11 display manager + cp $SYSROOT/usr/lib/${TOOLCHAIN_HOST}/libXau.so* $APP_DIR/usr/lib # X11 authentication + cp $SYSROOT/usr/lib/${TOOLCHAIN_HOST}/libffi.so* $APP_DIR/usr/lib # Foreign function interface } +# Create AppImage from AppDir +# Packages the AppDir into a single executable AppImage file +# +# 1. Create squashfs filesystem from AppDir +# 2. Concatenate runtime and squashfs to create AppImage +# 3. Make the result executable +# +# The runtime is a small ELF executable that: +# - Mounts the squashfs filesystem +# - Sets up environment variables +# - Executes the AppRun script create_appimage(){ rm -rf $APP_IMAGE mksquashfs $APP_DIR $APP_SQUASHFS -root-owned -noappend @@ -464,9 +543,10 @@ generate_ci_envs() } -# -# Helper functions -# +# ================ +# Helper Functions +# ================ + build_deps(){ clone build_libserialport @@ -487,23 +567,26 @@ build_deps(){ build_genalyzer } +# Complete CI workflow run_workflow(){ - install_packages - move_tools - move_sysroot - build_iio-emu - build_scopy - create_appdir - create_appimage - move_appimage + install_packages # Install host build tools + move_tools # Setup cmake and cross-compiler + move_sysroot # Setup ARM root filesystem + build_iio-emu # Build IIO emulator + build_scopy # Build Scopy application + create_appdir # Create AppImage directory + create_appimage # Create final AppImage + move_appimage # Rename and move AppImage based on architecture } +# Setup build tools only get_tools(){ install_packages move_tools move_sysroot } +# Build and package Scopy only generate_appimage(){ build_iio-emu build_scopy @@ -512,34 +595,37 @@ generate_appimage(){ } dev_setup(){ - # for the local development of Scopy arm the easyest method is to download the docker image - # after that, a temporary docker volume is created to bridge the local environment and the docker container - # and the compiling is done inside the container unsing the already prepared filesystem + # The easiest method for local ARM development is using Docker + # Docker provides: + # - Pre-built sysroot with all dependencies + # - Cross-compilation toolchain + # - Consistent build environment [ "$ARCHITECTURE" == "armhf" ] && ARCH=armhf || ARCH=arm64 - # for example, you would execute: - - docker pull cristianbindea/scopy2-$ARCH-appimage:latest # to download the image + # Step 1: Pull the Docker image + docker pull cristianbindea/scopy2-$ARCH-appimage:latest - # and to run the image, while creating a docker volume, you would run: + # Step 2: Run container with source code mounted docker run -it \ --mount type=bind,source="$SRC_DIR",target=/home/runner/scopy \ cristianbindea/scopy2-$ARCH-appimage:latest + # Now the repository is available inside the container at /home/runner/scopy - # now this repository folder it shared with the docker container + # Step 3: Build inside container + # For 32-bit ARM: scopy/ci/arm/arm_build_process.sh arm32 run_workflow + # For 64-bit ARM: scopy/ci/arm/arm_build_process.sh arm64 run_workflow - # to compile and package the application use "scopy/ci/arm/arm_build_process.sh arm32 run_workflow" - # or "scopy/ci/arm/arm_build_process.sh arm64 run_workflow", depending on the architecture - - # to continue using the same docker container use "docker start (container id)" and "docker attach (container id)" + # To resume work in the same container: + # docker start + # docker attach - # finally after the development is done use this to clean the system - # "docker container rm -v (container id)" - # "docker image rm cristianbindea/scopy2-$ARCH-appimage:latest" + # Cleanup when done: + # docker container rm -v + # docker image rm cristianbindea/scopy2-$ARCH-appimage:latest - # **to get the container id use "docker container ls -a" + # Get container ID with: docker container ls -a } for arg in "${@:2}"; do diff --git a/ci/arm/build_qt.sh b/ci/arm/build_qt.sh index a466921eb2..293b8703e4 100755 --- a/ci/arm/build_qt.sh +++ b/ci/arm/build_qt.sh @@ -1,5 +1,14 @@ #!/bin/bash +# Qt Cross-Compilation Build Script for ARM +# ========================================= +# Build Qt 5.15 from source for ARM platforms +# Usage: ./build_qt.sh [arm32|arm64] [function_name ...] +# +# This builds a minimal Qt configuration with only the modules +# needed by Scopy, significantly reducing build time and size + + set -ex SRC_SCRIPT=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) source $SRC_SCRIPT/arm_build_config.sh $1 @@ -10,16 +19,18 @@ install_packages(){ pigz libncurses-dev autoconf automake tar figlet libclang-dev } -# Download and extract QT Source (QT 5.15.2 for armhf and QT 5.15.10 for arm64) +# Download and patch Qt source code download_qt(){ mkdir -p ${STAGING_AREA} pushd ${STAGING_AREA} if [ ! -d qt-everywhere-src ];then wget --progress=dot:giga ${QT_DOWNLOAD_LINK} - tar -xf qt-everywhere-*.tar.xz && rm qt-everywhere-*.tar.xz && mv qt-everywhere-* qt-everywhere-src # unzip and rename + # unzip and rename + tar -xf qt-everywhere-*.tar.xz && rm qt-everywhere-*.tar.xz && mv qt-everywhere-* qt-everywhere-src cd qt-everywhere-src - # Patch QT Source + # Apply architecture-specific patches + # These patches are maintained in the ci/arm directory if [ $TOOLCHAIN_HOST == "aarch64-linux-gnu" ]; then patch -p1 < $SRC_SCRIPT/qt_patch_arm64.patch elif [ $TOOLCHAIN_HOST == "arm-linux-gnueabihf" ]; then @@ -36,15 +47,19 @@ download_crosscompiler(){ pushd ${STAGING_AREA} if [ ! -d cross-pi-gcc ];then wget --progress=dot:giga ${CROSSCOMPILER_DOWNLOAD_LINK} - tar -xf cross-gcc-*.tar.gz && rm cross-gcc-*.tar.gz && mv cross-pi-* cross-pi-gcc # unzip and rename + # unzip and rename + tar -xf cross-gcc-*.tar.gz && rm cross-gcc-*.tar.gz && mv cross-pi-* cross-pi-gcc else echo "Crosscompiler already downloaded" fi popd } +# Configures Qt for cross-compilation with minimal modules build(){ mkdir -p "$STAGING_AREA"/build-qt && cd "$STAGING_AREA"/build-qt + + # Configure Qt with cross-compilation settings ../qt-everywhere-src/configure \ -v \ -release \ @@ -57,7 +72,7 @@ build(){ -eglfs \ -reduce-exports \ -opengl desktop \ - -device "$QT_BUILD_DEVICE"\ + -device "$QT_BUILD_DEVICE" \ -device-option CROSS_COMPILE="$CROSS_COMPILER"/bin/"$TOOLCHAIN_HOST"- \ -skip qtandroidextras \ -skip qtcharts \ diff --git a/ci/arm/copy-deps.sh b/ci/arm/copy-deps.sh index 76d562a766..2861db2c1d 100755 --- a/ci/arm/copy-deps.sh +++ b/ci/arm/copy-deps.sh @@ -1,12 +1,36 @@ #!/bin/bash +# Library Dependency Scanner and Copier +# ==================================== +# Recursively finds and copies all shared library dependencies +# Usage: ./copy-deps.sh [options] binary_files... +# +# Options: +# --lib-dir PATH Colon-separated paths to search for libraries +# --output-dir DIR Directory to copy libraries to +# --help Show usage information +# +# Examples: +# # List dependencies only (no copy): +# ./copy-deps.sh --lib-dir /usr/lib:/usr/local/lib /usr/bin/scopy +# +# # Copy all dependencies: +# ./copy-deps.sh --lib-dir /sysroot:/build --output-dir ./libs /usr/bin/scopy +# +# Uses the AppImage excludelist to skip system libraries that should not be bundled + set -e usage() { - echo "test" - + echo "Usage: $0 [--lib-dir PATH] [--output-dir DIR] binary_files..." + echo " --lib-dir PATH Colon-separated library search paths" + echo " --output-dir DIR Copy libraries to this directory" + echo " --help Show this help message" + echo "" + echo "Without --output-dir, only lists dependencies without copying" } + COPY="" while [ "$1" != "" ] @@ -17,15 +41,18 @@ do exit ;; "--lib-dir" | "-l") + # Add to library search path (can be used multiple times) SYSROOT="$2:$SYSROOT" shift ;; "--output-dir" | "-o") + # Enable copy mode and set destination OUTPUT_DIR="$2" COPY="true" shift ;; *) + # Assume it's a file to analyze FILES="$FILES $1" ;; esac @@ -33,11 +60,13 @@ do done LIBS_ARRAY=() -# Separate SYSROOT into an array of SEARCH_PATHS + +# Convert colon-separated paths to array IFS=":" read -r -a SEARCH_PATHS <<< "$SYSROOT" -# If COPY is not true, display all dependencies without excluding any from the blacklist +# library blacklist if [[ $COPY != "true" ]]; then + # In list mode, show all dependencies BLACKLISTED="" else BLACKLISTED=($(wget --quiet https://raw.githubusercontent.com/probonopd/AppImages/master/excludelist -O - | sort | uniq | cut -d '#' -f 1 | grep -v "^#.*" | grep "[^-\s]")) @@ -48,45 +77,56 @@ else BLACKLISTED+=(libglib-2.0.so.0) fi + +# Searches for a library by name in all configured search paths findlib() { LIB=$1 for path in "${SEARCH_PATHS[@]}" do + # Strip version suffix and search for any version + # libfoo.so.1.2.3 -> search for libfoo.so* LIB_PATH=$(find "$path" -name "${LIB%%.so*}.so*" -type f 2>/dev/null | sort | head -1) - [ "$LIB_PATH" != "" ] && break # Exit for at first succesful find + [ "$LIB_PATH" != "" ] && break # Exit on first successful find done [ "$LIB_PATH" = "" ] && LIB_PATH="not found" echo "$LIB_PATH" } +# Copies a library file and recreates all associated symbolic links +# This preserves the standard library naming convention: +# libfoo.so -> libfoo.so.1 -> libfoo.so.1.2.3 copylib() { library=$1 destination=$2 - lib_location=${library%/*} - lib_name=${library##*/} - lib_name_without_version=${lib_name%%.so*}.so + lib_location=${library%/*} # Directory containing library + lib_name=${library##*/} # Library filename + lib_name_without_version=${lib_name%%.so*}.so # Base name echo "${lib_name_without_version}" echo -n copied: cp --verbose "${library}" "${destination}" - #IFS=$'\n' read -r -a symlinks < <(find "$lib_location" -name "${lib_name_without_version}*" -type l 2>/dev/null) || echo "No symlinks to copy" - + # Find all symbolic links to this library symlinks=($(find "$lib_location" -name "${lib_name_without_version}*" -type l 2>/dev/null)) for link in "${symlinks[@]}" do pushd ${destination} > /dev/null + # Remove existing link if present [ -L "${link##*/}" ] && rm "${link##*/}" + # Create new symbolic link echo -n linked: ln --symbolic --verbose "${library##*/}" "${link##*/}" popd > /dev/null done } + +# Use readelf to find NEEDED entries +# and recursively processes each dependency process_lib(){ DEPS=$(readelf -d "$1" 2> /dev/null | \ grep NEEDED | \ @@ -96,13 +136,15 @@ process_lib(){ for library in $DEPS do - # Check if the library is blacklisted or was already processed if [[ ! "${BLACKLISTED[*]}" =~ "${library}" && ! "${LIBS_ARRAY[*]}" =~ "${library}" ]]; then + # Mark as processed LIBS_ARRAY+=("${library}") - LIB_PATH=$(findlib "$library") # try to find the library in sysroot + + LIB_PATH=$(findlib "$library") if [[ $COPY == "true" ]]; then + # Copy mode if [[ $LIB_PATH == "not found" ]]; then echo "Error: ${library} not found" exit 1 @@ -110,8 +152,7 @@ process_lib(){ mkdir -p "$OUTPUT_DIR" copylib "$LIB_PATH" "$OUTPUT_DIR" fi - - # If COPY is not true, display only the library and its location if found + # List mode [ "$COPY" != "true" ] && echo "$library => $LIB_PATH" process_lib "$LIB_PATH" fi diff --git a/ci/arm/create_docker_image.sh b/ci/arm/create_docker_image.sh index c75d7e5a25..6c9bf573b7 100755 --- a/ci/arm/create_docker_image.sh +++ b/ci/arm/create_docker_image.sh @@ -1,12 +1,17 @@ #!/bin/bash +# Docker Image Creation Script for ARM Cross-Compilation +# ===================================================== +# Creates a Docker image with ARM cross-compilation environment +# Usage: ./create_docker_image.sh [arm32|arm64] + + set -ex SRC_DIR=$(git rev-parse --show-toplevel 2>/dev/null ) || \ SRC_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && cd ../../ && pwd ) ARCH=$1 source $SRC_DIR/ci/arm/arm_build_config.sh "$ARCH" -# install docker install_packages(){ sudo apt-get update sudo apt-get -y install apt-transport-https ca-certificates curl software-properties-common @@ -16,6 +21,7 @@ install_packages(){ sudo apt-get -y install containerd.io docker-ce docker-ce-cli docker-buildx-plugin } +# Create ARM sysroot from Kuiper Linux image create_sysroot(){ $SRC_DIR/ci/arm/create_sysroot.sh "$ARCH" \ install_packages \ @@ -25,7 +31,7 @@ create_sysroot(){ configure_sysroot } -# archive the sysroot and move it next to the Dockerfile in order to copy the tar in the docker image +# Package sysroot for Docker image tar_and_move_sysroot(){ pushd $STAGING_AREA sudo tar -czf "${SYSROOT_TAR##*/}" sysroot @@ -33,6 +39,7 @@ tar_and_move_sysroot(){ popd } +# Build Docker image create_image(){ pushd ${SRC_DIR}/ci/arm if [ "${ARCH}" == "arm32" ]; then diff --git a/ci/arm/create_sysroot.sh b/ci/arm/create_sysroot.sh index f69edd2503..3ee19f02a1 100755 --- a/ci/arm/create_sysroot.sh +++ b/ci/arm/create_sysroot.sh @@ -1,5 +1,21 @@ #!/bin/bash +# ARM Sysroot Creation Script +# ========================== +# Creates a sysroot from ADI Kuiper Linux image for cross-compilation +# Usage: ./create_sysroot.sh [arm32|arm64] [function_name ...] +# +# A sysroot is a directory structure that mimics the target system's root filesystem +# It contains headers, libraries, and tools needed for cross-compilation +# +# Functions available: +# - install_packages: Install host tools +# - download_kuiper: Download ADI Kuiper Linux image +# - install_qemu: Install QEMU for chroot operations +# - extract_sysroot: Extract rootfs from disk image +# - configure_sysroot: Install packages inside sysroot +# - fix_relativelinks: Convert absolute symlinks to relative + set -ex SRC_SCRIPT=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) @@ -16,6 +32,7 @@ download_kuiper(){ if [ $TOOLCHAIN_HOST == "aarch64-linux-gnu" ]; then echo "Downloading Linux Kuiper arm64 image" + # Only download if not already present if [ ! -f "image_$IMAGE_NAME.zip" ]; then wget \ --progress=bar:force:noscroll \ @@ -24,6 +41,7 @@ download_kuiper(){ else echo "image_$IMAGE_NAME.zip already downloaded" fi + # Extract if .img doesn't exist [ -f image_$IMAGE_NAME.img ] || unzip "image_$IMAGE_NAME.zip" elif [ $TOOLCHAIN_HOST == "arm-linux-gnueabihf" ]; then echo "Downloading Linux Kuiper arm32 image" @@ -47,43 +65,68 @@ download_kuiper(){ echo "image_$IMAGE_NAME.zip already downloaded" fi [ -f image_$IMAGE_NAME.img ] || unzip image_$IMAGE_NAME.zip + # Rename to consistent format if needed [ -f $IMAGE_NAME.img ] && mv $IMAGE_NAME.img image_$IMAGE_NAME.img fi popd } -# install qemu needed for the sysroot configuration + +# QEMU allows us to run ARM binaries on x86 host +# Required for chroot operations to configure the sysroot install_qemu(){ sudo apt update sudo apt -y install qemu-system qemu-user-static qemu-user } -# mount the Kuiper image and copy the entire rootfs partition +# Extract root filesystem from Kuiper disk image +# Disk images contain multiple partitions (boot, rootfs, etc.) +# We need to extract only the rootfs partition which contains +# the libraries, headers, and tools needed for cross-compilation +# +# Process: +# 1. Mount the rootfs partition using loop device with offset +# 2. Copy entire filesystem to sysroot directory +# 3. Add host's DNS configuration for network access +# 4. Clean up temporary files extract_sysroot(){ sudo rm -rf ${STAGING_AREA}/kuiper sudo mkdir -p ${STAGING_AREA}/kuiper IMAGE_FILE=image_$IMAGE_NAME.img - # with file ${IMAGE_FILE} we can see the start sector (4218880) and the length (19947520) of the second partition contained in the Kuiper image - # using this info we can directly mount that partition + + # Partition layout information obtained with 'file' + # The rootfs is the second partition in the image + # offset = start_sector * 512 (sector size) + # sizelimit = partition_size * 512 if [ $TOOLCHAIN_HOST == "aarch64-linux-gnu" ]; then + # arm64 image partition info: + # Start sector: 4218880, Size: 3571712 sectors sudo mount -v -o loop,offset=$((512*4218880)),sizelimit=$((512*3571712)) ${STAGING_AREA}/${IMAGE_FILE} ${STAGING_AREA}/kuiper elif [ $TOOLCHAIN_HOST == "arm-linux-gnueabihf" ]; then + # arm32 image partition info: + # Start sector: 4218880, Size: 19947520 sectors sudo mount -v -o loop,offset=$((512*4218880)),sizelimit=$((512*19947520)) ${STAGING_AREA}/${IMAGE_FILE} ${STAGING_AREA}/kuiper fi - # rm -rf ${SYSROOT} mkdir -p ${SYSROOT} + # -a: archive mode + # -r: recursive + # -p: preserve permissions sudo cp -arp ${STAGING_AREA}/kuiper/* ${SYSROOT} + + # Copy host's DNS configuration for package downloads in chroot sudo cp /etc/resolv.conf ${SYSROOT}/etc/resolv.conf + sudo umount ${STAGING_AREA}/kuiper sudo rm -rf ${STAGING_AREA}/kuiper rm -rf ${STAGING_AREA:?}/${IMAGE_FILE} rm -rf ${STAGING_AREA}/image*.zip } -# execute chroot inside the sysroot folder and install/remove packages using apt +# Uses chroot to execute commands inside the ARM filesystem +# This allows us to use ARM's package manager to install/remove packages configure_sysroot(){ if [ $TOOLCHAIN_HOST == "aarch64-linux-gnu" ]; then cat $SRC_SCRIPT/inside_chroot_arm64.sh | sudo chroot ${SYSROOT} @@ -92,6 +135,9 @@ configure_sysroot(){ fi } + +# The tarball is created by Docker image build and cached +# Used in CI to skip the sysroot creation process move_and_extract_sysroot(){ if [ -f $HOME/sysroot.tar.gz ]; then mkdir -p $STAGING_AREA @@ -100,6 +146,7 @@ move_and_extract_sysroot(){ fi } +# Fix absolute symbolic links in sysroot fix_relativelinks(){ pushd ${STAGING_AREA} [ -f sysroot-relativelinks.py ] || wget $SYSROOT_RELATIVE_LINKS diff --git a/ci/arm/inside_chroot_arm64.sh b/ci/arm/inside_chroot_arm64.sh index 4c7aaa8f03..cb29e9d2b5 100644 --- a/ci/arm/inside_chroot_arm64.sh +++ b/ci/arm/inside_chroot_arm64.sh @@ -1,23 +1,37 @@ #!/bin/bash +# ARM64 (aarch64) Sysroot Configuration Script +# ============================================ +# Purpose: Configure the ARM64 sysroot for cross-compilation +# This script runs inside chroot with QEMU emulation + +# Configure non-interactive package installation export DEBIAN_FRONTEND=noninteractive + +# Set timezone and locale ln -snf /usr/share/zoneinfo/Europe/Bucharest /etc/localtime && echo "Europe/Bucharest" > /etc/timezone echo "LC_ALL=en_US.UTF-8" | tee -a /etc/environment echo "LANG=en_US.UTF-8" | tee -a /etc/locale.conf locale-gen en_US.UTF-8 +# Remove pre-installed Qt packages apt -y remove *qt* apt -y autoremove +# Remove any leftover files from ADI libraries +# These will be built from source with correct versions rm -rfv $(find / | grep libiio) rm -rfv $(find / | grep m2k) rm -rfv $(find / | grep libad9361) rm -rfv $(find / | grep libad9166) +# Update system packages apt update && apt -y upgrade apt -y dist-upgrade +# Fix any broken packages dpkg --configure -a +# Core build tools and X11/Wayland libraries apt -y install '^libxcb.*-dev' autoconf automake bison build-essential cmake dh-python figlet flex freeglut3-dev g++ gawk gcc \ gdb-multiarch gdbserver git libavahi-client* libavahi-common* libboost1.81-all-dev libdrm-dev libgbm-dev libglib2.0-dev libgl1-mesa-dev libgles2-mesa-dev libglu1-mesa-dev \ libgmp-dev libinput-dev libopenal-dev libsndfile1-dev libspeechd-dev libts-dev libudev-dev libunwind-dev libxcb-icccm4 libxcb-xinerama0 \ @@ -28,15 +42,21 @@ apt -y install '^libxcb.*-dev' autoconf automake bison build-essential cmake dh- libjpeg-dev libpng-dev libxcomposite-dev libdouble-conversion-dev \ libxcb-icccm4 libxcb-image0 libxcb-keysyms1 libxcb-shape0-dev libxcb-xinerama0 libxcb-xkb1 libxkbcommon-x11-0 +# Remove any Qt packages that got pulled in as dependencies apt -y remove *qt* +# Create symbolic links for cross-compilation wget https://raw.githubusercontent.com/abhiTronix/raspberry-pi-cross-compilers/master/utils/SSymlinker sed -i 's/sudo//g' SSymlinker chmod +x SSymlinker + +# Link architecture-specific headers to standard locations ./SSymlinker -s /usr/include/aarch64-linux-gnu/asm -d /usr/include ./SSymlinker -s /usr/include/aarch64-linux-gnu/gnu -d /usr/include ./SSymlinker -s /usr/include/aarch64-linux-gnu/bits -d /usr/include ./SSymlinker -s /usr/include/aarch64-linux-gnu/sys -d /usr/include + +# Link C runtime objects for the linker ./SSymlinker -s /usr/lib/aarch64-linux-gnu/crtn.o -d /usr/lib/crtn.o ./SSymlinker -s /usr/lib/aarch64-linux-gnu/crt1.o -d /usr/lib/crt1.o ./SSymlinker -s /usr/lib/aarch64-linux-gnu/crti.o -d /usr/lib/crti.o diff --git a/ci/arm/inside_chroot_armhf.sh b/ci/arm/inside_chroot_armhf.sh index 5c9c494837..2789e8512f 100644 --- a/ci/arm/inside_chroot_armhf.sh +++ b/ci/arm/inside_chroot_armhf.sh @@ -1,44 +1,74 @@ #!/bin/bash +# ARM32 (armhf) Sysroot Configuration Script +# ========================================== +# Purpose: Configure the ARM sysroot for cross-compilation +# This script runs inside chroot with QEMU emulation + +# Configure non-interactive package installation export DEBIAN_FRONTEND=noninteractive + +# Set timezone and locale ln -snf /usr/share/zoneinfo/Europe/Bucharest /etc/localtime && echo "Europe/Bucharest" > /etc/timezone echo "LC_ALL=en_US.UTF-8" | tee -a /etc/environment echo "LANG=en_US.UTF-8" | tee -a /etc/locale.conf locale-gen en_US.UTF-8 +# Enable source repositories (needed for build-dep) sed -i 's/#deb-src/deb-src/' /etc/apt/sources.list +# Update system packages apt update && apt -y upgrade apt -y dist-upgrade + +# Remove unnecessary packages apt -y purge openjdk* tex-common + +# Remove pre-installed GNU Radio and Qt apt -y remove gnuradio gnuradio-* libgnuradio-* libvolk* apt -y remove *qt* + apt -y autoremove + +# Fix any broken packages dpkg --configure -a +# Remove any leftover files from ADI libraries +# These will be built from source with correct versions find / -name '*libiio*' -exec rm -rfv {} + find / -name '*m2k*' -exec rm -rfv {} + find / -name '*libad9361*' -exec rm -rfv {} + find / -name '*volk*' -exec rm -rfv {} + find / -name '*gnuradio*' -exec rm -rfv {} + +# Install Qt build dependencies apt -y build-dep qtbase5-dev + +# Install development packages needed for building apt -y install '^libxcb.*-dev' autoconf automake bison build-essential cmake figlet flex freeglut3-dev g++ gawk gcc \ gdb-multiarch gdbserver git libavahi-client* libavahi-common* libboost1.74-all-dev libdrm-dev libgbm-dev libglib2.0-dev libgl1-mesa-dev libgles2-mesa-dev \ libglu1-mesa-dev libgmp-dev libinput-dev libopenal-dev libsndfile1-dev libspeechd-dev libts-dev libudev-dev \ libunwind-dev libx11-xcb-dev libxi-dev libxkbcommon-dev libxkbcommon-x11-dev libxrender-dev libxml2-dev libxml2-utils \ mesa-common-dev mesa-utils* perl pkg-config dh-python unzip wget libmatio-dev -apt -y install libusb-1.0-0-dev libaio-dev libpkgconf-dev libpython3-all-dev libfftw3-dev swig swig4.0 libgsl-dev libfuse2 libzmq3-dev libwayland-egl-backend-dev libwayland-dev libwayland-egl1-mesa libssl-dev +apt -y install libusb-1.0-0-dev libaio-dev libpkgconf-dev libpython3-all-dev libfftw3-dev swig swig4.0 libgsl-dev libfuse2 libzmq3-dev libwayland-egl-backend-dev libwayland-dev libwayland-egl1-mesa libssl-dev +# Remove any Qt packages that got pulled in +# We build Qt from source apt -y remove *qt* +# Download symlink creation tool wget https://raw.githubusercontent.com/abhiTronix/raspberry-pi-cross-compilers/master/utils/SSymlinker +# Remove sudo calls (we're already root in chroot) sed -i 's/sudo//g' SSymlinker chmod +x SSymlinker + +# Create symlinks for system headers ./SSymlinker -s /usr/include/arm-linux-gnueabihf/asm -d /usr/include ./SSymlinker -s /usr/include/arm-linux-gnueabihf/gnu -d /usr/include ./SSymlinker -s /usr/include/arm-linux-gnueabihf/bits -d /usr/include ./SSymlinker -s /usr/include/arm-linux-gnueabihf/sys -d /usr/include + +# Create symlinks for C runtime objects ./SSymlinker -s /usr/lib/arm-linux-gnueabihf/crtn.o -d /usr/lib/crtn.o ./SSymlinker -s /usr/lib/arm-linux-gnueabihf/crt1.o -d /usr/lib/crt1.o ./SSymlinker -s /usr/lib/arm-linux-gnueabihf/crti.o -d /usr/lib/crti.o \ No newline at end of file diff --git a/ci/flatpak/README.md b/ci/flatpak/README.md index c8273c1a1e..db20d4deaf 100644 --- a/ci/flatpak/README.md +++ b/ci/flatpak/README.md @@ -1,21 +1,141 @@ -# Scopy Flatpak Recipe -Flatpak build recipe for Scopy +# Flatpak CI Directory -Scopy is a software oscilloscope and signal analysis toolset. [The official repository](https://github.com/analogdevicesinc/scopy) provides releases for Windows, Linux, macOS and Android. -This recipe is used to build the Linux Flatpak installer. +## Install Scopy using the Flatpak -### Install Scopy using the Flatpak release for Linux using these instructions: - - [Install Flatpak using these instructions](http://flatpak.org/getting.html) - - [Download Scopy.flatpak from the Releases Tab](https://github.com/analogdevicesinc/scopy/releases) - - Install the application using ``` flatpak install Scopy.flatpak ``` - - Run the application using ``` flatpak run org.adi.Scopy ``` +1. **Install Flatpak using [these instructions](http://flatpak.org/getting.html)** +2. **Download **Scopy.flatpak** from the [Releases Tab](https://github.com/analogdevicesinc/scopy/releases)** +3. **Install the application using:** + +```bash + flatpak install Scopy.flatpak +``` + +4. **Run the application using:** + +```bash + flatpak run org.adi.Scopy +``` + +## Overview + +This directory contains scripts and configuration for building Scopy as a Flatpak package for Linux. Flatpak provides a sandboxed, distribution-independent packaging format that works across different Linux distributions. + +### Scripts + +#### `flatpak_build_process.sh` - Main build orchestration script for Flatpak builds + +#### `build_flatpak_deps.sh`- Builds Flatpak dependencies + +### Docker Support + +#### `create_docker_image.sh` - Creates Docker image for Flatpak builds + +#### `docker/Dockerfile`- Defines the Docker image for Flatpak builds + +### Build Configuration + +#### `Makefile` - Contains the compile commands for building Flatpak + +#### `org.adi.Scopy.json.c` - Template file for generating the Flatpak manifest + +#### `shared-modules/` - Git submodule containing shared Flatpak modules ## Building the Docker image + To build the Docker image just execute the ***create_docker.sh*** script. +```bash + ./create_docker.sh +``` + The provided Dockerfile will install the KDE Runtime over the Ubuntu 20.04 base image. This Docker image is built in such a way that it contains the dependencies needed for the packaging of the Scopy application. It leverages the Flatpak Builder caching system, where after each step in the build process the result is saved as cache in order to be reused for later builds. -To build the Flatpak package for Scopy inside this Docker image, it needs to be run using ***--privileged***, otherwise there is a lack of access to necessary utilities. +`TIP:` To build the Flatpak package for Scopy inside this Docker image, it needs to be run using ***--privileged***, otherwise there is a lack of access to necessary utilities. ## Generating the Scopy.flatpak artifact -Running ```make``` inside the ***scopy/ci/flatpak*** folder will build the Scopy.flatpak artifact. \ No newline at end of file + +`Prerequisites:` + +- flatpak-builder +- flatpak platform and sdk + +All of these can be installed using: + +```bash + sudo apt install flatpak jq flatpak-builder + sudo flatpak remote-add --if-not-exists flathub https://dl.flathub.org/repo/flathub.flatpakrepo + sudo flatpak install flathub org.kde.Platform//5.15-23.08 -y + sudo flatpak install flathub org.kde.Sdk//5.15-23.08 -y +``` + +Running ```make``` inside the ***scopy/ci/flatpak*** folder will build the Scopy.flatpak artifact. +This command will build the whole dependency tree and create an installer. + +## Build Process + +### Build using the Docker Image + +1. Pull Docker image + + ```bash + docker pull cristianbindea/scopy2-flatpak:latest + ``` + +2. Run container with privileged access and bind the local Scopy repo to a volume inside the container + + ```bash + docker run -it --privileged --mount type=bind,source=path/to/scopy/repository,target=/home/runner/scopy cristianbindea/scopy2-flatpak:latest + ``` + +3. Execute the build script inside container + + ```bash + /home/runner/scopy/flatpak_build_process.sh + ``` + + Script handles: + +- Dependency setup from cache +- Manifest generation +- Flatpak build execution + +### For Local Development + +1. Initialize submodules `git submodule update --init` + +2. Build Flatpak using: `make` + + This will build all the projects listed in the manifest, the last one being Scopy. + +3. Output: *Scopy.flatpak* installer in current directory + +## Manifest Structure + +The manifest file `org.adi.Scopy.json` defines: + +- Permissions required by the application +- Build dependencies +- The order of building the dependencies +- Source locations +- Build commands + +Flatpak documentation for [manifest structure](https://docs.flatpak.org/en/latest/manifests.html). + +## Caching System + +The Docker image leverages Flatpak Builder's caching system: + +- Each build step result is cached +- Subsequent builds reuse cached results +- CI copies pre-built cache from Docker image +- The result is that using the Docker Image, one can build only the Scopy app with the rest of the dependencies already cached + +## GitHub Actions Integration + +- Workflow: `.github/workflows/linuxflatpakbuild.yml` + +## Notes + +- Flatpak provides sandboxed environment for application +- All dependencies are bundled in the package +- Provides consistent runtime across Linux distributions diff --git a/ci/flatpak/create_docker_image.sh b/ci/flatpak/create_docker_image.sh index 649286da81..c48d6f7d78 100755 --- a/ci/flatpak/create_docker_image.sh +++ b/ci/flatpak/create_docker_image.sh @@ -1,5 +1,9 @@ #!/bin/bash -xe +# Create Docker Image for Flatpak Build +# =========================== +# Usage: ./create_docker_image.sh + SRC_DIR=$(git rev-parse --show-toplevel 2>/dev/null ) || \ SRC_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && cd ../../ && pwd ) diff --git a/ci/flatpak/flatpak_build_process.sh b/ci/flatpak/flatpak_build_process.sh index 50bace4f33..350ce99df4 100755 --- a/ci/flatpak/flatpak_build_process.sh +++ b/ci/flatpak/flatpak_build_process.sh @@ -1,38 +1,55 @@ #!/bin/bash -xe +# Flatpak Build Process Script +# =========================== +# Purpose: Build Scopy as a Flatpak package +# Usage: ./flatpak_build_process.sh +# +# This script: +# 1. Generates the Flatpak manifest (JSON) +# 2. Modifies it for CI builds +# 3. Builds the Flatpak package + +# Get repository root directory SCOPY_DIR=$(git rev-parse --show-toplevel 2>/dev/null ) || \ SCOPY_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && cd ../../ && pwd ) +# Flatpak manifest file (generated from template) SCOPY_JSON=$SCOPY_DIR/ci/flatpak/org.adi.Scopy.json pushd $SCOPY_DIR/ci/flatpak + git submodule update --init -# Run the preprocess step to generate org.adi.Scopy.json +# Generate Flatpak manifest from template +# The template (.json.c) contains C preprocessor directives make preprocess -# Disable the preprocess step; The Json file will now be modified and -# we don't want to re-generate it at the build step export EN_PREPROCESS=false if [ "$CI_SCRIPT" == "ON" ]; then SOURCE_DIR=$GITHUB_WORKSPACE - # this is needed in order to be used by flatpak caching system - # the docker image already contains the built dependencies so we just have to move them + # Use pre-built dependencies from Docker image + # .flatpak-builder: Flatpak's build cache + # build: Compiled dependencies cp -r /home/runner/flatpak_tools/.flatpak-builder $SOURCE_DIR/ci/flatpak cp -r /home/runner/flatpak_tools/build $SOURCE_DIR/ci/flatpak else SOURCE_DIR=$SCOPY_DIR fi -# check the number of elements in the json file in order to get the last element, which is Scopy +# The manifest normally fetches Scopy from git +# For CI, we need to use the local source directory + +# Find Scopy module (it's the last one in the modules array) cnt=$( echo $(jq '.modules | length' $SCOPY_JSON) ) cnt=$(($cnt-1)) -# We are building in Github Actions and we use the current directory folder on a CLEAN Docker image +# Replace git source with local directory source cat $SCOPY_JSON | jq --tab '.modules['$cnt'].sources[0].type = "dir"' > tmp.json cp tmp.json $SCOPY_JSON +# Set path to current source directory cat $SCOPY_JSON | jq --tab '.modules['$cnt'].sources[0].path = "'$SOURCE_DIR'"' > tmp.json cp tmp.json $SCOPY_JSON cat $SCOPY_JSON | jq --tab 'del(.modules['$cnt'].sources[0].url)' > tmp.json @@ -41,7 +58,7 @@ cat $SCOPY_JSON | jq --tab 'del(.modules['$cnt'].sources[0].branch)' > tmp.json cp tmp.json $SCOPY_JSON rm tmp.json -# Generate build status info for the about page +# Extract version info from all git modules for the build status file jq '.modules[] | select(type == "object" and .sources[]?.type == "git") | "\(.name): \(.sources[] | select(.type == "git") | .branch // .tag // .commit // "no branch, tag, or commit")"' ./org.adi.Scopy.json >> build-status cp build-status $SOURCE_DIR/build-status @@ -52,6 +69,8 @@ cp build-status $SOURCE_DIR/build-status # cat $SCOPY_JSON | jq --tab '."build-options".env += ('$CI_ENVS')' > tmp.json # cp tmp.json $SCOPY_JSON +# Build the Flatpak +# This uses flatpak-builder with the generated manifest make # Copy the Scopy.flatpak file in $SOURCE_DIR (which is the external location, mount when docker starts) diff --git a/ci/macOS/README.md b/ci/macOS/README.md new file mode 100644 index 0000000000..145899d533 --- /dev/null +++ b/ci/macOS/README.md @@ -0,0 +1,64 @@ +# macOS CI Directory + +## Overview + +This directory contains scripts for building Scopy on macOS. The build process uses Homebrew for dependency management and creates a DMG installer for distribution. + +### Scripts + +#### `macos_config.sh`- Configuration file for macOS builds + +#### `build_azure_macos.sh`- Main build script for Azure Pipelines + +#### `install_macos_deps.sh`- Installs macOS build dependencies + +#### `before_install_lib.sh`- Pre-installation setup for libraries (not used anymore) + +#### `package_darwin.sh`- Creates the macOS DMG installer + +## Build Process + +### Prerequisites + +- **Homebrew**: Package manager for macOS. Can be installed from here, [Homebrew Installation](https://docs.brew.sh/Installation). + +### Build Steps + +1. **Setup the environment and install the dependencies**: + + ```bash + ./install_macos_deps.sh + ``` + + This will install packages using brew, so you will have to make sure that you have brew installed on the machine. The rest of the dependencies that can’t be found on brew will be built from the source files. + +2. **Build Scopy**: + + ```bash + ./build_azure_macos.sh + ``` + +3. **Create Installer**: + + ```bash + ./package_darwin.sh + ``` + + To run the application, the final step is linking the dependencies to the Scopy binary, enabling the operating system to locate them at runtime. + + This process is handled by a script that manages both the linking and packaging. Once the script completes, inside the build folder, it generates a file named Scopy.app, which can be opened either by running “open Scopy.app” in the terminal or by double-clicking it in the file explorer. + +## Output + +- **Scopy.app**: macOS application bundle +- **Scopy.dmg**: Distributable disk image installer + +## CI Integration + +- Built on Azure: See `azure-pipelines.yml` in repository root + +## Notes + +- Only x86_64 architecture is currently supported +- Apple Silicon (M1/M2) support is provided by [Rosetta](https://support.apple.com/en-us/102527) +- All dependencies must be built with same architecture diff --git a/ci/macOS/build_azure_macos.sh b/ci/macOS/build_azure_macos.sh index d7d1d55264..090b6fb2da 100755 --- a/ci/macOS/build_azure_macos.sh +++ b/ci/macOS/build_azure_macos.sh @@ -1,16 +1,33 @@ #!/bin/bash + +# macOS Build Script +# ===================================== +# Build Scopy and IIO-Emulator on macOS +# Usage: ./build_azure_macos.sh +# +# This script is designed for Azure Pipelines CI but can be run locally on macOS systems. +# It assumes dependencies are already installed via install_macos_deps.sh + set -ex REPO_SRC=$(git rev-parse --show-toplevel) +# Load macOS-specific configuration source $REPO_SRC/ci/macOS/macos_config.sh +# Build IIO Emulator +# ================= +# Virtual IIO device for testing without hardware build_iio-emu(){ echo "### Clone and Build IIO-Emulator" pushd $REPO_SRC + # Clone if not present if [ ! -d "$REPO_SRC/iio-emu" ]; then git clone https://github.com/analogdevicesinc/iio-emu $REPO_SRC/iio-emu fi mkdir -p $REPO_SRC/iio-emu/build cd $REPO_SRC/iio-emu/build + + # Configure with CMake + # macOS needs explicit library and include paths cmake \ -DCMAKE_LIBRARY_PATH="$STAGING_AREA_DEPS" \ -DCMAKE_INSTALL_PREFIX="$STAGING_AREA_DEPS" \ @@ -21,6 +38,8 @@ build_iio-emu(){ -DCMAKE_EXE_LINKER_FLAGS="-L${STAGING_AREA_DEPS}/lib" \ -DCMAKE_POLICY_VERSION_MINIMUM=3.5 \ ../ + + # Build with explicit paths for macOS CFLAGS=-I${STAGING_AREA_DEPS}/include LDFLAGS=-L${STAGING_AREA_DEPS}/lib make ${JOBS} popd } diff --git a/ci/macOS/install_macos_deps.sh b/ci/macOS/install_macos_deps.sh index c587134843..8c5cc9353f 100755 --- a/ci/macOS/install_macos_deps.sh +++ b/ci/macOS/install_macos_deps.sh @@ -1,5 +1,16 @@ #!/bin/bash +# macOS Build Script +# ===================================== +# Build and install all the dependencies needed for Scopy on macOS +# Usage: ./install_macos_deps.sh + +# This script is designed for Azure Pipelines CI but can be run locally on macOS systems. +# It installs all dependencies into a staging area defined by $STAGING_AREA_DEPS, +# which is used by the build script to find the dependencies when building Scopy and IIO-Emulator + +# For Azure Pipelines it also implements caching of Homebrew packages and built dependencies. + set -ex REPO_SRC=$(git rev-parse --show-toplevel) source $REPO_SRC/ci/macOS/macos_config.sh @@ -108,8 +119,6 @@ setup_dependencies_cache() { OS_VERSION=${1:-$(sw_vers -productVersion)} echo "MacOS version $OS_VERSION" -source ${REPO_SRC}/ci/macOS/before_install_lib.sh - install_packages() { # Workaround: Homebrew fails to upgrade Python's 2to3 due to conflicting symlinks https://github.com/actions/runner-images/issues/6817 @@ -526,6 +535,7 @@ setup_git_cache setup_dependencies_cache check_cache_sizes +# Install dependencies, clone repositories, and build all dependencies install_packages export_paths clone diff --git a/ci/macOS/package_darwin.sh b/ci/macOS/package_darwin.sh index 9a9946efec..13017d2ffa 100755 --- a/ci/macOS/package_darwin.sh +++ b/ci/macOS/package_darwin.sh @@ -1,4 +1,11 @@ #!/bin/bash + +# macOS Build Script +# ===================================== +# After the build is complete, this script packages the Scopy.app bundle by fixing library paths and bundling dependencies. +# It also creates a DMG installer and a ZIP archive for distribution. +# Usage: ./package_darwin.sh + set -ex REPO_SRC=$(git rev-parse --show-toplevel) source $REPO_SRC/ci/macOS/macos_config.sh diff --git a/ci/old/README.md b/ci/old/README.md new file mode 100644 index 0000000000..af2faf666d --- /dev/null +++ b/ci/old/README.md @@ -0,0 +1,3 @@ +# Old CI Directory + +This directory contains deprecated CI scripts from previous build systems. These scripts are preserved for reference but are no longer actively used. They represent the Travis CI era of Scopy's build infrastructure before migration to GitHub Actions and AppVeyor. \ No newline at end of file diff --git a/ci/macOS/before_install_lib.sh b/ci/old/before_install_lib.sh similarity index 100% rename from ci/macOS/before_install_lib.sh rename to ci/old/before_install_lib.sh diff --git a/ci/packaging/debian/README.md b/ci/packaging/debian/README.md new file mode 100644 index 0000000000..e67db1a403 --- /dev/null +++ b/ci/packaging/debian/README.md @@ -0,0 +1,9 @@ +# Packaging Directory + +## Folder overview + +This directory contains scripts and configuration for packaging Scopy as a debian package. + +Currently there is support only for ARM 32/64 architecture and in order to solve problem of locally built dependencies, the deb package actually installs the Scopy.AppImage along with the desktop configuration files. + +The script `scopy-deb-build.sh` is used in the *.github/workflows/appimage-armhf.yml* and *.github/workflows/appimage-arm64.yml* workflows. diff --git a/ci/ubuntu/README.md b/ci/ubuntu/README.md new file mode 100644 index 0000000000..9c2474434f --- /dev/null +++ b/ci/ubuntu/README.md @@ -0,0 +1,47 @@ +# Ubuntu CI Directory + +## Overview + +This directory contains scripts for building Scopy on Ubuntu Linux. These builds are primarily for development purposes. Using the scripts from this folder the development environment for Linux can be easily set-up. + +### Scripts + +#### `ubuntu_build_process.sh`- Main build orchestration script + +#### `create_docker_image.sh` - Creates Docker image for Ubuntu builds + +#### `docker_ubuntu/Dockerfile`- Defines the Ubuntu build environment + +## Development Setup + +The build script steps: + +1. Updates package repositories +2. Installs build dependencies, including the Qt framework +3. Builds and installs the dependencies in correct order +4. Builds Scopy + +To run the environment setup process run: + +```bash + ./ubuntu_build_process.sh configure_system +``` + +This will install and build all required dependencies, but it will not build Scopy itself. +By default, the script requires administrator permissions to install the dependencies. However, if multiple versions of the same library are needed, the required libraries can instead be installed locally. To enable this local build, edit the *ubuntu_build_process.sh* script and change the `USE_STAGING` option from `OFF` to `ON`. + +To built the Scopy application: + +```bash + ./ubuntu_build_process.sh build_scopy +``` + +## CI Integration + +- **GitHub Actions**: `.github\workflows\ubuntubuild.yml` + +## Notes + +- This is primarily for development and testing, not end-user distribution +- Requires Ubuntu-specific package versions +- May need adjustments for different Linux distributions diff --git a/ci/ubuntu/create_docker_image.sh b/ci/ubuntu/create_docker_image.sh index ad665e83d9..dd1dbe26fa 100755 --- a/ci/ubuntu/create_docker_image.sh +++ b/ci/ubuntu/create_docker_image.sh @@ -1,5 +1,9 @@ #!/bin/bash -ex +# Create Docker Image for Ubuntu Build +# =========================== +# Usage: ./create_docker_image.sh + SRC_SCRIPT=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) ubuntu20(){ diff --git a/ci/ubuntu/ubuntu_build_process.sh b/ci/ubuntu/ubuntu_build_process.sh index 6cb9606d62..95d1bba04d 100755 --- a/ci/ubuntu/ubuntu_build_process.sh +++ b/ci/ubuntu/ubuntu_build_process.sh @@ -1,5 +1,9 @@ #!/bin/bash -xe +# Build and Install Dependencies for Ubuntu +# =========================== +# Usage: ./ubuntu_build_process.sh configure_system + ## Set STAGING USE_STAGING=OFF ## @@ -137,6 +141,14 @@ clone() { popd } +# Generic CMake build function +# Used by all dependency build functions to handle CMake-based projects +# +# Process: +# 1. Creates clean build directory +# 2. Runs CMake with cross-compilation options +# 3. Builds with parallel jobs +# 4. Records build info to status file build_with_cmake() { download_cmake INSTALL=$1 @@ -168,8 +180,8 @@ install_packages() { libxkbcommon-x11-0 libqt5gui5 libncurses-dev libtool libaio-dev libzmq3-dev libxml2-dev } +# Installing Qt using the aqt tool https://github.com/miurahr/aqtinstall install_qt() { - # installing Qt using the aqt tool https://github.com/miurahr/aqtinstall [ "$PYTHON_VERSION" == "python3.12" ] && sudo pip3 install --no-cache-dir --break-system-packages aqtinstall [ "$PYTHON_VERSION" == "python3.11" ] && sudo pip3 install --no-cache-dir aqtinstall [ "$PYTHON_VERSION" == "python3.9" ] && sudo pip3 install --no-cache-dir aqtinstall diff --git a/ci/windows/README.md b/ci/windows/README.md index df18217aaa..8060823874 100644 --- a/ci/windows/README.md +++ b/ci/windows/README.md @@ -1,69 +1,185 @@ -# Scopy MinGW Recipe +# Windows CI Directory -Build Scopy for Windows using MinGW64 +## Folder overview -Scopy is a software oscilloscope and signal analysis toolset. [The official repository](https://github.com/analogdevicesinc/scopy) provides releases for Windows, Linux, macOS and Android. +This directory contains scripts and resources for building Scopy on Windows using the MinGW toolchain. +The build process uses MSYS2/MinGW environment to build and install the dependencies and creates Windows installers using Inno Setup. +More detailed information can be found here: [Windows Build Instructions](https://analogdevicesinc.github.io/scopy/user_guide/build_instructions/windowsBuild.html#windows-build-instructions). -## Building the Docker image -To build the Docker image just execute the command +### Scripts -``` docker build --tag --isolation=hyperv --memory=16GB --file docker/Dockerfile .``` +#### `mingw_toolchain.sh` - Sets up MinGW toolchain environment variables -The Dockerfile is available in the docker folder. +#### `windows_build_process.sh` - Main Windows build orchestration script -[MSYS2](https://www.msys2.org/) will be installed inside the image along with all dependencies that are required in order to build and package the Scopy app. +#### `build_and_create_installer.sh` - Builds Scopy and creates Windows installer +### Supporting Files -## Setting up the environment +#### `scopy-64.iss.cmakein` - Inno Setup installer template (will be processed by CMake to generate final .iss file) -### Build prerequisites -- An IDE: - - [QTCreator](https://doc.qt.io/qtcreator/) - - [Visual Studio Code](https://code.visualstudio.com/download) +#### `mingw_dll_deps` - Lists all DLLs needed for the installer - - For development the [MSYS2](https://www.msys2.org/) shell is needed. +#### `sigrokdecode-windows-fix.patch` - Windows-specific patch for libsigrokdecode -### Setup dependencies +### Docker Support -1. Launch the mingw64 by executing the binary file from the location where the MSYS2 was installed +#### `docker/Dockerfile` - Creates Windows build environment -2. Execute the following script in the mingw64 terminal +### Drivers Directory - ```bash - windows_build_process.sh - ``` +#### `drivers/` - Contains Windows driver files for hardware support -- Optionally install GDB for build debugging - ```bash - pacman --noconfirm -S mingw-w64-x86_64-gdb - ``` +## Build Process -### Building inside MinGW64 shell +### Building Scopy using the Docker Image -1. Enter the root directory of the repository -2. Create a build directory ```mkdir build``` -3. Enter the build directory ```cd build``` -4. Execute ```cmake ../``` -5. Execute ``` make``` +`Prerequisites:` [Docker](https://docs.docker.com/desktop/setup/install/windows-install/) -Inside the build folder a binary file named Scopy.exe will be generated. This is the starting file of the application. +1. **Pull the Docker image** -### Building in Visual Studio Code + ```bash + docker pull cristianbindea/scopy2-mingw64:latest + ``` -1. In VS Code, install [**C/C++ Extension Pack**](vscode:extension/ms-vscode.cpptools-extension-pack) +2. **Start the Docker Image** + ```bash + docker run -it cristianbindea/scopy2-mingw64:latest + ``` + +3. **Enter the MSYS env** + + ```bash + C:\msys64\usr\bin\bash.exe + ``` + +4. **Clone and build the application** + + ```bash + git clone https://github.com/analogdevicesinc/scopy + cd scopy + mkdir build && cd build + cmake ../ + make -j$(nproc) + ``` + +This will compile the entire project. The process may take 15 minutes or more, depending on your CPU. More information about how to create a shared folder between the Docker container and the local machine can be found here [Building Scopy using the Docker Image](https://analogdevicesinc.github.io/scopy/user_guide/build_instructions/windowsBuild.html#building-scopy-using-the-docker-image). + +### Building the Docker Image + +`Prerequisites:` [Docker](https://docs.docker.com/desktop/setup/install/windows-install/) + +To build the Docker image just execute the command: + + ```bash + docker build --tag scopy-windows --isolation=hyperv --memory=16GB --file docker/Dockerfile . + ``` + +This will create the Docker Image and will install on it every dependency needed to build Scopy and to create an installer. + +### Building locally from sources + +`Prerequisites:` + +- [MSYS2](https://www.msys2.org/) with MinGW64 +- [Inno Setup](https://jrsoftware.org/isdl.php) (only if you want to create and installer) +- ~20GB free disk space + +In order to emulate a UNIX-like environment for building Scopy on Windows, we use MSYS2. + +To install MSYS2, follow these instructions: MSYS2 Installation. We suggest installing the MSYS2 in root of the C:\ partition. In this tutorial the MSYS2 install path is C:\msys64, if you choose a different install directory, make sure to also update the paths. + +1. **Launch MinGW64 shell** from MSYS2 installation + +2. **Install Git and clone Scopy** + + ```bash + pacman -S git + git clone https://github.com/analogdevicesinc/scopy + ``` + +3. **Setup dependencies** + + ```bash + ./windows_build_process.sh + ``` + +This will configure the system for building Scopy. + +4. **Build Scopy** + +You can manually build the application using: + + ```bash + mkdir build && cd build + cmake ../ + make -j$(nproc) + ``` + +Or you can run the build script. This will do a clean build each time is called. + + ```bash + ./build_and_create_installer.sh build_scopy + ``` + +Inside the build folder two executable binary files named Scopy.exe and Scopy-console.exe will be generated. This is the starting file of the application. More information about how to add the app to the PATH and open it from the file explorer, not just from the MINGW64 command line, can be found here [Running Scopy](https://analogdevicesinc.github.io/scopy/user_guide/build_instructions/windowsBuild.html#running-scopy). + +### Create installer + +In order to create an installer make sure you have Inno Setup installed at the default location. Or you can change the PATH variable from *build_and_create_installer.sh / create_installer()*. + + ```bash + ./build_and_create_installer.sh build_scopy build_iio-emu deploy_app extract_debug_symbols create_installer + ``` + +## Development Setup + +In order to have a complete Development setup you should follow the first 3 steps from [Build from source](###Building-locally-from-sources). + +A shortcut in the setup can be taken by first building the app from the command line using the step 4 from [Build from source](###Building-locally-from-sources). This will configure the build and will compile the application for the first time, later you can just import this configuration intro VS Code or Qt Creator. + +### Visual Studio Code + +1. Install C/C++ Extension Pack 2. Open Scopy folder in VS Code +3. Configure CMake project (CMake tool → Configure All Projects) +4. Build project (CMake tool → Build) + +### Qt Creator + +1. Open project CMakeLists.txt +2. Configure with MinGW64 kit +3. Build and run + +Here you can find a more detailed tutorial on [Qt Setup](https://analogdevicesinc.github.io/scopy/user_guide/build_instructions/windowsBuild.html#windows-qt-setup-instructions). + +### Debugging + +Install GDB for debugging: + +```bash + pacman --noconfirm -S mingw-w64-x86_64-gdb +``` + +## Output - > When opening the Scopy folder for the first time, a popup may appear to ask you to trust the authors of the files in this folder. Simply click on **`Yes, I trust the authors`** +- **Scopy.exe**: Main executable in build directory +- **Scopy-console.exe**: Main executable in build directory, but with console logs +- **Scopy-setup.exe**: Inno Setup installer +- All required DLLs and runtime files -3. In VS Code, go to the toolbar on your left and locate the CMake tool. On the **`PROJECT OUTLINE`** dropdown, click on the icon for *Configure All Projects*. This will instruct CMake to generate the files necessary for building the source code. +## CI Integration -4. Under the **`PROJECT STATUS`** dropdown in the CMake tool, click on the icon for *Build* to build the project. +- **GitHub Actions**: `.github/workflows/mingwbuild.yml` -## Generating the Scopy installer -The installer is created using [Inno Setup](https://jrsoftware.org/isinfo.php). This tool is already installed in the Docker Image, but to install Inno Setup locally use this [installer](https://files.jrsoftware.org/is/6/innosetup-6.2.2.exe). +## Troubleshooting -First follow the **Setup dependencies** part. +Common issues: -In order to build Scopy and create an installer locally use the **build_and_create_installer.sh** bash script executed from the mingw shell. +1. **"command not found"**: Ensure running in MinGW64 shell, not CMD +2. **Missing DLLs**: Check `mingw_dll_deps` list +3. **CMake errors**: Verify all dependencies built successfully +4. **Installer fails**: Ensure Inno Setup is installed +5. **Out of memory**: Docker builds need 16GB RAM diff --git a/ci/windows/build_and_create_installer.sh b/ci/windows/build_and_create_installer.sh index 06381d9a41..03e38ece5a 100755 --- a/ci/windows/build_and_create_installer.sh +++ b/ci/windows/build_and_create_installer.sh @@ -1,4 +1,17 @@ #!/bin/bash + +# Windows Installer Creation Script +# ================================ +# Build Scopy and create Windows installer package +# Usage: ./build_and_create_installer.sh [function_name ...] +# +# This script: +# 1. Builds Scopy and IIO-Emulator +# 2. Collects all dependencies (DLLs, Qt plugins, etc.) +# 3. Bundles device drivers +# 4. Creates Inno Setup installer +# 5. Generates debug symbols package + if [ "$CI_SCRIPT" == "ON" ]; then set -ex @@ -13,25 +26,23 @@ fi BUILD_TARGET=x86_64 ARCH_BIT=64 -## Set STAGING +## Staging configuration USE_STAGING=OFF ## SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) source $SCRIPT_DIR/mingw_toolchain.sh $USE_STAGING -INSTALL_FOLDER=$STAGING_AREA/scopy-install -BUILD_FOLDER=$WORKDIR/build_$ARCH -ARTIFACT_FOLDER=$SRC_FOLDER/artifacts -export DEST_FOLDER=$ARTIFACT_FOLDER/scopy-$ARCH # the export is needed for the packaging step -DEBUG_FOLDER=$ARTIFACT_FOLDER/debug-$ARCH -PYTHON_FILES=$STAGING_DIR/lib/python3.* -EMU_BUILD_FOLDER=$WORKDIR/iio-emu/build +INSTALL_FOLDER=$STAGING_AREA/scopy-install # Where 'make install' puts files +BUILD_FOLDER=$WORKDIR/build_$ARCH # CMake build directory +ARTIFACT_FOLDER=$SRC_FOLDER/artifacts # Final output location +export DEST_FOLDER=$ARTIFACT_FOLDER/scopy-$ARCH # Bundle directory (exported for Inno Setup) +DEBUG_FOLDER=$ARTIFACT_FOLDER/debug-$ARCH # Unstripped binaries for debugging +PYTHON_FILES=$STAGING_DIR/lib/python3.* # Python runtime +EMU_BUILD_FOLDER=$WORKDIR/iio-emu/build # IIO emulator build directory download_tools() { mkdir -p $STAGING_AREA - - # check if wget2 is installed pacman -Qs mingw-w64-x86_64-wget2 > /dev/null || pacman -S --noconfirm mingw-w64-x86_64-wget2 pushd $STAGING_AREA @@ -57,12 +68,16 @@ download_tools() { popd } + +# Configures and builds the main Scopy executable build_scopy(){ echo "### Building Scopy" download_cmake + # Copy build status from dependencies build [ -f $HOME/build-status ] && cp $HOME/build-status $SRC_FOLDER/build-status mkdir -p $BUILD_FOLDER cd $BUILD_FOLDER + $CMAKE $RC_COMPILER_OPT -DPYTHON_EXECUTABLE=$STAGING_DIR/bin/python3.exe \ -DENABLE_TESTING=OFF -DCMAKE_INSTALL_PREFIX=$INSTALL_FOLDER \ $SRC_FOLDER @@ -73,6 +88,7 @@ build_scopy(){ build_iio-emu(){ echo "### Building IIO-EMU" download_cmake + # Clone if not present if [ ! -d "$WORKDIR/iio-emu" ]; then git clone https://github.com/analogdevicesinc/iio-emu $WORKDIR/iio-emu fi @@ -87,6 +103,7 @@ build_iio-emu(){ popd } +# Bundle device drivers with installer bundle_drivers(){ echo "### Bundling drivers" cp -R $SRC_FOLDER/ci/windows/drivers $DEST_FOLDER @@ -99,6 +116,8 @@ bundle_drivers(){ fi } + +# Assembles all files needed for the Windows distribution deploy_app(){ echo "### Deploying application and dependencies" @@ -109,12 +128,14 @@ deploy_app(){ rm -rf $DEST_FOLDER mkdir -p $DEST_FOLDER - cp -v $INSTALL_FOLDER/bin/Scopy.exe $DEST_FOLDER/ - cp -v $INSTALL_FOLDER/bin/Scopy-console.exe $DEST_FOLDER/ - cp -v $INSTALL_FOLDER/bin/qt.conf $DEST_FOLDER/ - cp -v $EMU_BUILD_FOLDER/iio-emu.exe $DEST_FOLDER/ - cp -v $EMU_BUILD_FOLDER/tools/iio-emu_gen_xml.exe $DEST_FOLDER/ + cp -v $INSTALL_FOLDER/bin/Scopy.exe $DEST_FOLDER/ # GUI version + cp -v $INSTALL_FOLDER/bin/Scopy-console.exe $DEST_FOLDER/ # GUI with console output (for debugging) + cp -v $INSTALL_FOLDER/bin/qt.conf $DEST_FOLDER/ # Qt configuration + cp -v $EMU_BUILD_FOLDER/iio-emu.exe $DEST_FOLDER/ # IIO emulator + cp -v $EMU_BUILD_FOLDER/tools/iio-emu_gen_xml.exe $DEST_FOLDER/ # XML generator + + # This Qt tool automatically finds and copies required Qt DLLs and plugins $STAGING_DIR/bin/windeployqt.exe \ --dir $DEST_FOLDER \ --no-translations \ @@ -127,20 +148,21 @@ deploy_app(){ cp -vr $INSTALL_FOLDER/lib/libscopy*.dll $DEST_FOLDER cp -vr $INSTALL_FOLDER/lib/scopy/* $DEST_FOLDER - cp -vr $INSTALL_FOLDER/resources $DEST_FOLDER + cp -vr $INSTALL_FOLDER/resources $DEST_FOLDER # Resources (filters, etc.) cp -vr $STAGING_DIR/share/libsigrokdecode/decoders $DEST_FOLDER/ rm -vfr $(find $DEST_FOLDER -name "*.dll.a" -type f) + # Copy additional Qt plugins not handled by windeployqt cp -vr $STAGING_DIR/share/qt5/plugins/renderers $DEST_FOLDER/ cp -vr $STAGING_DIR/share/qt5/plugins/sceneparsers $DEST_FOLDER/ + # List is maintained in mingw_dll_deps file pushd $STAGING_DIR/bin source $SRC_FOLDER/ci/windows/mingw_dll_deps cp -vn "${DLL_DEPS[@]}" $DEST_FOLDER/ cp -nv iio_*.exe $DEST_FOLDER/ popd - # Copy genalyzer DLL if [ "$USE_STAGING" == "ON" ]; then cp -v $STAGING_DIR/bin/libgenalyzer.dll $DEST_FOLDER/ || echo "Warning: genalyzer DLL not found in staging" else @@ -154,20 +176,29 @@ deploy_app(){ bundle_drivers } + +# This reduces installer size while preserving debugging capability extract_debug_symbols(){ echo "### Duplicating unstripped bundle" rm -rf $DEBUG_FOLDER mkdir -p $DEBUG_FOLDER cp -r $DEST_FOLDER/* $DEBUG_FOLDER/ + echo "### Stripping bundle for installer" + # --strip-debug: Remove debugging symbols + # --strip-unneeded: Remove all symbols not needed for relocation /$MINGW_VERSION/bin/strip.exe --verbose --strip-debug --strip-unneeded $(find $DEST_FOLDER -name "*.exe" -type f) /$MINGW_VERSION/bin/strip.exe --verbose --strip-debug --strip-unneeded $(find $DEST_FOLDER -name "*.dll" -type f) } +# Create Windows installer using Inno Setup create_installer() { echo "### Creating installer" pushd $ARTIFACT_FOLDER + # Add Inno Setup to PATH PATH="/c/innosetup:/c/Program Files (x86)/Inno Setup 6:$PATH" + # Compile installer script + # //p flag shows progress iscc //p $BUILD_FOLDER/windows/scopy-$ARCH_BIT.iss if [ "$CI_SCRIPT" == "ON" ]; then @@ -188,7 +219,7 @@ create_installer() { popd } -# move the staging folder that contains the tools needed for the build to the known location +# In Docker Image, tools are pre-downloaded in the Docker image move_tools(){ [ -d /home/docker/staging ] && mv /home/docker/staging $STAGING_AREA || echo "Staging folder not found or already moved" if [ ! -d $STAGING_AREA ]; then @@ -197,14 +228,15 @@ move_tools(){ fi } - run_workflow(){ + # Setup tools (move or download) [ "$CI_SCRIPT" == "ON" ] && move_tools || download_tools - build_scopy - build_iio-emu - deploy_app - extract_debug_symbols - create_installer + # Build steps + build_scopy # Build main application + build_iio-emu # Build emulator + deploy_app # Collect all files + extract_debug_symbols # Strip binaries + create_installer # Create installer } for arg in $@; do diff --git a/ci/windows/mingw_toolchain.sh b/ci/windows/mingw_toolchain.sh index bb432371c2..e8eb31cc48 100644 --- a/ci/windows/mingw_toolchain.sh +++ b/ci/windows/mingw_toolchain.sh @@ -1,11 +1,18 @@ #!/usr/bin/bash.exe +# MinGW Toolchain Configuration Script +# =================================== +# Configure build environment for Windows cross-compilation +# Usage: source mingw_toolchain.sh [USE_STAGING] + set -ex -# get the full directory path of the script +# Get directory containing this script export WORKFOLDER=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) +# File to track build status and versions BUILD_STATUS_FILE=$WORKFOLDER/build-status +# Dependency versions/branches LIBSERIALPORT_BRANCH=master LIBIIO_VERSION=v0.26 LIBAD9361_BRANCH=main @@ -24,10 +31,13 @@ ECM_BRANCH=kf5 KARCHIVE_BRANCH=kf5 GENALYZER_BRANCH=main + STAGING_AREA=$WORKFOLDER/staging MINGW_VERSION=mingw64 ARCH=x86_64 +# USE_STAGING=ON: Install to isolated directory +# USE_STAGING=OFF: Install to system MinGW directory USE_STAGING=$1 if [ ! -z "$USE_STAGING" ] && [ "$USE_STAGING" == "ON" ] then @@ -44,22 +54,28 @@ if [ ! -z "$USE_STAGING" ] && [ "$USE_STAGING" == "ON" ] fi RC_COMPILER_OPT="-DCMAKE_RC_COMPILER=/mingw64/bin/windres.exe" + +# Build PATH with all required tools PATH="/bin:$STAGING_DIR/bin:$WORKFOLDER/cv2pdb:/c/Program Files (x86)/Inno Setup 6:/c/innosetup/:/bin:/usr/bin:${STAGING_DIR}/bin:/c/Program\ Files/Git/cmd:/c/Windows/System32:/c/Program\ Files/Appveyor/BuildAgent:$PATH" + QMAKE=${STAGING_DIR}/bin/qmake PKG_CONFIG_PATH=$STAGING_DIR/lib/pkgconfig CC=${STAGING_DIR}/bin/${ARCH}-w64-mingw32-gcc.exe CXX=${STAGING_DIR}/bin/${ARCH}-w64-mingw32-g++.exe + JOBS="-j22" MAKE_BIN=/usr/bin/make.exe MAKE_CMD="$MAKE_BIN $JOBS" + export CMAKE_GENERATOR="Unix Makefiles" +# These options are passed to every CMake invocation export CMAKE_OPTS=( \ -DCMAKE_BUILD_TYPE=RelWithDebInfo \ -DCMAKE_VERBOSE_MAKEFILE=ON \ -DCMAKE_C_COMPILER:FILEPATH=${CC} \ -DCMAKE_CXX_COMPILER:FILEPATH=${CXX} \ - -DCMAKE_MAKE_PROGRAM:FILEPATH=${MAKE_BIN}\ + -DCMAKE_MAKE_PROGRAM:FILEPATH=${MAKE_BIN} \ -DPKG_CONFIG_EXECUTABLE=${STAGING_DIR}/bin/pkg-config.exe \ -DCMAKE_MODULE_PATH=$STAGING_DIR \ -DCMAKE_PREFIX_PATH=$STAGING_DIR\;$STAGING_DIR/lib/cmake \ @@ -68,13 +84,17 @@ export CMAKE_OPTS=( \ -DCMAKE_POLICY_VERSION_MINIMUM=3.5 \ ) +# Complete CMake command with all options export CMAKE="${STAGING_AREA}/cmake/bin/cmake ${CMAKE_OPTS[@]} " + +# Used by projects that use autotools instead of CMake AUTOCONF_OPTS="--prefix=$STAGING_DIR \ --host=${ARCH}-w64-mingw32 \ --enable-shared \ --disable-static" +# Debug output - show configuration echo -- BUILD_STATUS_FILE:$BUILD_STATUS_FILE echo -- MAKE_BIN:$MAKE_BIN echo -- STAGING_DIR:$STAGING_DIR @@ -86,14 +106,13 @@ echo -- USING CMAKE COMMAND echo $CMAKE echo -# an older version of cmake was needed to fix some gnuradio and boost build errors download_cmake() { echo "#######DOWNLOAD CMAKE#######" mkdir -p ${STAGING_AREA} pushd ${STAGING_AREA} if [ ! -d cmake ];then wget https://github.com/Kitware/CMake/releases/download/v4.1.1/cmake-4.1.1-windows-x86_64.zip - # unzip and rename + # Extract and rename to 'cmake' directory unzip cmake*.zip && rm cmake*.zip && mv cmake* cmake else echo "Cmake already downloaded" diff --git a/ci/windows/windows_build_process.sh b/ci/windows/windows_build_process.sh index e5ea6f92b1..7b3d6c95ec 100755 --- a/ci/windows/windows_build_process.sh +++ b/ci/windows/windows_build_process.sh @@ -1,11 +1,18 @@ #!/usr/bin/bash.exe + +# Windows Build Process Script for Scopy +# ===================================== +# Purpose: Build Scopy for Windows using MSYS2/MinGW-w64 +# Usage: ./windows_build_process.sh + set -xe -# get the full directory path of the script +# Get the directory containing this script SCRIPT_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) -## Set STAGING +# Staging configuration +# OFF: Install to system directories +# ON: Install to isolated staging area USE_STAGING=OFF -## source $SCRIPT_DIR/mingw_toolchain.sh $USE_STAGING @@ -17,6 +24,7 @@ install_packages() { unzip\ zip\ " + TOOLS_PKGS="\ mingw-w64-${ARCH}-wget2\ mingw-w64-${ARCH}-cmake\ @@ -33,6 +41,9 @@ install_packages() { mingw-w64-${ARCH}-boost\ mingw-w64-${ARCH}-ccache " + + # Pre-built libraries from MSYS2 + # Using these saves significant build time PACMAN_SYNC_DEPS="\ mingw-w64-${ARCH}-fftw\ mingw-w64-${ARCH}-orc\ @@ -54,6 +65,7 @@ install_packages() { if [ "$USE_STAGING" == "ON" ]; then mkdir -p $STAGING_DIR/var/lib/pacman/local mkdir -p $STAGING_DIR/var/lib/pacman/sync + # Update core packages first $PACMAN -Syuu bash filesystem mintty pacman fi @@ -88,6 +100,7 @@ clone() { popd } +# Records information about the build environment create_build_status_file() { touch $BUILD_STATUS_FILE echo "Scopy2-MinGW" >> $BUILD_STATUS_FILE @@ -96,6 +109,7 @@ create_build_status_file() { echo "" >> $BUILD_STATUS_FILE echo "All explicitly installed packages on build machine" >> $BUILD_STATUS_FILE echo "" >> $BUILD_STATUS_FILE + # List all explicitly installed packages pacman --noconfirm -Qe >> $BUILD_STATUS_FILE echo "" >> $BUILD_STATUS_FILE echo "Deps built from sources" >> $BUILD_STATUS_FILE @@ -109,6 +123,8 @@ clean_build_dir() { cd $BUILD_FOLDER } +# Generic CMake build function for dependencies +# Handles the common build pattern for CMake-based projects build_with_cmake() { INSTALL=$1 [ -z $INSTALL ] && INSTALL=ON @@ -125,17 +141,19 @@ build_with_cmake() { make install fi eval $CURRENT_BUILD_POST_MAKE + + # Record build info echo "$(basename -a "$(git config --get remote.origin.url)") - $(git rev-parse --abbrev-ref HEAD) - $(git rev-parse --short HEAD)" \ >> $BUILD_STATUS_FILE - #clean deps folder + # Clean source directory in CI to save space if [ "$INSTALL" == "ON" ] && [ "$CI_SCRIPT" == "ON" ];then git clean -xdf fi popd - # clear vars + # Clear variables for next build CURRENT_BUILD_CMAKE_OPTS="" CURRENT_BUILD_POST_CLEAN="" CURRENT_BUILD_PATCHES="" @@ -157,11 +175,12 @@ build_libserialport(){ make $JOBS [ "$INSTALL" == "ON" ] && make install - #clean deps folder + # Clean source in CI builds if [ "$INSTALL" == "ON" ] && [ "$CI_SCRIPT" == "ON" ];then git clean -xdf fi + # Record build info echo "$(basename -a "$(git config --get remote.origin.url)") - $(git rev-parse --abbrev-ref HEAD) - $(git rev-parse --short HEAD)" \ >> $BUILD_STATUS_FILE popd @@ -170,14 +189,14 @@ build_libserialport(){ build_libiio() { CURRENT_BUILD=libiio CURRENT_BUILD_CMAKE_OPTS="\ - ${RC_COMPILER_OPT}\ - -DWITH_USB_BACKEND:BOOL=ON\ - -DWITH_SERIAL_BACKEND:BOOL=ON\ - -DCSHARP_BINDINGS:BOOL=OFF\ - -DPYTHON_BINDINGS:BOOL=OFF\ - -DHAVE_DNS_SD:BOOL=ON\ - -DENABLE_IPV6:BOOL=OFF\ - -DWITH_EXAMPLES:BOOL=ON\ + ${RC_COMPILER_OPT}\ # Windows resource compiler + -DWITH_USB_BACKEND:BOOL=ON\ # Enable USB support + -DWITH_SERIAL_BACKEND:BOOL=ON\ # Enable serial port support + -DCSHARP_BINDINGS:BOOL=OFF\ # No C# bindings needed + -DPYTHON_BINDINGS:BOOL=OFF\ # No Python bindings needed + -DHAVE_DNS_SD:BOOL=ON\ # Enable network discovery + -DENABLE_IPV6:BOOL=OFF\ # Disable IPv6 (simplifies) + -DWITH_EXAMPLES:BOOL=ON\ # Build example programs " build_with_cmake $1 } @@ -280,12 +299,14 @@ build_qwt() { pushd $STAGING_AREA/$CURRENT_BUILD git clean -xdf +# Apply inline patch to fix install prefix +# Changes empty prefix to /mingw64 patch -p1 <<-EOF --- a/qwtconfig.pri +++ b/qwtconfig.pri @@ -24,7 +24,7 @@ unix { } - + win32 { - QWT_INSTALL_PREFIX = "" + QWT_INSTALL_PREFIX = "/mingw64" @@ -313,11 +334,12 @@ EOF cp $STAGING_DIR/lib/qwt.dll $STAGING_DIR/bin/qwt.dll - #clean deps folder + # Clean source in CI if [ "$INSTALL" == "ON" ] && [ "$CI_SCRIPT" == "ON" ];then git clean -xdf fi + # Record build info echo "$(basename -a "$(git config --get remote.origin.url)") - $(git rev-parse --abbrev-ref HEAD) - $(git rev-parse --short HEAD)" \ >> $BUILD_STATUS_FILE popd @@ -329,6 +351,7 @@ build_libsigrokdecode() { pushd $STAGING_AREA/$CURRENT_BUILD git reset --hard git clean -xdf + # Apply Windows compatibility patch patch -p1 < ${WORKFOLDER}/sigrokdecode-windows-fix.patch ./autogen.sh @@ -337,7 +360,9 @@ build_libsigrokdecode() { if [ "$USE_STAGING" == "ON" ] then + # LIBSIGROKDECODE_EXPORT needed for Windows DLL CPPFLAGS="-DLIBSIGROKDECODE_EXPORT=1" ./configure --prefix $STAGING_AREA_DEPS ${AUTOCONF_OPTS} + # LD_RUN_PATH helps find libraries at runtime LD_RUN_PATH=$STAGING_AREA_DEPS/lib make $JOBS else CPPFLAGS="-DLIBSIGROKDECODE_EXPORT=1" ./configure ${AUTOCONF_OPTS} @@ -348,11 +373,12 @@ build_libsigrokdecode() { make install fi - #clean deps folder + # Clean source in CI if [ "$INSTALL" == "ON" ] && [ "$CI_SCRIPT" == "ON" ];then git clean -xdf fi + # Record build info echo "$(basename -a "$(git config --get remote.origin.url)") - $(git rev-parse --abbrev-ref HEAD) - $(git rev-parse --short HEAD)" \ >> $BUILD_STATUS_FILE popd @@ -396,35 +422,34 @@ build_genalyzer() { build_with_cmake $1 } -# -# Helper functions -# - +# Build all dependencies in correct order build_deps() { - install_packages - create_build_status_file - clone - build_libserialport ON - build_libiio ON - build_libad9361 ON - build_libm2k ON - build_spdlog ON - build_libsndfile ON - build_volk ON - build_gnuradio ON - build_grscopy ON - build_grm2k ON - build_qwt ON - build_libsigrokdecode ON - build_libtinyiiod ON - build_kddock ON - build_ecm ON - build_karchive ON - build_genalyzer ON + install_packages # Install MSYS2 packages + create_build_status_file # Initialize build log + clone # Clone all source repositories + + build_libserialport ON # Serial port library + build_libiio ON # Industrial I/O library + build_libad9361 ON # AD9361 transceiver library + build_libm2k ON # ADALM2000 library + build_spdlog ON # Fast logging library + build_libsndfile ON # Sound file I/O + build_volk ON # Vector-Optimized Library of Kernels + build_gnuradio ON # GNU Radio framework + build_grscopy ON # GNU Radio Scopy blocks + build_grm2k ON # GNU Radio M2K blocks + build_qwt ON # Qt plotting widgets + build_libsigrokdecode ON # Protocol decoding + build_libtinyiiod ON # Tiny IIO daemon library + build_kddock ON # Docking framework + build_ecm ON # Extra CMake modules + build_karchive ON # KDE archive library + build_genalyzer ON # Signal analysis library } for arg in $@; do $arg done +# Default action: build all dependencies build_deps diff --git a/ci/x86_64/README.md b/ci/x86_64/README.md new file mode 100644 index 0000000000..b08d40cd76 --- /dev/null +++ b/ci/x86_64/README.md @@ -0,0 +1,140 @@ +# x86_64 CI Directory + +## Folder overview + +This directory contains scripts for building Scopy as an AppImage for x86_64 Linux systems. AppImage is a portable application format that runs on most Linux distributions without installation. + +### Scripts + +#### `x86-64_appimage_process.sh`- Main build script for creating x86_64 AppImage + +#### `copy-deps.sh`- Copies runtime dependencies into AppDir + +### Docker Support + +#### `create_docker_image.sh`- **Purpose**: Creates Docker image using Ubuntu20 or Ubuntu24 as starting image + +#### `docker/Dockerfile`- Defines x86_64 build environment + +## Build Process + +### Building Scopy using the Docker Image + +`Prerequisites:` + +- [Docker](https://docs.docker.com/desktop/setup/install/windows-install/) +- the rest of the dependencies are installed using the scripts + +1. Pull the Docker image + + ```bash + docker pull cristianbindea/scopy2-x86_64-appimage + ``` + +2. Run build in container, and bind the scopy repo to a volume inside the container + + ```bash + docker run -v $(pwd):/scopy cristianbindea/scopy2-x86_64-appimage + ``` + +3. Execute the build script inside the container + + ```bash + scopy/ci/x86-64_appimage_process.sh + ``` + +### Building the Docker Image + +To build the Docker image, just run the script and select the required architecture. + + ```bash + ci/x86_64/create_docker_image.sh ubuntu20 + # or + ci/x86_64/create_docker_image.sh ubuntu24 + ``` + +### Building locally from sources + +Run build script: + + ```bash + ./x86-64_appimage_process.sh configure_system generate_appimage + ``` + +## AppImage Creation Process + +1. **Build Phase**: + - Compiles Scopy + +2. **AppDir Assembly**: + - Creates AppDir directory structure + - Copies executables to AppDir/usr/bin + - Copies libraries to AppDir/usr/lib + - Copies the rest of the required files in AppDir + +3. **Dependency Resolution**: + - Runs `copy-deps.sh` to bundle libraries + - Includes Qt plugins and platform files + - Bundles required system libraries + +4. **AppImage Generation**: + - Uses appimagetool to create final image + - Embeds AppRun and desktop files + - Creates self-contained executable + +## Output + +- **scopy.AppDir/**: Unpacked application directory (for debugging) +- **Scopy-x86_64.AppImage**: Portable application + +## AppDir Structure + +```text +scopy.AppDir/ +├── AppRun # Entry point script +├── scopy.desktop # Desktop integration +├── scopy.png # Application icon +└── usr/ + ├── bin/ # Executables + │ └── scopy + ├── lib/ # Libraries + │ ├── *.so # Shared libraries + │ └── qt5/ # Qt plugins + └── share/ # Resources + └── scopy/ # Application data +``` + +## Running the AppImage + +```bash + chmod +x Scopy-x86_64.AppImage + ./Scopy-x86_64.AppImage +``` + +You can even extract and inspect the files inside the AppImage: + +```bash + ./Scopy-x86_64.AppImage --appimage-extract +``` + +## CI Integration + +- **GitHub Actions**: `.github/workflows/appimage-x86_64.yml` + +## Troubleshooting + +Common issues: + +- **"No such file"**: Make executable with `chmod +x` +- **Missing libraries**: Run `QT_DEBUG_PLUGINS=1 ./Scopy-x86_64.AppImage` for details +- **FUSE errors**: Extract with `--appimage-extract` if FUSE unavailable + +## Notes + +- AppImage provides portability +- No root permissions required +- Compatible with most x86_64 Linux distributions +- Works on systems without Qt installed +- Larger file size due to bundled dependencies +- Can run from USB drives or network shares +- Updates require downloading new AppImage diff --git a/ci/x86_64/copy-deps.sh b/ci/x86_64/copy-deps.sh index 8b5e771b9e..850e657663 100755 --- a/ci/x86_64/copy-deps.sh +++ b/ci/x86_64/copy-deps.sh @@ -1,35 +1,70 @@ #!/bin/bash +# x86_64 Library Dependency Copier +# =============================== +# Purpose: Recursively copy shared library dependencies for AppImage +# Usage: ./copy-deps.sh binary_file destination_directory +# +# This script: +# 1. Uses ldd to find dependencies +# 2. Recursively processes each dependency +# 3. Excludes system libraries using AppImage excludelist +# 4. Strips debug symbols to reduce size + set -e SRC_DIR=$(git rev-parse --show-toplevel 2>/dev/null ) || \ SRC_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && cd ../../ && pwd ) -BINARY=$1 -LOCATION=$2 -LIBS_ARRAY=() +# Script parameters +BINARY=$1 # Binary or library to analyze +LOCATION=$2 # Where to copy dependencies +LIBS_ARRAY=() # Track processed libraries to avoid duplicates + +# Download and parse AppImage excludelist +# These are system libraries that should not be bundled BLACKLISTED=($(wget --quiet https://raw.githubusercontent.com/probonopd/AppImages/master/excludelist -O - | sort | uniq | cut -d '#' -f 1 | grep -v "^#.*" | grep "[^-\s]")) BLACKLISTED+=(libglib-2.0.so.0) +# Add AppDir libraries to search path +# This ensures we find our bundled libraries first export LD_LIBRARY_PATH="${APP_DIR}/usr/lib:${SRC_DIR}/build:$LD_LIBRARY_PATH" + +# Recursive function to process library dependencies +# ================================================= +# Uses ldd to find dependencies and recursively processes each one run_ldd(){ + # Parse ldd output to extract library paths + # ldd format: "libfoo.so.1 => /usr/lib/libfoo.so.1 (0x...)" for library in $(ldd "$1" | cut -d '>' -f 2 | awk '{print $1}') do - # check if the library exists at that path and if it was processed already or blacklisted + # Skip if library is blacklisted if ! [[ "${BLACKLISTED[*]}" =~ "${library##*/}" ]]; then + # Process if: + # - File exists + # - Not already processed (avoid infinite loops) if [ -f "${library}" ] && ! [[ "${LIBS_ARRAY[*]}" =~ "${library}" ]]; then + # Mark as processed LIBS_ARRAY+=("${library}") echo "---Added new lib: ${library}" + + # Copy library if not already present if [ ! -f "${LOCATION}"/"${library##*/}" ]; then cp "${library}" "${LOCATION}" + # If it's a symlink, also copy the target [ -L "${library}" ] && cp "$(realpath "${library}")" "${LOCATION}" + # Strip debug symbols to reduce size strip --strip-unneeded "${LOCATION}"/"${library##*/}" fi + + # Recursively process this library's dependencies run_ldd "${library}" fi fi done } +# Main execution +# Process each binary/library passed as argument for arg in $BINARY; do run_ldd "${arg}" done diff --git a/ci/x86_64/create_docker_image.sh b/ci/x86_64/create_docker_image.sh index d3f117ecf9..ba0b8fdd27 100755 --- a/ci/x86_64/create_docker_image.sh +++ b/ci/x86_64/create_docker_image.sh @@ -1,5 +1,11 @@ #!/bin/bash -ex +# Create Docker Image for x86_64 AppImage Build +# =========================== +# Usage: ./create_docker_image.sh ubuntu20 +# or +# Usage: ./create_docker_image.sh ubuntu24 + SRC_SCRIPT=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) ubuntu20(){ diff --git a/ci/x86_64/x86-64_appimage_process.sh b/ci/x86_64/x86-64_appimage_process.sh index c48e1e47ea..cf465eecb8 100755 --- a/ci/x86_64/x86-64_appimage_process.sh +++ b/ci/x86_64/x86-64_appimage_process.sh @@ -1,20 +1,38 @@ #!/bin/bash + +# x86_64 AppImage Build Script +# =========================== +# Build portable AppImage for x86_64 Linux systems +# Usage: ./x86-64_appimage_process.sh [function_name ...] +# +# This script creates a portable AppImage that includes: +# - Scopy application +# - All required libraries +# - Qt framework +# - Python runtime for decoders +# - IIO emulator +# +# The AppImage can run on most x86_64 Linux distributions +# without requiring installation + set -ex -## Set STAGING +# Staging configuration +# OFF: Use system libraries where possible +# ON: Use isolated dependency tree USE_STAGING=OFF -## +# Determine source directories SRC_DIR=$(git rev-parse --show-toplevel 2>/dev/null ) || \ SRC_DIR=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && cd ../../ && pwd ) SRC_SCRIPT=$( cd -- "$( dirname -- "${BASH_SOURCE[0]}" )" &> /dev/null && pwd ) +# CI-specific configuration if [ "$CI_SCRIPT" == "ON" ]; then USE_STAGING=OFF SRC_DIR=$GITHUB_WORKSPACE fi - export APPIMAGE=1 LIBSERIALPORT_BRANCH=master @@ -35,6 +53,7 @@ ECM_BRANCH=kf5 KARCHIVE_BRANCH=kf5 GENALYZER_BRANCH=main +# Python version detection if [ -f /etc/os-release ]; then . /etc/os-release @@ -60,38 +79,46 @@ else echo "/etc/os-release not found. Cannot determine OS." fi +# Qt is installed using aqtinstall QT_INSTALL_LOCATION=/opt/Qt QT=$QT_INSTALL_LOCATION/5.15.2/gcc_64 STAGING_AREA=$SRC_SCRIPT/staging QMAKE_BIN=$QT/bin/qmake CMAKE_BIN=${STAGING_AREA}/cmake/bin/cmake -JOBS=-j14 +JOBS=-j14 # Parallel build jobs +# AppImage structure +# ================= APP_DIR_NAME=scopy.AppDir APP_DIR=$SRC_SCRIPT/$APP_DIR_NAME +# Different names for different Ubuntu versions APP_IMAGE_FOLDER=scopy-x86_64-appimage [ "$VERSION_ID" == "20.04" ] && APP_IMAGE_FOLDER=scopy-x86_64-appimage-ubuntu20 APP_IMAGE_NAME=${APP_IMAGE_FOLDER}.AppImage APP_IMAGE=$SRC_SCRIPT/$APP_IMAGE_NAME +# Export variables for GitHub Actions [ $CI_SCRIPT ] && echo "app_image_folder=$APP_IMAGE_FOLDER" >> "$GITHUB_ENV" [ $CI_SCRIPT ] && echo "app_image_name=$APP_IMAGE_NAME" >> "$GITHUB_ENV" +# Build status tracking BUILD_STATUS_FILE=$SRC_SCRIPT/build-status if [ "$USE_STAGING" == "ON" ] then echo -- USING STAGING FOLDER: $STAGING_AREA_DEPS STAGING_AREA_DEPS=$STAGING_AREA/dependencies + # Add staging libraries to runtime path export LD_LIBRARY_PATH=$STAGING_AREA_DEPS/lib:$QT/lib:$LD_LIBRARY_PATH + CMAKE_OPTS=(\ -DCMAKE_LIBRARY_PATH=$STAGING_AREA_DEPS \ -DCMAKE_INSTALL_PREFIX=$STAGING_AREA_DEPS \ -DCMAKE_PREFIX_PATH=$QT\;$STAGING_AREA_DEPS \ - -DCMAKE_EXE_LINKER_FLAGS=-L$STAGING_AREA_DEPS\;-L$STAGING_AREA_DEPS/lib \ + -DCMAKE_EXE_LINKER_FLAGS=-L$STAGING_AREA_DEPS\;-L$STAGING_AREA_DEPS/lib \ # Linker paths -DCMAKE_SHARED_LINKER_FLAGS=-L$STAGING_AREA_DEPS\;-L$STAGING_AREA_DEPS/lib \ -DCMAKE_BUILD_TYPE=RelWithDebInfo \ -DCMAKE_VERBOSE_MAKEFILE=ON \ @@ -101,6 +128,7 @@ if [ "$USE_STAGING" == "ON" ] echo -- NO STAGING: INSTALLING IN SYSTEM STAGING_AREA_DEPS=/usr/local export LD_LIBRARY_PATH=$QT/lib:$LD_LIBRARY_PATH: + CMAKE_OPTS=(\ -DCMAKE_PREFIX_PATH=$QT\;$STAGING_AREA_DEPS \ -DCMAKE_BUILD_TYPE=RelWithDebInfo \ @@ -108,6 +136,7 @@ if [ "$USE_STAGING" == "ON" ] ) fi +# Complete CMake command CMAKE="$CMAKE_BIN ${CMAKE_OPTS[*]}" echo -- USING CMAKE COMMAND: echo $CMAKE @@ -137,8 +166,8 @@ clone() { popd } +# Installing Qt using the aqt tool https://github.com/miurahr/aqtinstall install_qt() { - # installing Qt using the aqt tool https://github.com/miurahr/aqtinstall [ "$PYTHON_VERSION" == "python3.12" ] && sudo pip3 install --no-cache-dir --break-system-packages aqtinstall [ "$PYTHON_VERSION" == "python3.11" ] && sudo pip3 install --no-cache-dir aqtinstall [ "$PYTHON_VERSION" == "python3.9" ] && sudo pip3 install --no-cache-dir aqtinstall @@ -151,20 +180,22 @@ download_tools() { if [ ! -d cmake ];then wget https://github.com/Kitware/CMake/releases/download/v3.29.0-rc2/cmake-3.29.0-rc2-linux-x86_64.tar.gz - tar -xf cmake*.tar.gz && rm cmake*.tar.gz && mv cmake* cmake # unzip and rename + tar -xf cmake*.tar.gz && rm cmake*.tar.gz && mv cmake* cmake fi - # download tools for creating the AppDir and the AppImage + # linuxdeploy: tool for creating AppDirs if [ ! -f linuxdeploy-x86_64.AppImage ];then wget https://github.com/linuxdeploy/linuxdeploy/releases/download/1-alpha-20240109-1/linuxdeploy-x86_64.AppImage chmod +x linuxdeploy-x86_64.AppImage fi + # Handles Qt-specific deployment if [ ! -f linuxdeploy-plugin-qt-x86_64.AppImage ];then wget https://github.com/linuxdeploy/linuxdeploy-plugin-qt/releases/download/1-alpha-20240109-1/linuxdeploy-plugin-qt-x86_64.AppImage chmod +x linuxdeploy-plugin-qt-x86_64.AppImage fi + # Creates the AppImage from AppDir if [ ! -f linuxdeploy-plugin-appimage-x86_64.AppImage ];then wget https://github.com/linuxdeploy/linuxdeploy-plugin-appimage/releases/download/1-alpha-20230713-1/linuxdeploy-plugin-appimage-x86_64.AppImage chmod +x linuxdeploy-plugin-appimage-x86_64.AppImage @@ -445,6 +476,8 @@ build_scopy() { popd } +# Assembles all files needed for the AppImage following the AppDir specification +# https://docs.appimage.org/reference/appdir.html create_appdir(){ rm -rf $APP_DIR @@ -458,13 +491,17 @@ create_appdir(){ LIBS=$(find $APP_DIR/usr -type f -name "libscopy*.so") COPY_DEPS=${SRC_DIR}/ci/x86_64/copy-deps.sh - export QMAKE=$QMAKE_BIN # this is needed for deploy-plugin-qt.AppImage - # inside a docker image you can't run an appimage executable without privileges - # so the solution is to extract the appimage first and only then to run it + export QMAKE=$QMAKE_BIN # AppImage Qt plugin needs to find qmake + + # Workaround for running AppImages in Docker + # AppImages require FUSE which isn't available in Docker + # This flag makes them extract and run instead export PATH=$QT:$PATH LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$STAGING_AREA_DEPS/lib:$QT/lib:$APP_DIR/usr/lib sudo ldconfig export APPIMAGE_EXTRACT_AND_RUN=1 + + # Run linuxdeploy to create basic AppDir structure ${STAGING_AREA}/linuxdeploy-x86_64.AppImage \ --appdir $APP_DIR \ --executable $APP_DIR/usr/bin/scopy \ @@ -478,11 +515,12 @@ create_appdir(){ cp $EMU_BUILD_FOLDER/iio-emu $APP_DIR/usr/bin cp ${STAGING_AREA_DEPS}/lib/tinyiiod.so* $APP_DIR/usr/lib - # search for the python version linked by cmake and copy inside the appimage the same version + # Copy Python runtime FOUND_PYTHON_VERSION=$(grep 'PYTHON_VERSION' $SRC_DIR/build/CMakeCache.txt | awk -F= '{print $2}' | grep -o 'python[0-9]\+\.[0-9]\+') python_path=/usr/lib/$FOUND_PYTHON_VERSION cp -r $python_path $APP_DIR/usr/lib + # Copy protocol decoders if [ -d $STAGING_AREA_DEPS/share/libsigrokdecode/decoders ]; then cp -r $STAGING_AREA_DEPS/share/libsigrokdecode/decoders $APP_DIR/usr/lib elif [ -d $STAGING_AREA/libsigrokdecode/decoders ];then @@ -494,11 +532,14 @@ create_appdir(){ cp $STAGING_AREA_DEPS/lib/libspdlog.so* $APP_DIR/usr/lib cp -r $QT/plugins $APP_DIR/usr + cp $QT/lib/libQt5XcbQpa.so* $APP_DIR/usr/lib cp $QT/lib/libQt5WaylandClient.so* $APP_DIR/usr/lib cp $QT/lib/libQt5EglFSDeviceIntegration.so* $APP_DIR/usr/lib cp $QT/lib/libQt5DBus.so* $APP_DIR/usr/lib + cp $STAGING_AREA_DEPS/lib/libgenalyzer.so* $APP_DIR/usr/lib + cp /usr/lib/x86_64-linux-gnu/libXdmcp.so* $APP_DIR/usr/lib cp /usr/lib/x86_64-linux-gnu/libbsd.so* $APP_DIR/usr/lib cp /usr/lib/x86_64-linux-gnu/libXau.so* $APP_DIR/usr/lib @@ -506,6 +547,7 @@ create_appdir(){ popd } +# Packages the AppDir into a single executable file create_appimage(){ rm -rf $APP_IMAGE @@ -521,7 +563,6 @@ generate_ci_envs(){ $SRC_DIR/ci/general/gen_ci_envs.sh > $SRC_DIR/ci/x86_64/gh-actions.envs } -# move the staging folder that contains the tools needed for the build to the known location move_tools(){ [ -d /home/runner/staging ] && mv /home/runner/staging $STAGING_AREA || echo "Staging folder not found or already moved" if [ ! -d $STAGING_AREA ]; then @@ -535,10 +576,9 @@ move_appimage(){ } -# -# Helper functions -# +# Helper Functions +# Build all dependencies from source build_deps(){ clone download_tools @@ -560,20 +600,23 @@ build_deps(){ build_genalyzer ON } +# Complete build from source to AppImage run_workflow(){ [ "$CI_SCRIPT" == "ON" ] && move_tools || download_tools - build_iio-emu - build_scopy - create_appdir - create_appimage - move_appimage + build_iio-emu # Build emulator + build_scopy # Build application + create_appdir # Assemble files + create_appimage # Package AppImage + move_appimage # Move to output location } +# Download build tools only get_tools(){ install_packages download_tools } +# Build AppImage assuming dependencies exist generate_appimage(){ download_tools build_iio-emu @@ -582,11 +625,12 @@ generate_appimage(){ create_appimage } +# Complete system setup including Qt configure_system(){ - install_packages - install_qt - build_deps - download_tools + install_packages # System packages + install_qt # Qt framework + build_deps # All dependencies + download_tools # Build tools } for arg in $@; do