Make build/conf/checksum consistent between different environments

[tjallingt]

I'm using the setup-environment both on my own computer to update the configuration to the latest version and in GitHub actions before running bitbake.

I figured out that running it like this:

MACHINE="verdin-am62" DISTRO="torizon" EULA="1" source setup-environment build

Ensures that it will only "initialize" the build/conf directory if it detects that the script itself has changed.

Unfortunately the shasum differs between my local computer and CI. This is both because of the full path to the file being different and likely differences in line-endings.

For example the file currently looks like this:

a76a0d20cb25b52b4ae4823bcc6b1697e2f0887120d7f5589e8324f86662bc962e18626089792e64ad1b3c3c0d42e07b7c6ef7022f7444cb44d2060b7d0ad1aa */c/Users/ttolle/Dev/qm-goingdutch-yocto/layers/meta-toradex-torizon/scripts/lib/setup-devices/setup-environment-toradex

On my CI machine the checksum looks like this:

e02964df958473bce72be55b18d91c167ddc979e2c97f6cdc0e2b9ad90822c940f7a1c6a5c8f3bb3524ef83a5dbcc8173f5bf8f1de116b4cdb4794c229175439  /home/docker/actions-runner/_work/qm-goingdutch-yocto/qm-goingdutch-yocto/layers/meta-toradex-torizon/scripts/lib/setup-devices/setup-environment-toradex

I'm not sure if i'm using this script in the way you're supposed to.
I can solve this issue for myself by just taking the start of the setup-environment-toradex script and using only that in CI:

#!/bin/bash
# -*- mode: shell-script-mode; tab-width: 4; indent-tabs-mode: nil; c-basic-offset: 4 -*-
#
# Copyright (C) 2012-13 O.S. Systems Software LTDA.
# Copyright (C) 2017 Open Source Foundries Ltd.
# Authored-by:  Otavio Salvador <otavio@ossystems.com.br>
# Adopted to Angstrom:  Khem Raj <raj.khem@gmail.com>
#
# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License version 2 as
# published by the Free Software Foundation.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License along
# with this program; if not, write to the Free Software Foundation, Inc.,
# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
#

_TDX_OEROOT=$(pwd)
cd "$_TDX_OEROOT"
if [ -n "$ZSH_VERSION" ]; then
    setopt sh_word_split
    setopt clobber
elif [ -n "$BASH_VERSION" ]; then
    set +o noclobber
fi

# DISTRO defaults to 'torizon' for am62/imx8/qemuarm64/genericx86-64 machines
# DISTRO defaults to 'torizon-upstream' for other machines
_TDX_DISTRO_DEFAULT="torizon-upstream" && echo "$MACHINE" | grep -E -q '(imx8|am62|am69|qemuarm64|genericx86-64)' && _TDX_DISTRO_DEFAULT="torizon"
DISTRO=${DISTRO:-$_TDX_DISTRO_DEFAULT}

if [ -z "${SDKMACHINE}" ]; then
    SDKMACHINE='x86_64'
fi

_TDX_MANIFESTS="${_TDX_OEROOT}"/.repo/manifests
_TDX_SCRIPTS="${_TDX_OEROOT}"/layers/meta-toradex-torizon/scripts

# We can be called with only 1 parameter max (build folder)
_TDX_BUILDDIR=${1:-build-$DISTRO}
# If current BUILDDIR is relative then prepend OEROOT
case ${_TDX_BUILDDIR} in
    /* ) ;;
    *  ) _TDX_BUILDDIR=$(readlink -f ${_TDX_BUILDDIR});;
esac
# Get rid of double slash. sed calls in OE staging_processfixme
# seems to have troubles if this makes it into OE variables.
_TDX_BUILDDIR=${_TDX_BUILDDIR%%/}

# Set BBPATH for recipetool
BBPATH=${_TDX_BUILDDIR}

# Clean up PATH, because if it includes tokens to current directories somehow,
# wrong binaries can be used instead of the expected ones during task execution
PATH=$(echo "${PATH}" | sed 's/\(:.\|:\)*:/:/g;s/^.\?://;s/:.\?$//')
PATH="${_TDX_OEROOT}/bitbake/bin:${_TDX_OEROOT}/.repo/repo:${PATH}"
PATH="${_TDX_OEROOT}/layers/openembedded-core/bitbake/bin:${PATH}"
PATH="${_TDX_OEROOT}/layers/openembedded-core/scripts:${PATH}"
# Remove duplicate path entries
PATH=$(echo "$PATH" |
       awk -F: '{for (i=1;i<=NF;i++) { if ( !x[$i]++ ) printf("%s:",$i); }}' |
       sed 's/:$//')

BB_ENV_PASSTHROUGH_ADDITIONS_OE="MACHINE DISTRO TCMODE TCLIBC HTTP_PROXY http_proxy \
HTTPS_PROXY https_proxy FTP_PROXY ftp_proxy FTPS_PROXY ftps_proxy ALL_PROXY \
all_proxy NO_PROXY no_proxy SSH_AGENT_PID SSH_AUTH_SOCK BB_SRCREV_POLICY \
SDKMACHINE BB_NUMBER_THREADS BB_NO_NETWORK PARALLEL_MAKE GIT_PROXY_COMMAND \
SOCKS5_PASSWD SOCKS5_USER SCREENDIR STAMPS_DIR BBPATH_EXTRA BB_SETSCENE_ENFORCE"

BB_ENV_PASSTHROUGH_ADDITIONS="$(echo $BB_ENV_PASSTHROUGH_ADDITIONS $BB_ENV_PASSTHROUGH_ADDITIONS_OE | tr ' ' '\n' | LC_ALL=C sort --unique | tr '\n' ' ')"

export PATH
export _TDX_BUILDDIR
export BBPATH
export BB_ENV_PASSTHROUGH_ADDITIONS

However I would prefer to just use the provided setup-environment if possible. Ideally I would have the one in CI never update the build/conf files and instead error if they are out of sync.

So my questions are:

• Am I using setup-environment incorrectly? If so what should I be doing instead?
• If I'm using it (more or less) correctly, can the checksum be changed to something that is consistent across different environments? maybe a git hash or manually updated version inside the script?
• Could an option (for example CI="1") be added that prevents the overwriting of the build/conf files and instead just returns an error?

[tjallingt]

I'm just looking at the other setup-environment-* scripts, maybe setup-environment-toradex could work like setup-enviroment-ti where it generates the conf/setenv file.