Skip to content

Commit 2723593

Browse files
quic-mtharuquic-mtharu
authored
CI: Add qcs6490 target support with unified build matrix and path handling improvements (#246)
This PR introduces a set of tightly related changes, all aimed at enabling support for the new **qcs6490** target in the CI pipeline. While the update touches multiple areas, each change is interdependent and collectively required to ensure proper integration and functionality for `qcs6490`. ### Key Changes: 1. **Matrix Consolidation** - Removed the separate `full_matrix` definition. - Unified the configuration under a single `build_matrix`, now including all necessary fields: `machine`, `firmware`, `lavaname`, and `target`. - This simplifies the pipeline setup and ensures consistent parameter availability for job naming and test execution. 2. **Target-Specific Enhancements** - **qcs6490 Support Added**: Enabled support for the `qcs6490` target alongside existing `sa8775p` and `qcs8300` targets. - **Target Path Resolution**: For qcs6490, the DSP firmware search path uses `qcs6490`, while the repo folder path uses `qcm6490`. To handle this discrepancy, a new `target` attribute is introduced: ```yaml target: description: target identifier type: string required: true ``` 3. **Flexible Matching Logic** - Updated the path matching logic to allow matching the target name **anywhere** in the path, rather than only at the end. This improves compatibility with diverse directory structures. 4. **Simplified LAVA Machine Assignment** - Removed the logic for extracting the LAVA machine name dynamically. - The machine name is now directly assigned from `build_matrix.lavaname`, reducing complexity and improving maintainability. 5. **Attribute Renaming** - Standardized all four target configuration attributes for clarity and consistency: - `deviceTree`: Device tree blob name - `linuxFirmware`: Firmware identifier - `lavaDeviceName`: LAVA machine name - `hexDSPBinary`: Hexagon DSP binary identifier --- All these changes are introduced as part of a single PR to ensure **qcs6490 target support is complete and functional**. Each commit is tightly coupled and contributes to the overall goal of enabling qcs6490 in a clean and maintainable way.
2 parents c469d38 + df3e9ac commit 2723593

File tree

9 files changed

+101
-111
lines changed

9 files changed

+101
-111
lines changed

‎.github/actions/aws_s3_helper/action.yml‎

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -21,8 +21,8 @@ inputs:
2121
description: Mode of operation (single-upload, multi-upload, download)
2222
required: true
2323
default: single-upload
24-
machine:
25-
description: Target machine name or identifier, used to organize uploads in S3.
24+
deviceTree:
25+
description: Target device Tree name or identifier, used to organize uploads in S3.
2626
type: string
2727
required: true
2828

@@ -38,16 +38,16 @@ runs:
3838
id: sync-data
3939
shell: bash
4040
env:
41-
UPLOAD_LOCATION: ${{ github.repository_owner }}/${{ github.event.repository.name }}/${{ github.workflow }}/${{ github.head_ref != '' && github.head_ref || github.run_id }}/${{ inputs.machine }}/
41+
UPLOAD_LOCATION: ${{ github.repository_owner }}/${{ github.event.repository.name }}/${{ github.workflow }}/${{ github.head_ref != '' && github.head_ref || github.run_id }}/${{ inputs.deviceTree }}/
4242
run: |
4343
echo "::group::$(printf '__________ %-100s' 'Process' | tr ' ' _)"
4444
case "${{ inputs.mode }}" in
4545
multi-upload)
4646
echo "Uploading files to S3 bucket..."
4747
first_line=true
4848
# Start the JSON object
49-
mkdir -p "${{ github.workspace }}/${{ inputs.machine }}"
50-
echo "{" > ${{ github.workspace }}/${{ inputs.machine }}/presigned_urls_${{ inputs.machine }}.json
49+
mkdir -p "${{ github.workspace }}/${{ inputs.deviceTree }}"
50+
echo "{" > ${{ github.workspace }}/${{ inputs.deviceTree }}/presigned_urls_${{ inputs.deviceTree }}.json
5151
while IFS= read -r file; do
5252
if [ -f "$file" ]; then
5353
echo "Uploading $file..."
@@ -59,17 +59,17 @@ runs:
5959
if [ "$first_line" = true ]; then
6060
first_line=false
6161
else
62-
echo "," >> ${{ github.workspace }}/${{ inputs.machine }}/presigned_urls_${{ inputs.machine }}.json
62+
echo "," >> ${{ github.workspace }}/${{ inputs.deviceTree }}/presigned_urls_${{ inputs.deviceTree }}.json
6363
fi
6464
# Append the pre-signed URL to the file
65-
echo " \"${file}\": \"${presigned_url}\"" >> ${{ github.workspace }}/${{ inputs.machine }}/presigned_urls_${{ inputs.machine }}.json
65+
echo " \"${file}\": \"${presigned_url}\"" >> ${{ github.workspace }}/${{ inputs.deviceTree }}/presigned_urls_${{ inputs.deviceTree }}.json
6666
echo "Pre-signed URL for $file: $presigned_url"
6767
else
6868
echo "Warning: $file does not exist or is not a regular file."
6969
fi
7070
done < "${{ inputs.local_file }}"
7171
# Close the JSON object
72-
echo "}" >> ${{ github.workspace }}/${{ inputs.machine }}/presigned_urls_${{ inputs.machine }}.json
72+
echo "}" >> ${{ github.workspace }}/${{ inputs.deviceTree }}/presigned_urls_${{ inputs.deviceTree }}.json
7373
;;
7474
single-upload)
7575
echo "Uploading single file to S3 bucket..."
@@ -94,6 +94,6 @@ runs:
9494
if: ${{ inputs.mode == 'multi-upload' }}
9595
uses: actions/upload-artifact@v4
9696
with:
97-
name: presigned_urls_${{ inputs.machine }}.json
98-
path: ${{ github.workspace }}/${{ inputs.machine }}/presigned_urls_${{ inputs.machine }}.json
97+
name: presigned_urls_${{ inputs.deviceTree }}.json
98+
path: ${{ github.workspace }}/${{ inputs.deviceTree }}/presigned_urls_${{ inputs.deviceTree }}.json
9999
retention-days: 3

‎.github/actions/lava_job_render/action.yml‎

Lines changed: 5 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ runs:
3333
}
3434
const filePath = p.join(
3535
process.env.GITHUB_WORKSPACE,
36-
`presigned_urls_${process.env.MACHINE}.json`
36+
`presigned_urls_${process.env.DEVICE_TREE}.json`
3737
);
3838
3939
if (fs.existsSync(filePath)) {
@@ -49,7 +49,7 @@ runs:
4949
const imageUrl = findUrlByFilename('Image');
5050
const vmlinuxUrl = findUrlByFilename('vmlinux');
5151
const firmwareUrl = findUrlByFilename('ramdisk_fastrpc.gz');
52-
const dtbFilename = `${process.env.MACHINE}.dtb`;
52+
const dtbFilename = `${process.env.DEVICE_TREE}.dtb`;
5353
const dtbUrl = findUrlByFilename(dtbFilename);
5454
// Set outputs
5555
core.setOutput('modules_url', modulesTarUrl);
@@ -76,7 +76,7 @@ runs:
7676
-v "$(dirname "$PWD")":"$(dirname "$PWD")" \
7777
-e dtb_url="${{ steps.process_urls.outputs.dtb_url }}" \
7878
${{ inputs.docker_image }} \
79-
jq '.artifacts["dtbs/qcom/${{ env.MACHINE }}.dtb"] = env.dtb_url' data/metadata.json > temp.json && mv temp.json data/metadata.json
79+
jq '.artifacts["dtbs/qcom/${{ env.DEVICE_TREE }}.dtb"] = env.dtb_url' data/metadata.json > temp.json && mv temp.json data/metadata.json
8080
8181
- name: Upload metadata.json
8282
id: upload_metadata
@@ -174,8 +174,8 @@ runs:
174174
--user "$(id -u):$(id -g)" \
175175
--workdir="$PWD" \
176176
-v "$(dirname "$PWD")":"$(dirname "$PWD")" \
177-
-e TARGET="${{ env.LAVA_NAME }}" \
178-
-e TARGET_DTB="${{ env.MACHINE }}" \
177+
-e TARGET="${{ env.LAVA_DEVICE_NAME }}" \
178+
-e TARGET_DTB="${{ env.DEVICE_TREE }}" \
179179
${{ inputs.docker_image }} \
180180
sh -c 'export BOOT_METHOD=fastboot && \
181181
export TARGET=${TARGET} && \

‎.github/actions/loading/action.yml‎

Lines changed: 21 additions & 24 deletions
Original file line numberDiff line numberDiff line change
@@ -2,21 +2,14 @@ name: Load Parameters
22
description: |
33
This composite action loads build parameters from a `MACHINES.json` file.
44
It parses the JSON file to create two matrices:
5-
- `build_matrix`: A slimmed-down matrix containing only 'machine' and 'firmware' for better visibility in job names.
6-
- `full_matrix`: A complete matrix including 'machine', 'firmware', and 'lavaname' for passing all necessary details to test jobs.
5+
- `build_matrix`: A complete matrix containing 'deviceTree', 'linuxFirmware', 'lavaDeviceName' and 'hexDSPBinary'
76
It expects the `MACHINES.json` file to be located at `ci/MACHINES.json` relative to the workspace root.
8-
Each entry in `MACHINES.json` should be a key-value pair where the key is the machine name
9-
and the value is an array `[firmware, lavaname]`.
107
118
outputs:
129
build_matrix:
1310
description: Build matrix
1411
value: ${{ steps.set-matrix.outputs.build_matrix }}
1512

16-
full_matrix:
17-
description: full matrix containing lava devails
18-
value: ${{ steps.set-matrix.outputs.full_matrix }}
19-
2013
runs:
2114
using: "composite"
2215
steps:
@@ -28,27 +21,31 @@ runs:
2821
const fs = require('fs');
2922
const path = require('path');
3023
const filePath = path.join(process.env.GITHUB_WORKSPACE, 'ci', 'MACHINES.json');
31-
let file;
24+
let fileData;
3225
try {
3326
if (!fs.existsSync(filePath)) {
34-
core.setFailed(`MACHINES.json not found at ${filePath}`);
27+
core.setFailed(`Error: MACHINES.json not found at ${filePath}`);
3528
return;
3629
}
37-
file = JSON.parse(fs.readFileSync(filePath, 'utf-8'));
30+
fileData = JSON.parse(fs.readFileSync(filePath, 'utf-8'));
3831
} catch (err) {
39-
core.setFailed(`Failed to load or parse MACHINES.json: ${err.message}`);
32+
core.setFailed(`Error loading or parsing MACHINES.json: ${err.message}`);
4033
return;
4134
}
42-
// Slim matrix for better job visibility
43-
const slim_matrix = Object.entries(file).map(([machine, [firmware]]) => ({ machine, firmware }));
44-
core.setOutput('build_matrix', JSON.stringify(slim_matrix));
45-
console.log(slim_matrix);
4635
47-
// Full matrix to pass to test jobs
48-
const complete_matrix = Object.entries(file).map(([machine, [firmware, lavaname]]) => ({
49-
machine,
50-
firmware,
51-
lavaname
52-
}));
53-
core.setOutput('full_matrix', JSON.stringify(complete_matrix));
54-
console.log(complete_matrix);
36+
// fileData is an object, not an array
37+
const matrix = Object.values(fileData)
38+
.filter(item =>
39+
typeof item === 'object' && item !== null &&
40+
'deviceTree' in item && 'linuxFirmware' in item && 'lavaDeviceName' in item && 'hexDSPBinary' in item
41+
)
42+
.map(({ deviceTree, linuxFirmware, lavaDeviceName, hexDSPBinary }) => ({
43+
deviceTree,
44+
linuxFirmware,
45+
lavaDeviceName,
46+
hexDSPBinary
47+
}));
48+
49+
core.setOutput('build_matrix', JSON.stringify(matrix));
50+
console.log("Generated build_matrix:");
51+
console.log(matrix);

‎.github/actions/sync/action.yml‎

Lines changed: 23 additions & 22 deletions
Original file line numberDiff line numberDiff line change
@@ -1,21 +1,22 @@
11
name: Sync workspace
22
description: |
3-
This composite action synchronizes a workspace by cleaning it up,
4-
checking out necessary code repositories, and preparing a firmware
5-
directory with required binaries. It clones and copies FastRPC hexagon
6-
DSP binaries and Linux firmware (specifically QCOM firmware)
7-
based on the provided `firmware` input. It also clones the
8-
`qualcomm-linux/kernel` repository.
9-
The action sets an output `workspace_path` to the GitHub workspace
10-
directory.
3+
This composite action synchronizes a workspace by cleaning it
4+
up,checking out necessary code repositories, and preparing a
5+
linux Firmware directory with required binaries.
6+
It clones and copies FastRPC hexagon DSP binaries and
7+
Linux firmware (specifically QCOM firmware) based on
8+
the provided `hexDSPBinary` and `linuxFirmware` input.
9+
It also clones the `qualcomm-linux/kernel` repository.
10+
The action sets an output `workspace_path` to the GitHub
11+
workspace directory.
1112
1213
inputs:
13-
machine:
14-
description: Target machine name
14+
linuxFirmware:
15+
description: Linux Firmware identifier
1516
type: string
1617
required: true
17-
firmware:
18-
description: Firmware identifier
18+
hexDSPBinary:
19+
description: Hexagon DSP binaries identifier
1920
type: string
2021
required: true
2122

@@ -48,7 +49,7 @@ runs:
4849
cd hexagon-dsp-binaries
4950
git fetch --all --tags --prune
5051
51-
TARGET="${{ inputs.firmware }}"
52+
TARGET="${{ inputs.hexDSPBinary }}"
5253
DSP_REF=""
5354
CONFIG="config.txt"
5455
@@ -58,8 +59,9 @@ runs:
5859
exit 1
5960
fi
6061
while read -r _ subdir dsp version; do
61-
echo "DSP binaries for $dsp in $subdir - $version"
62-
if [[ "${subdir,,}" == */"${TARGET,,}" ]]; then
62+
echo "DSP binaries for $dsp in $subdir - $version"
63+
# match anywhere in the path
64+
if [[ "${subdir,,}" == *"${TARGET,,}/"* ]]; then
6365
echo "Found matching DSP binaries for target '$TARGET': $subdir - $version"
6466
echo "Copying DSP binaries for $dsp in $subdir/$version"
6567
mkdir -p "../firmware_dir/usr/lib/dsp/${dsp}"
@@ -76,26 +78,25 @@ runs:
7678
run: |
7779
cd ${{ github.workspace }}
7880
git clone https://git.kernel.org/pub/scm/linux/kernel/git/firmware/linux-firmware.git
79-
TARGET_RAW="${{ inputs.firmware }}"
80-
# Remove "-ride" suffix if present
81-
TARGET="${TARGET_RAW%-ride}"
81+
TARGET="${{ inputs.hexDSPBinary }}"
82+
LINUX_FIRMWARE="${{ inputs.linuxFirmware }}"
8283
8384
# Construct the source path for qcom firmware in the cloned linux-firmware repo
8485
# It's usually under <cloned_repo_root>/qcom/<target>
8586
LINUX_FIRMWARE_QCOM_PATH="linux-firmware/qcom/${TARGET}"
8687
8788
# Check if the target qcom firmware directory exists in the cloned repo
8889
if [ ! -d "$LINUX_FIRMWARE_QCOM_PATH" ]; then
89-
echo "ERROR: Directory '$LINUX_FIRMWARE_QCOM_PATH' not found in linux-firmware repo."
90+
echo "ERROR: Directory '$LINUX_FIRMWARE_QCOM_PATH' not found in linux_firmware repo."
9091
exit -1
9192
else
9293
# Create the destination directory in firmware_dir
93-
mkdir -p "${{ github.workspace }}/firmware_dir/usr/lib/firmware/qcom/${TARGET}"
94+
mkdir -p "${{ github.workspace }}/firmware_dir/usr/lib/firmware/qcom/${LINUX_FIRMWARE}"
9495
9596
# Copy the contents of the qcom target directory
9697
# Use trailing slashes to copy contents, not the directory itself
97-
cp -rf "${LINUX_FIRMWARE_QCOM_PATH}/"* "${{ github.workspace }}/firmware_dir/usr/lib/firmware/qcom/${TARGET}/"
98-
echo "Copied QCOM firmware from '${LINUX_FIRMWARE_QCOM_PATH}' to '${{ github.workspace }}/firmware_dir/usr/lib/firmware/qcom/${TARGET}/' successfully!"
98+
cp -rf "${LINUX_FIRMWARE_QCOM_PATH}/"* "${{ github.workspace }}/firmware_dir/usr/lib/firmware/qcom/${LINUX_FIRMWARE}/"
99+
echo "Copied QCOM firmware from '${LINUX_FIRMWARE_QCOM_PATH}' to '${{ github.workspace }}/firmware_dir/usr/lib/firmware/qcom/${LINUX_FIRMWARE}/' successfully!"
99100
fi
100101
101102
- name: List all subdirectories and files in firmware_dir

‎.github/workflows/loading.yml‎

Lines changed: 1 addition & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -8,16 +8,12 @@ on:
88
description: Build matrix
99
value: ${{ jobs.loading.outputs.build_matrix }}
1010

11-
full_matrix:
12-
description: Full Matrix containing lava description
13-
value: ${{ jobs.loading.outputs.full_matrix }}
14-
1511
jobs:
1612
loading:
1713
runs-on: ubuntu-latest
1814
outputs:
1915
build_matrix: ${{ steps.loading.outputs.build_matrix }}
20-
full_matrix: ${{ steps.loading.outputs.full_matrix }}
16+
2117
steps:
2218
- name: Checkout Code
2319
uses: actions/checkout@v4

‎.github/workflows/pre_merge.yml‎

Lines changed: 5 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,7 @@ description: |
55
on pull request events (opened, synchronize, reopened) targeting 'main' or 'development',
66
or manually via workflow_dispatch.
77
Loading: Loads the build matrix from MACHINES.json.
8-
Build: Syncs and Builds code for each machine/firmware pair using sync_build.yml.
8+
Build: Syncs and Builds code for each target using sync_build.yml.
99
Test: Runs LAVA tests using built artifacts via test.yml.
1010
All jobs use reusable workflows and inherit secrets from the caller.
1111
@@ -30,7 +30,6 @@ jobs:
3030
uses: qualcomm/fastrpc/.github/workflows/loading.yml@development
3131
secrets: inherit
3232

33-
# Each job in build and test will run for every {machine, firmware} pair
3433
# The build job will run on the machines specified in the build_matrix output
3534
# from the loading job. It will use the build.yml workflow to build the code.
3635
build:
@@ -42,10 +41,11 @@ jobs:
4241
secrets: inherit
4342
with:
4443
docker_image: fastrpc-image:latest
45-
machine: ${{ matrix.machine }}
46-
firmware: ${{ matrix.firmware }}
44+
deviceTree: ${{ matrix.deviceTree }}
45+
linuxFirmware: ${{ matrix.linuxFirmware }}
46+
hexDSPBinary: ${{ matrix.hexDSPBinary }}
4747

48-
# The test job will run on the machines specified in the full_matrix output
48+
# The test job will run on the machines specified in the build_matrix output
4949
# from the loading job. It will use the test.yml workflow to run tests.
5050
test:
5151
needs: [loading, build]
@@ -54,4 +54,3 @@ jobs:
5454
with:
5555
docker_image: fastrpc-image:latest
5656
build_matrix: ${{ needs.loading.outputs.build_matrix }}
57-
full_matrix: ${{ needs.loading.outputs.full_matrix }}

‎.github/workflows/sync_build.yml‎

Lines changed: 12 additions & 8 deletions
Original file line numberDiff line numberDiff line change
@@ -13,12 +13,16 @@ on:
1313
description: Docker image
1414
type: string
1515
required: true
16-
machine:
17-
description: Target machine name
16+
deviceTree:
17+
description: device Tree name
1818
type: string
1919
required: true
20-
firmware:
21-
description: Firmware identifier
20+
linuxFirmware:
21+
description: Linux Firmware identifier
22+
type: string
23+
required: true
24+
hexDSPBinary:
25+
description: Hexagon DSP binaries identifier
2226
type: string
2327
required: true
2428

@@ -44,8 +48,8 @@ jobs:
4448
id: sync
4549
uses: qualcomm/fastrpc/.github/actions/sync@development
4650
with:
47-
machine: ${{ inputs.machine }}
48-
firmware: ${{ inputs.firmware }}
51+
linuxFirmware: ${{ inputs.linuxFirmware }}
52+
hexDSPBinary: ${{ inputs.hexDSPBinary }}
4953

5054
- name: Build workspace
5155
id: build_workspace
@@ -68,7 +72,7 @@ jobs:
6872
$workspace/kobj/arch/arm64/boot/Image
6973
$workspace/kobj/vmlinux
7074
$workspace/artifacts/ramdisk_fastrpc.gz
71-
$workspace/kobj/arch/arm64/boot/dts/qcom/${{ inputs.machine }}.dtb
75+
$workspace/kobj/arch/arm64/boot/dts/qcom/${{ inputs.deviceTree }}.dtb
7276
EOF
7377
7478
echo "Generated artifact list:"
@@ -81,7 +85,7 @@ jobs:
8185
# local_file points to the list of files to upload
8286
local_file: ${{ steps.sync.outputs.workspace_path }}/artifacts/file_list.txt
8387
mode: multi-upload
84-
machine: ${{ inputs.machine }}
88+
deviceTree: ${{ inputs.deviceTree }}
8589

8690
- name: Clean up
8791
# This step is crucial for self-hosted runners to manage disk space.

0 commit comments

Comments
 (0)