Skip to content

Commit be221c6

Browse files
authored
Adding Models to Test Matrix for Zephyr CI Test (#13193)
### Summary Previously, only the example Add model was included in the matrix of models that the Zephyr CI job tests. The Add model isn't very representative of real-world models, and may give a false sense of security on the functionality of incorporating ExecuTorch with Zephyr. This PR adds the simple `Softmax` and `Mobilenet V2` models to the test matrix. While the `Softmax` model is like the `Add` model (a toy example), `Mobilenet V2` includes using an image from the `imagenet` validation set, and verifies that the top predicted label matches expectation (i.e. predict "Samoyed" as best label). All three of these tests show example of this flow can be extend further for other models. Thus this PR integrates these test to ensure compatibility between ExecuTorch and Zephyr is maintained. ### Test plan Verified that these models work with the Zephyr ecosystem manually in a separate docker image.
1 parent a8fe653 commit be221c6

File tree

1 file changed

+14
-6
lines changed

1 file changed

+14
-6
lines changed

.github/workflows/trunk.yml

Lines changed: 14 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -60,7 +60,7 @@ jobs:
6060
uses: pytorch/test-infra/.github/workflows/linux_job.yml@main
6161
strategy:
6262
matrix:
63-
model: [add]
63+
model: [add, softmax, mv2]
6464
fail-fast: false
6565
with:
6666
runner: linux.2xlarge
@@ -72,6 +72,16 @@ jobs:
7272
MODEL_NAME=${{ matrix.model }}
7373
CONDA_ENV=$(conda env list --json | jq -r ".envs | .[-1]")
7474
conda activate "${CONDA_ENV}"
75+
if [[ ${{ matrix.model}} == "add" ]]; then
76+
SIM_LIMIT_SEC=60
77+
elif [[ ${{ matrix.model}} == "softmax" ]]; then
78+
SIM_LIMIT_SEC=60
79+
elif [[ ${{ matrix.model}} == "mv2" ]]; then
80+
SIM_LIMIT_SEC=5000
81+
else
82+
echo "Failed unsupported model selection ${{ matrix.model }}"
83+
exit 1
84+
fi
7585
7686
source .ci/scripts/utils.sh
7787
source .ci/scripts/zephyr-utils.sh
@@ -118,24 +128,22 @@ jobs:
118128
-C mps3_board.uart0.out_file='sim.out' \
119129
-C cpu0.CFGITCMSZ=15 \
120130
-C cpu0.CFGDTCMSZ=15 \
121-
--simlimit 120
131+
--simlimit ${SIM_LIMIT_SEC}
122132
123133
# Disable exit on error
124134
set +e
125135
# Report failure if any of the ouptut verification checks fail
126-
# store 0 if found (failure), 1 if not (success)
127136
grep -qF "ERROR" sim.out
128-
exit_status=$?
137+
exit_status=$? #store 0 if found (failure), 1 if not (success)
129138
if [[ "$exit_status" -eq "0" ]]; then
130139
cat sim.out
131140
set -e
132141
exit 1
133142
fi
134143
135144
# Report fail if simulation does not complete successfully
136-
# store 0 if found (success), 1 if not (failure)
137145
grep -qF "SUCCESS: Program complete, exiting." sim.out
138-
exit_status=$?
146+
exit_status=$? #store 0 if found (success), 1 if not (failure)
139147
if [[ "$exit_status" -eq "1" ]]; then
140148
cat sim.out
141149
set -e

0 commit comments

Comments
 (0)