Twister sample // How to EXECUTE test configuration on platform locally

[Dermiste]

Hi all,

That may seem trivial, sorry in advance.
I would like to build and execute a testcase locally on qemu_x86 (install verified), I found a simple one which I run using:
west twister -p qemu_x86 -s tests/ztest/base/testing.ztest.base.verbose_0_userspace

Which gives the following output stating that nothing got executed on platforms:

INFO    - Using Ninja..
INFO    - Zephyr version: zephyr-v3.5.0-2054-geeb6bf7dd998
INFO    - Using 'zephyr' toolchain.
INFO    - Building initial testsuite list...
INFO    - Writing JSON report C:\Users\ABC\zephyrproject\zephyr\twister-out\testplan.json
INFO    - Adding tasks to the queue...
INFO    - Added initial list of jobs to queue
INFO    - Total complete: ←[32m   1/   1←[39m  100%  skipped: ←[39m   0←[39m, failed: ←[39m   0←[39m, error: ←[39m   0←[39m
INFO    - 1 test scenarios (1 test instances) selected, 0 configurations skipped (0 by static filter, 0 at runtime).
INFO    - ←[32m1 of 1←[39m test configurations passed (100.00%), ←[39m0←[39m failed, ←[39m0←[39m errored, 0 skipped with ←[39m0←[39m warnings in 43.63 seconds
INFO    - In total 9 test cases were executed, 0 skipped on 1 out of total 647 platforms (0.15%)
!!! INFO !!!    - 0 test configurations executed on platforms, 1 test configurations were only built.
INFO    - Saving reports...
INFO    - Writing JSON report C:\Users\ABC\zephyrproject\zephyr\twister-out\twister.json
INFO    - Writing xunit report C:\Users\ABC\zephyrproject\zephyr\twister-out\twister.xml...
INFO    - Writing xunit report C:\Users\ABC\zephyrproject\zephyr\twister-out\twister_report.xml...
INFO    - Run completed

Here is an extract of the the testcase.yaml which already has qemu_x86 as integration platform:
common:

tags:
   - test_framework
tests:
 testing.ztest.base:
   type: unit
 [ ... ]
 testing.ztest.base.verbose_0_userspace:
   filter: CONFIG_USERSPACE
   extra_args: CONF_FILE=prj_verbose_0.conf
   tags:
     - userspace
   extra_configs:
     - CONFIG_TEST_USERSPACE=y
   integration_platforms:
     - qemu_x86
  [ ... ]

I then thought running the same command with integration param would help:
west twister -p qemu_x86 -s tests/ztest/base/testing.ztest.base.verbose_0_userspace --integration

I'm running this on windows 11.

But I got the same output, I cannot get the test configuration to actually be executed on platforms.

Am I missing something? Any url, link, couple of words, much appreciated.

Thanks in advance

[digiexchris]

did you ever figure this out? I have the same issue. What's weird is running the app with west runs fine on the qemu_cortex_m3

west build -b qemu_cortex_m3 -t run

but not via twister, it skips them just like it does for you.

[yashi]

For me, it seems to work as exptected.

$ west twister -p qemu_x86 -s tests/ztest/base/testing.ztest.base.verbose_0_userspace
Renaming output directory to /home/yashi/src/zephyr-project/twister-out.3
INFO    - Using Ninja..
INFO    - Zephyr version: zephyr-v3.5.0-4527-g53da5f6f87fa
INFO    - Using 'zephyr' toolchain.
INFO    - Building initial testsuite list...
INFO    - Writing JSON report /home/yashi/src/zephyr-project/twister-out/testplan.json
INFO    - JOBS: 24
INFO    - Adding tasks to the queue...
INFO    - Added initial list of jobs to queue
INFO    - Total complete:    1/   1  100%  skipped:    0, failed:    0, error:    0
INFO    - 1 test scenarios (1 test instances) selected, 0 configurations skipped (0 by static filter, 0 at runtime).
INFO    - 1 of 1 test configurations passed (100.00%), 0 failed, 0 errored, 0 skipped with 0 warnings in 13.56 seconds
INFO    - In total 9 test cases were executed, 0 skipped on 1 out of total 673 platforms (0.15%)
INFO    - 1 test configurations executed on platforms, 0 test configurations were only built.
INFO    - Saving reports...
INFO    - Writing JSON report /home/yashi/src/zephyr-project/twister-out/twister.json
INFO    - Writing xunit report /home/yashi/src/zephyr-project/twister-out/twister.xml...
INFO    - Writing xunit report /home/yashi/src/zephyr-project/twister-out/twister_report.xml...
INFO    - Run completed

Do you have QEmu installed? What if you add -v -v -v to west twister?

[Dermiste]

On windows:
Execution with Qemu (i.e running directly with west) works fine, it's running with twister and capturing the output that does not.

Other platform:
All seem to work as espected

[yashi]

what does your log say? Did you try with -v -v -v?

[Dermiste]

Didn't try but I also found in another ticket that Qemu + Twister support on Windows is not to be featured atm, also found traces of people currently developping that feature.

[yashi]

• QEMU and twister support on windows not possible? #64970

Indeed.