run-test.py issue with UNIX coverage

[yf13]

I am getting familiar by adding cmake support for UNIX port, so far both standard and coverage variants basically work on Ubuntu 22.04 with stock gcc 11.4.0-1ubuntu1 toolchain.

I got one weird issue when checking my cmake artifact with run-tests,py. It seems test resullts are different for different number of jobs:

$ cd micropython/tests
$ BUILD=/tmp/aaa MICROPYTHON=${BUILD}/firmware.elf ./run-tests.py
platform=linux arch=x64
...
983 tests performed (29469 individual testcases)
983 tests passed
17 tests skipped: nanbox_smallint machine_disable_irq machine_i2s_rate machine_rtc machine_soft_timer machine_spi_rate machine_uart_irq_txidle machine_uart_tx select_ipoll sys_settrace_features sys_settrace_generator sys_settrace_loop ffi_int_base ffi_int_long32 ffi_int_long64 ffi_int_types asyncio_as_uasyncio

$ BUILD=/tmp/aaa MICROPY_MICROPYTHON=${BUILD}/firmware.elf ./run-tests.py -j2
platform=linux arch=x64
...
984 tests performed (29475 individual testcases)
984 tests passed
16 tests skipped: nanbox_smallint machine_disable_irq machine_i2s_rate machine_rtc machine_soft_timer machine_spi_rate machine_uart_irq_txidle machine_uart_tx asyncio_as_uasyncio sys_settrace_features sys_settrace_generator sys_settrace_loop ffi_int_base ffi_int_long32 ffi_int_long64 ffi_int_types

So I can only get same test results like make artifact does only when using multiple jobs. The attched cmake artifact can be used on Ubuntu 22.04 to review this issue.

Can I get some some hints about this?

Updates

Exception log in select_ipoll.py tells that is an [Errno 98] EADDRINUSE error. As the same UDP port are used in multiple tests, maybe some earlier tests are not closing it timely?

[yf13]

So it was caused by the test scripts, not by run-tests.py.