Kill the nodes after timeout

[michalkucharczyk]

Is your feature request related to a problem? Please describe.

Sometimes the network cannot be terminated because the node process goes into some undesired state and is not terminated on its own.

Example given in the following output ([polkadot] <defunct>):

miszka    173917  0.0  0.0  40192 22428 pts/67   Ss   Jul07   0:00  \_ -bash
miszka    791163  0.0  0.0  12640  3840 pts/67   S+   Jul08   0:00  |   \_ /bin/bash -x ./run-test-tx-proto
miszka    828091  0.0  0.4 344128 267084 pts/67  S+   Jul08   0:01  |       \_ /home/miszka/.rustup/toolchains/1.84.1-x86_64-unknown-linux-gnu/bin/cargo test --release --test integration -- --ignored tx_proto_test
miszka    828092  0.0  0.0  12640  2124 pts/67   S+   Jul08   0:00  |           \_ /bin/bash -x ./run-test-tx-proto
miszka    828093  0.0  0.0  10916  2048 pts/67   S+   Jul08   0:00  |           |   \_ tee /home/miszka/parity/14-txpool-forks/polkadot-sdk/substrate/client/transaction-pool/test-tx-proto/run_250708_183615/test_ts8000_nc3/cargo_log
miszka    828137  0.1  0.3 9160440 254504 pts/67 Sl+  Jul08   0:49  |           \_ /home/miszka/parity/14-txpool-forks/polkadot-sdk/target/release/deps/integration-f1d0f575363e9fe5 --ignored tx_proto_test
miszka    829735  0.0  0.0      0     0 pts/67   Z+   Jul08   0:02  |               \_ [polkadot] <defunct>
miszka    829803  0.8  0.9 407475564 637136 pts/67 Sl+ Jul08   4:54  |               \_ polkadot --chain /home/miszka/parity/14-txpool-forks/polkadot-sdk/substrate/client/transaction-pool/test-tx-proto/run_250708_183615/test_ts8000_nc3/test_20250708_215757/bob/cfg/rococo-lo
miszka    831384  0.0  0.0 221568 17440 pts/67   S+   Jul08   0:00  |               |   \_ /home/miszka/parity/14-txpool-forks/polkadot-sdk-2506/target/release/polkadot-prepare-worker prepare-worker --node-impl-version 1.19.0 --socket-path /tmp/pvf-host-prepare-8sgkuN2GiM -
miszka    831395  0.0  0.0  48604 35364 pts/67   S+   Jul08   0:00  |               |   \_ /home/miszka/parity/14-txpool-forks/polkadot-sdk-2506/target/release/polkadot-execute-worker execute-worker --node-impl-version 1.19.0 --socket-path /tmp/pvf-host-execute-cFOUD9oQns -
miszka    829805  0.8  0.9 407433828 621200 pts/67 Sl+ Jul08   4:58  |               \_ polkadot --chain /home/miszka/parity/14-txpool-forks/polkadot-sdk/substrate/client/transaction-pool/test-tx-proto/run_250708_183615/test_ts8000_nc3/test_20250708_215757/charlie/cfg/rococ
miszka    831203  0.0  0.0 221564 18128 pts/67   S+   Jul08   0:00  |               |   \_ /home/miszka/parity/14-txpool-forks/polkadot-sdk-2506/target/release/polkadot-prepare-worker prepare-worker --node-impl-version 1.19.0 --socket-path /tmp/pvf-host-prepare-zs8zDjcFas -
miszka    831219  0.0  0.0  48604 35108 pts/67   S+   Jul08   0:00  |               |   \_ /home/miszka/parity/14-txpool-forks/polkadot-sdk-2506/target/release/polkadot-execute-worker execute-worker --node-impl-version 1.19.0 --socket-path /tmp/pvf-host-execute-bctwB3jRog -
miszka    829887  1.4  1.0 812450164 722028 pts/67 Sl+ Jul08   8:35  |               \_ polkadot-parachain --chain /home/miszka/parity/14-txpool-forks/polkadot-sdk/substrate/client/transaction-pool/test-tx-proto/run_250708_183615/test_ts8000_nc3/test_20250708_215757/node01/
miszka    829959  1.3  1.1 812474228 731676 pts/67 Sl+ Jul08   7:54  |               \_ polkadot-parachain --chain /home/miszka/parity/14-txpool-forks/polkadot-sdk/substrate/client/transaction-pool/test-tx-proto/run_250708_183615/test_ts8000_nc3/test_20250708_215757/node02/

Describe the solution you'd like

Maybe there should be a fuse in zombienet-sdk that simply kills (sends KILL signal) to all the nodes if they don't terminate on its own after some timeout?

Describe alternatives you've considered

none

Additional context

For reference here is the test script that hung on my machine.

[pepoviola]

Hi @michalkucharczyk, thanks for reporting. Did you know if this was a node that crash or something similar? I will try to reproduce to fix it.
Thx!