runtimetest: Make TAP output more granular

Granular TAP output makes it easier to see what's being tested (and
what's going wrong).  You could feed this into any TAP harness you
like, but I've chosen prove (it seems popular).  You need a TAP
harness looking for test failures, because runtimetest now returns
zero when it successfully runs the tests (regardless of whether the
tests all pass).

With this commit, runtimetest will exit zero when it can successfully
*run the tests* (regardless of whether all the tests pass).  A TAP
harness like prove should be used to consume the output TAP to decide
if validation succeeded or not.  To avoid confusing the TAP consumer,
all of the non-TAP logging goes to stderr (and not stdout).

Signed-off-by: W. Trevor King <[email protected]>

Author: wking

README.md

@@ -29,25 +29,25 @@ INFO[0000] Bundle validation succeeded.
 
 ## Testing OCI runtimes
 
-```sh
+You can run [`test_runtime.sh`][test_runtime.sh] with any [TAP consumer][tap-consumer].
+For example, with [prove][]:
+
+```
 $ make
 $ sudo make install
-$ sudo ./test_runtime.sh -r runc
------------------------------------------------------------------------------------
-VALIDATING RUNTIME: runc
------------------------------------------------------------------------------------
-validating container process
-validating capabilities
-validating hostname
-validating rlimits
-validating sysctls
-Runtime runc passed validation
+$ sudo prove ./test_runtime.sh -r /usr/bin/runc
+./test_runtime.sh .. ok
+All tests successful.
+Files=1, Tests=90,  0 wallclock secs ( 0.02 usr  0.00 sys +  0.04 cusr  0.00 csys =  0.06 CPU)
+Result: PASS
 ```
 
 [bundle]: https://github.com/opencontainers/runtime-spec/blob/master/bundle.md
 [config.json]: https://github.com/opencontainers/runtime-spec/blob/master/config.md
+[prove]: http://perldoc.perl.org/prove.html
 [runC]: https://github.com/opencontainers/runc
 [runtime-spec]: https://github.com/opencontainers/runtime-spec
+[tap-consumer]: https://testanything.org/consumers.html
 
 [generate.1]: man/oci-runtime-tool-generate.1.md
 [validate.1]: man/oci-runtime-tool-validate.1.md