tap reporter with skipped test reports wrong number of tests

[mgreter]

The tap reporter does not count or print any skipped tests, although they are accounted for in the test number. E.g. first line prints "1..59", but will only print 56 oks, since 3 tests are skipped.

Here an example by using https://github.com/sass/sass-spec (needs sassc installed)

ruby sass-spec.rb -r spec/basic --impl libsass -V 3.5 -c sassc --tap | tapout tap
1..59
ok 1 - test__sass-spec/spec/basic/33_ambiguous_imports
...
ok 56 - test__sass-spec/spec/basic/57_function_exists

I get it to pass correctly by patching tap_reporter.rb:

    def todo(entry)
      super(entry)
      @i += 1
      puts "ok #{@i} # SKIP #{entry['label']}"
    end

On another note tapout does not seem to distinguish todo and skipped tests, which is IMO not correct according to https://testanything.org/tap-specification.html. Todo tests are expected to fail but should eventually pass (they are considered bugs) and skipped tests might be skipped because they e.g. don't apply to the current version being tested against or are only meant for certain operating systems. Edit: seems the underlying minitest framework does not have the concept of todo tests, so it's probably not relevant.

[trans]

Thanks for reporting this. I'll look into it when I find the time.