hedgehog swallows failing expect and only displays message of last (potentially successful) expect

[gregmuellegger]

Given the following minimal example, I do not get information in the output about the obvious error in expect_equal(n, n + 1) but only see as expected in the output:

library(testthat)
library(hedgehog)

test_that("Demo: shows 'as expected' as error message", {
    forall(
        gen.element(1:10),
        function (n) {
            expect_equal(n, n + 1)
            
            # The last expect is shown as "error", in this case the message is "as expected"
            # Information of the above failing expect is swallowed.
            expect_equal(n, n)
        }
    )
})

The output:

-- FAILURE (???): Demo: shows 'as expected' as error message -------------------
Falsifiable after 1 tests, and 1 shrinks
As expected 
Counterexample:
[1] 1

(this can be directly reproduced in e.g. https://rdrr.io/snippets/)

The last expect succeeds and this is where the strange "failure" message of As expected presumably comes from.

If we switch the expects, I get again the result of the last expect, which in the example below is at least actually failing:

library(testthat)
library(hedgehog)

test_that("Demo: shows 'as expected' as error message", {
    forall(
        gen.element(1:10),
        function (n) {
            expect_equal(n, n)
            expect_equal(n, n + 1)
            expect_equal(n, n + 2)
        }
    )
})

output:

-- FAILURE (???): Demo: shows 'as expected' as error message -------------------
Falsifiable after 1 tests, and 1 shrinks
`n` not equal to n + 2.
1/1 mismatches
[1] 1 - 3 == -2 
Counterexample:
[1] 1

However I only see one failing expect, even though there are two failing in the test.

Expected behaviour

Provide information about every failing expect, like testthat does by default, and don't show information about succeeding expects:

library(testthat)

test_that("Demo: testthat shows all failing expects", {
    n <- 1

    expect_equal(n, n + 1)
    expect_equal(n, n + 2)
    expect_equal(n, n)
})

output:

-- FAILURE (???): Demo: shows 'as expected' as error message -------------------
`n` not equal to n + 1.
1/1 mismatches
[1] 1 - 2 == -1

-- FAILURE (???): Demo: shows 'as expected' as error message -------------------
`n` not equal to n + 2.
1/1 mismatches
[1] 1 - 3 == -2

[gregmuellegger]

I'm proposing a fix for this in #20 which is functionally working.

Since I'm rather new to R, I'm not sure about which other maintenance tasks need to be done before considering this PR "done/mergable".

I'm open to any change requests, or requests for more docs/changelog/etc in order to incorporate this into r-hedgehog.