Content of character vector can trip up expect_snapshot_value()

[jennybc]

Discovered while converting reprex to testthat 3e.

Here's an example of a specific character vector that is difficult to use with expect_snapshot_value(). (The reprex content doesn't matter -- the problem appears to be the backticks associated with fenced code blocks.)

Put this inside a package's tests and run test():

• The default (no style specified) is unable to save the snapshot (other styles can save snapshot).
• The default also reports Warning (test-foo.R:12:1): (code run outside of test_that()), which seems wrong.

f <- function() {
  c(
    "``` r",
    "utils::getParseData(parse(text = 'a'))",
    "#>   line1 col1 line2 col2 id parent  token terminal text",
    "#> 1     1    1     1    1  1      3 SYMBOL     TRUE    a",
    "#> 3     1    1     1    1  3      0   expr    FALSE",
    "```"
  )
}

test_that("no style specified", {
  expect_snapshot_value(f())
})

test_that("json2", {
  expect_snapshot_value(f(), style = "json2")
})

test_that("deparse", {
  expect_snapshot_value(f(), style = "deparse")
})

test_that("serialize", {
  expect_snapshot_value(f(), style = "serialize")
})

On the first run (so no snapshots exist yet), I see:

Error (test-foo.R:13:3): no style specified
Error: Serialization round-trip is not symmetric.

`value` is a character vector ('``` r', 'utils::getParseData(parse(text = \'a\'))', '#>   line1 col1 line2 col2 id parent  token terminal text', '#> 1     1    1     1    1  1      3 SYMBOL     TRUE    a', '#> 3     1    1     1    1  3      0   expr    FALSE', ...)
`roundtrip` is a list

i You may need to consider serialization `style`
Backtrace:
 1. testthat::expect_snapshot_value(f()) test-foo.R:13:2
 2. testthat:::expect_snapshot_helper(...)
 3. snapshotter$take_snapshot(val, save = save, load = load, ...)
 4. testthat:::check_roundtrip(value, load(save(value)))

Warning (test-foo.R:12:1): (code run outside of `test_that()`)
Snapshots reset after error/skip

Warning (test-foo.R:17:3): json2
Adding new snapshot:
{
  "type": "character",
  "attributes": {},
  "value": ["``` r", "utils::getParseData(parse(text = 'a'))", "#>   line1 col1 line2 col2 id parent  token terminal text", "#> 1     1    1     1    1  1      3 SYMBOL     TRUE    a", "#> 3     1    1     1    1  3      0   expr    FALSE", "```"]
}

Warning (test-foo.R:21:3): deparse
Adding new snapshot:
c("``` r", "utils::getParseData(parse(text = 'a'))", "#>   line1 col1 line2 col2 id parent  token terminal text", 
"#> 1     1    1     1    1  1      3 SYMBOL     TRUE    a", 
"#> 3     1    1     1    1  3      0   expr    FALSE", "```"
)

Warning (test-foo.R:25:3): serialize
Adding new snapshot:
WAoAAAACAAQAAgACAwAAAAAQAAAABgAEAAkAAAAFYGBgIHIABAAJAAAAJnV0aWxzOjpnZXRQ
YXJzZURhdGEocGFyc2UodGV4dCA9ICdhJykpAAQACQAAADkjPiAgIGxpbmUxIGNvbDEgbGlu
ZTIgY29sMiBpZCBwYXJlbnQgIHRva2VuIHRlcm1pbmFsIHRleHQABAAJAAAAOSM+IDEgICAg
IDEgICAgMSAgICAgMSAgICAxICAxICAgICAgMyBTWU1CT0wgICAgIFRSVUUgICAgYQAEAAkA
AAA0Iz4gMyAgICAgMSAgICAxICAgICAxICAgIDEgIDMgICAgICAwICAgZXhwciAgICBGQUxT
RQAEAAkAAAADYGBg

I understand that certain nontrivial objects might need different methods of saving, hence the existence of style. But I wouldn't expect a character vector to ever need special treatment.

[hadley]

Minimal reprex:

f <- function() {
  c("a", "b", "c")
}


save <- jsonlite::toJSON(f(), auto_unbox = TRUE, pretty = TRUE)
load <- jsonlite::fromJSON(save, simplifyVector = FALSE)

testthat:::check_roundtrip(load, f())
#> Error: Serialization round-trip is not symmetric.
#> 
#> `value` is a list
#> `roundtrip` is a character vector ('a', 'b', 'c')
#> 
#> ℹ You may need to consider serialization `style`

^{Created on 2021-02-07 by the reprex package (v1.0.0)}

[hadley]

Same problem with any vector. Should simplifyVector be TRUE?

[hadley]

I don't currently see how we can get the desired behaviour using fromJSON() and toJSON(), so I'm going to leave this for now:

json <- jsonlite::toJSON(list("a", 1.5), auto_unbox = TRUE)
cat(json)
#> ["a",1.5]
jsonlite::fromJSON(json)
#> [1] "a"   "1.5"

json <- jsonlite::toJSON(c("a", "b"), auto_unbox = TRUE)
cat(json)
#> ["a","b"]
jsonlite::fromJSON(json)
#> [1] "a" "b"

^{Created on 2021-02-11 by the reprex package (v1.0.0)}

[krlmlr]

Would you consider supporting custom load/save function pairs in expect_snapshot_value()? We could allow a list for the style argument:

expect_snapshot_value(expr, style = list(load = function(x) ..., save = function(x) ...)

This would give users more flexibility and move some of the burden away from testthat.

[martinju]

Any progress on this issue?

I'm experiencing the same issue when the output of my function is a data.table. Convering to a data.frame first seems to work, but the suggestion from @krlmlr where this conversion could be passed to style seems like a cleaner solution.

[hadley]

Coming back to this after a few years, I don't think it's going to be worth fixing, mostly because I don't think we can do much to improve the situation without introducing a new serialisation method. And there's an simple existing workaround: first convert your object to something that serializes naturally to JSON.