testthat 3.3.1

• Fixed support for shinytest2::AppDriver$expect_values() screenshot snapshot failing on CI (#2293, #2288).

testthat 3.3.0

Lifecycle changes

• testthat now requires R 4.1.
• expect_snapshot(binary), soft deprecated in 3.0.3 (2021-06-16), is now fully deprecated.
• is_null()/matches(), deprecated in 2.0.0 (2017-12-19), and is_true()/is_false(), deprecated in 2.1.0 (2019-04-23), have been removed (#2109).
• local_mock() and with_mock(), deprecated in 3.0.0 (2020-10-31), are now defunct. They technique that made them work is no longer permitted in R 4.5.0.
• test_files(wrap), deprecated in 3.0.0 (2020-10-31) has now been removed.

Expectations and tests

• All expect_ functions have had their failure messages rewritten. They now all state what was expected, what was actually received, and, if possible, clearly illustrate the difference (#2142). They also consistently check that their inputs are the correct type (#1754). They consistently return the value of the first argument, regardless of whether the expectation succeeds or fails (the only exception is expect_message() and friends which return the condition). This shouldn't affect existing tests, but will make failures clearer when you chain together multiple expectations (#2246).
• It's now much easier to create custom expectations, thanks to an overhauled vignette("custom-expectations") (#2113, #2132, #2072), a new pass() function to use in place of succeed() (#2113), and revisions to how expectation() works (#2125). expect_no_failures() and expect_no_successes() are now deprecated as now expect_success() also tests for the absence of failures and expect_failure() tests for the absence of successes.
• Tests (i.e. test_that(), describe(), and it() calls) can now be arbitrarily nested with a shared stack of descriptions so that the failures can show the full path through all subtests. Skips in subtests are now correctly scoped (#2007), and each subtest and will skip if and only if it and its subtests don't contain any expectations.
• On CRAN, test_that() will automatically skips if a package is not installed (#1585). Practically, this means that you no longer need to check that suggested packages are installed. (We already didn't check the installation of suggested packages in the tidyverse because we think it has limited payoff, but other styles advise differently.)
• When running a test interactively, testthat now reports the number of successes and failures. If you're using nested tests, the results should be more useful and you will no longer see duplicated failures (#2063, #2188).
• New expect_all_equal(), expect_all_true(), and expect_all_false() check that every element of a vector has the same value (#1836, #2235), giving better error messages than expect_true(all(...)).
• New expect_disjoint() expects values to to be absent (@stibu81, #1851).
• New expect_r6_class() expects an R6 objects (#2030).
• New expect_shape() expects a specific shape (i.e., nrow(), ncol(), or dim()) (#1423, @MichaelChirico).

Other new features

• New extract_test() function to extract a reprex from a failing expectation. tests run in R CMD check will use this to automatically create a reprex in the _problems/ directory for each failing expectation. You can turn this behaviour off by setting TESTTHAT_PROBLEMS=false (#2263).
• New local_mocked_s3_method(), local_mocked_s4_method(), and local_mocked_r6_class() allow you to mock S3 and S4 methods and R6 classes (#1892, #1916)
• New local_on_cran(TRUE) allows you to simulate how your tests will run on CRAN (#2112).
• test_dir(), test_file(), test_package(), test_check(), test_local(), source_file() gain a shuffle argument that uses sample() to randomly reorder the top-level expressions in each test file (#1942). This random reordering surfaces dependencies between tests and code outside of any test, as well as dependencies between tests. This helps you find and eliminate unintentional dependencies.
• try_again() is now publicised, making it easier to test flaky code. We changed the first argument to represent the number of retries, not tries (#2050).
• New skip_unless_r() skip tests on unsuitable versions of R. It has a convenient sytnax so you can use, e.g., skip_unless_r(">= 4.1.0") to skip tests that require ...names() (@MichaelChirico, #2022).
• New snapshot_reject() rejects all modified snapshots by deleting the .new variants (#1923), and snapshot_download_gh() makes it easy to get snapshots off GitHub and into your local package (#1779).
• expect_snapshot() and friends will now fail when creating a new snapshot on CI. This is usually a signal that you've forgotten to run it locally before committing (#1461).
• New SlowReporter makes it easier to find the slowest tests in your package. You can run it with devtools::test(reporter = "slow") (#1466).
• New vignette("mocking") explains mocking in detail (#1265).
• New vignette("challenging-functions") provides an index to other documentation organised by various challenges (#1265).

Minor improvements and bug fixes

• Interrupting a test now prints the test name. This makes it easier to tell where a very slow test might be hanging (#1464).
• Fixed an issue preventing compilation from succeeding due to deprecation / removal of std::uncaught_exception() (@kevinushey, #2047).
• expect_lt(), expect_gt(), and friends have a refined display that is more likely to display the correct number of digits and shows you the actual values compared (#2006). They have a better display for non-numeric data (@stibu81, #2268).
• expect_matches() failures should be a little easier to read (#2135, #2181).
• expect_no_*() now executes the entire code block, rather than stopping at the first message or warning (#1991).
• expect_named() now gives more informative errors (#2091).
• expect_s4_class() now supports unquoting (@stibu81, #2064).
• expect_snapshot() no longer loses the last snapshot if file is missing the final newline (#2092). It's easy to accidentally remove this because there are two trailing new lines in snapshot files and many editors will automatically remove one if you touch the file.
• expect_snapshot() on reports how to resolve failures once when running inside R CMD check.
• expect_snapshot() no longer skips on CRAN, as that skips the rest of the test. Instead it just returns, neither succeeding nor failing (#1585).
• expect_snapshot() and expect_snapshot_file() hints now include the path to the package, if it's not the current working directory (#1577).
• expect_snapshot() gives a more informative backtrace when the code inside the snapshot errors (#2277).
• expect_snapshot_file() now clearly errors if the path doesn't exist (#2191), or has been used alreaday (#1592). It now considers .json to be a text file (#1593), and shows differences for text files in the console (#1593).
• expect_snapshot_value() can now handle expressions that generate - (#1678) or zero length atomic vectors (#2042).
• expect_vector() fails, instead of erroring, if object is not a vector (@plietar, #2224).
• JunitReporter() no longer fails with "no applicable method for xml_add_child" for warnings outside of tests (#1913). Additionally, warnings now save their backtraces and JunitReporter() strips ANSI escapes in more places (#1852, #2032).
• local_edition() now gives a useful error for bad values (#1547).
• ParallelProgressReporter now respects max_failures (#1162), and test files with syntax errors are no longer silently ignored (#1360).
• test_file(desc = ...) (i.e. test filtering) no longer loses snapshot results (#2066), works with it(), and can now be character vector in order to recursively filter subtests (#2118).
• test_that() no longer warns about the absence of {} since it no longer seems to be necessary.
• set_state_inspector() gains tolerance argument and ignores minor FP differences by default (@mcol, #2237).
• skip_on_os() gains an option "emscripten" of the os argument to skip tests on Emscripten (@eitsupi, #2103).
• snapshot_accept(test) now works when the test file name contains . (#1669).
• snapshot_review() includes a reject button and only displays the file navigation and the skip button if there are multiple files to review (#2025). It now passes ... on to shiny::runApp() (#1928).

testthat 3.2.3

• Fixed an issue where expect_no_error(1) was failing (#2037).

• Fixed an issue where calling skip() outside of an active test could
  cause an unexpected error (@kevinushey, #2039).

testthat 3.2.2

New expectations

• expect_s7_class() tests if an object is an S7 class (#1580).

• expect_no_failure(), expect_no_success() and expect_snapshot_failure()
  provide more options for testing expectations.

Bug fixes and minor improvements

• testthat now requires waldo 0.6.0 or later to access the latest features
  (#1955).

• expect_condition() and related functions now include the class of the
  expected condition in the failure message, if provided (#1987).

• expect_error() and friends now error if you supply ... but not pattern
  (#1932). They no longer give an uninformative error if they fail inside
  a magrittr pipe (#1994).

• expect_no_*() expectations no longer incorrectly emit a passing test result
  if they in fact fail (#1997).

• expect_setequal() correctly identifies what is missing where (#1962).

• expect_snapshot() now strips line breaks in test descriptions
  (@LDSamson, #1900), and errors when called from a test_that() that has an
  empty description (@kevinushey, #1980).

• expect_true() and expect_false() give better errors if actual isn't a
  vector (#1996).

• expect_visible() and expect_invisible() have clearer failure messages
  (#1966).

• local_reproducible_output() (used in test_that() blocks) now sets
  LANGUAGE to "C" instead of "en" to disable translations,
  avoiding warnings on some platforms (#1925).

• skip_if_not_installed() generates a clearer message that sorts better
  (@MichaelChirico, #1959).

• with_mock() and local_mock() have been unconditionally deprecated as
  they will no longer work in future versions of R (#1999).

testthat 3.2.1

• Fix incorrect format string detected by latest R-devel. Fix thanks to
  Tomas Kalibera.

• expect_snapshot() handles unexpected errors like errors outside of
  snapshots, i.e. they terminate the entire test and get a traceback (#1906).

• JunitReporter() now uses ensures numeric values are saved the xml file
  with . as decimal separator. (@maksymiuks, #1660)

• local_mocked_bindings() can now mock any object, not just functions
  (#1896).

• skip_if_offline() now uses captive.apple.com by default. This is the
  hostname that Apple devices use to check that they're online so it should
  have a higher reliability than r-project.org (@jdblischak, #1890).

• test_file(desc = ) will now find describe() tests as well as test_that()
  tests (#1903).

testthat 3.2.0

Lifecycle changes

• is_informative_error() and the wrap argument to test_dir() and friends
  are now defunct.

• expect_no_error(), expect_no_warning(), expect_no_message(),
  expect_no_condition(), local_mocked_bindings(), and
  with_mocked_bindings() are now stable, not experimental.

New features

• All packages, regardless of whether or not they use rlang 1.0.0, now
  use the new snapshot display for errors, warnings, and messages (#1856).
  This no longer shows the class name, instead focussing on a display that
  more closely mimics what you'll see interactively, including showing the
  error call.

• testthat uses an improved algorithm for finding the srcref associated with
  an expectation/error/warning/skip. It now looks for the most recent call
  that has known source and is found inside the test_that() call. This
  generally gives more specific locations than the previous approach and
  gives much better locations if an error occurs in an exit handler.

Minor features and bug fixes

• Helpers are no longer run twice.

• expect_setequal() correctly displays results when only one of actual and
  expected is missing elements (#1835).

• expect_snapshot() and friends no longer create a temporary file on every
  invocation.

• expect_snapshot_file() now generates clickable links to review changes
  (#1821).

• expect_snapshot_value() has an improved error if the object can't be
  safely serialized using the specified style (#1771).

• options(rlang_interactive = TRUE) no longer causes skip_on_cran() to
  not run on CRAN (#1868).

• skip_if_offline() now errors if you don't have curl installed (#1854).

• StopReporter gains the ability to suppress praise when a test passes.

• ProgressReporter now uses is a two characters wide skip column in order
  to have a consistent width when 10 or more tests are skipped in a single file
  (@mgirlich, #1844).

• test_file() gains a desc argument which allows you to run a single
  test from a file (#1776).

testthat 3.1.10

• Fix for upcoming R-devel release.

• testthat now sets the _R_CHECK_BROWSER_NONINTERACTIVE_ environment variable
  when running tests. This should ensure that left-over browser() statements
  will trigger an error if encountered while running tests. This functionality
  is only enabled with R (>= 4.3.0). (#1825)

testthat 3.1.9

• New expect_contains() and expect_in() that works similarly to
  expect_true(all(expected %in% object)) or
  expect_true(all(object %in% expected)) but give more informative failure
  messages (#1346).

• New is_snapshot() returns TRUE if code is running inside a snapshot test
  (#1796) and is_checking() returns TRUE if test is running inside of
  R CMD check (#1795)

• ProgressReporter only reports the run time of test files that take longer
  than 1s, rather than 0.1s. (#1806) and re-displays all failures at the end
  of the results. Skips are now only shown at the end of reporter summaries,
  not as tests are run. This makes them less intrusive in interactive tests
  while still allowing you to verify that the correct tests are skipped (#1801).
  When using parallel tests, links to failed tests (#1787) and links to
  accept/review snapshot (#1802) now work.

• set_state_inspector() allows to to register a function that's called
  before and after every test, reporting on any differences. This
  is very useful for detecting if any of your tests have made changes to
  global state (like options, env vars, or connections) (#1674). This
  function was inspired by renv's testing infrastructure.

• skip_on_cran() no longer skips (errors) when run interactively.

• teardown_env() works in more cases.

• All packages, regardless of whether or not they use rlang, now
  use the new snapshot display for errors, warnings, and messages.

• testthat no longer truncates tracebacks and uses rlang's default tree
  display.

testthat 3.1.8

• expect_snapshot() differences no longer use quotes.

• expect_error(), expect_warning(), and expect_message() now correctly
  enforce that the condition is of the expected base class (e.g. error,
  warning, messsage) even when the class argument is used (#1168).

• it() now calls local_test_context() so that it behaves more
  similarly to test_that() (#1731), and is now exported so that you
  can more easily run BDD tests interactively (#1587)

• skip_on_bioc() now uses the documented environment variable
  (IS_BIOC_BUILD_MACHINE) (#1712).

• source_file(), which is used by various parts of the helper and
  setup/teardown machinery, now reports the file name in the case of
  errors (#1704).

• test_path() now works when called within helper files (#1562).

• New vignette("special-files") describes the various special files
  that testthat uses (#1638).

• with_mocked_bindings() and local_mocked_bindings() now also bind in the
  imports namespace and can mock S3 methods. These changes make them good
  substitutes for the deprecated functions with_mock() and local_mock(), so
  those older functions now recommend switching to the newer equivalents
  instead of using the mockr or mockery packages.

testthat 3.1.7

• expect_setequal() gives more actionable feedback (#1657).

• expect_snapshot() no longer elides new lines when run interactively (#1726).

• Experimental new with_mocked_bindings() and local_mocked_bindings()
  (#1739).