Vignette on challenging tests

.Rbuildignore

@@ -25,3 +25,4 @@
 ^\.git-blame-ignore-rev$
 ^CLAUDE\.md$
 ^\.claude$
+^vignettes/articles$

CLAUDE.md

@@ -6,7 +6,7 @@ This file provides guidance to Claude Code (claude.ai/code) when working with co
 
 testthat is R's most popular unit testing framework, used by thousands of CRAN packages. It provides functions to make testing R code as fun and addictive as possible, with clear expectations, visual progress indicators, and seamless integration with R package development workflows.
 
-## Key Development Commands
+## Key development commands
 
 General advice:
 * When running R from the console, always run it with `--quiet --vanilla`
@@ -25,10 +25,11 @@ General advice:
 
 ### Documentation
 
-- Always run `devtools::document()` after changing any roxygen2 docs.
+- Run `devtools::document()` after changing any roxygen2 docs.
 - Every user facing function should be exported and have roxygen2 documentation.
 - Whenever you add a new documentation file, make sure to also add the topic name to `_pkgdown.yml`. 
 - Run `pkgdown::check_pkgdown()` to check that all topics are included in the reference index.
+- Use sentence case for all headings
 
 ## Core Architecture
 

_pkgdown.yml

@@ -83,6 +83,25 @@ reference:
   - ends_with("Reporter")
   - -Reporter
 
+articles:
+- title: Setup and configuration
+  navbar: Setup
+  contents:
+  - third-edition
+  - parallel
+  - special-files
+  - custom-expectation
+
+- title: Testing techniques
+  navbar: Techniques
+  contents:
+  - challenging-tests
+  - mocking
+  - nested-tests
+  - skipping
+  - snapshotting
+  - test-fixtures
+
 news:
   releases:
   - text: "Version 3.2.0"

vignettes/challenging-tests.Rmd

@@ -0,0 +1,145 @@
+---
+title: "Testing challenging functions"
+output: rmarkdown::html_vignette
+vignette: >
+  %\VignetteIndexEntry{Testing challenging functions}
+  %\VignetteEngine{knitr::rmarkdown}
+  %\VignetteEncoding{UTF-8}
+---
+
+```{r}
+#| include: false
+library(testthat)
+knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
+
+# Pretend we're snapshotting
+snapper <- local_snapshotter(fail_on_new = FALSE)
+snapper$start_file("snapshotting.Rmd", "test")
+
+# Pretend we're testing testthat so we can use mocking
+Sys.setenv(TESTTHAT_PKG = "testthat")
+```
+
+This vignette is a quick reference guide for testing challenging functions. It's organised by the problem, rather than technique used to solve it, so you can quickly skim the whole vignette, spot the problem your facing, then learn more about useful tools for solving it.
+
+## Options and environment variables
+
+If your function depends on options or environment variables, first try refactoring the functions to make the [inputs explicit](https://design.tidyverse.org/inputs-explicit.html). If that's not possible, then you can use a function like `withr::local_options()` or `withr::local_envvar()` to temporarily change options and environment values within a test. Learn more in `vignette("test-fixtures")`.
+
+## Random numbers
+
+Random number generators generate different numbers each time you can them because they update a special `.Random.seed` variable. You can temporarily set this seed to a known value to make your random numbers reproducible with `withr::local_seed()`, making random numbers a special case of test fixtures.
+
+```{r}
+#| label: random-local-seed
+
+dice <- function() {
+  sample(6, 1)
+}
+
+test_that("dice returns different numbers", {
+  withr::local_seed(1234)
+
+  expect_equal(dice(), 4)
+  expect_equal(dice(), 2)
+  expect_equal(dice(), 6)
+})
+```
+
+Alternatively, you might want to mock your function that wraps a random number generator:
+
+```{r}
+#| label: random-mock
+
+roll_three <- function() {
+  sum(dice(), dice(), dice())
+}
+
+test_that("three dice adds values of individual calls", {
+  local_mocked_bindings(dice = mock_output_sequence(1, 2, 3))
+  expect_equal(roll_three(), 6)
+})
+```
+
+## HTTP requests
+
+If you're trying to test functions that rely on HTTP requests, we recommend using {vcr} or {httptest2}. These packages provide the ability to record and then later reply HTTP requests so that you can test without an active internet connection. If your package is going to CRAN, we highly recommend either using one of these packages or using `skip_on_cran()` for your internet facing tests. This ensures that your package won't break on CRAN just because the service you're using is temporarily down.
+
+## User interaction
+
+If you're testing a function that relies on user feedback from `readline()` or `menu()` or similar, you can use mocking (`vignette("mocking")`) to temporarily make those functions return fixed values. For example, imagine that you have the following function that asks the user if they want to continue:
+
+```{r}
+#| label: continue
+
+continue <- function(prompt) {
+  cat(prompt, "\n", sep = "")
+
+  repeat {
+    val <- readline("Do you want to continue? (y/n) ")
+    if (val %in% c("y", "n")) {
+      return(val == "y")
+    }
+    cat("! You must enter y or n\n")
+  }  
+}
+
+readline <- NULL
+```
+
+You can test its behaviour by mocking `readline()` and using a snapshot test:
+
+```{r}
+#| label: mock-readline
+
+test_that("user must respond y or n", {
+  mock_readline <- local({
+    i <- 0
+    function(prompt) {
+      i <<- i + 1
+      cat(prompt)
+      val <- if (i == 1) "x" else "y"
+      cat(val, "\n", sep = "")
+      val
+    }
+  })
+
+  local_mocked_bindings(readline = mock_readline)
+  expect_snapshot(val <- continue("This is dangerous"))
+  expect_true(val)
+})
+```
+
+If you were testing the behaviour of some function that used `continue()`, you might choose to mock it directly:
+
+```{r}
+#| label: mock-continue
+
+save_file <- function(path, data) {
+  if (file.exists(path)) {
+    if (!continue("`path` already exists")) {
+      stop("Failed to continue")
+    }
+  }
+  writeLines(data, path)
+}
+
+test_that("save_file() requires confirmation to overwrite file", {
+  path <- withr::local_tempfile(lines = letters)
+
+  local_mocked_bindings(continue = function(...) TRUE)
+  save_file(path, "a")
+  expect_equal(readLines(path), "a")
+
+  local_mocked_bindings(continue = function(...) FALSE)
+  expect_snapshot(save_file(path, "a"), error = TRUE)
+})
+```
+
+## User-facing text
+
+Errors, warnings, and other user-facing text should be tested to ensure they're helpful and consistent. Obviously you can't test this 100% automatically, but you can ensure that such messaging is clearly shown in PRs, so another human can take a look. This is exactly the point of snapshot tests; learn more in `vignette("snapshotting")`.
+
+## Testing interfaces
+
+Sometimes you want to ensure multiple functions obey the same interface (e.g. they have the same arguments, or all error in the same way). In this case you can write a helper function that calls `test_that()` and then use this inside your other tests. This works because testthat (as of 3.3.0) supports nested tests, which you can learn more about in `vignette("nested-tests")`.

vignettes/custom-expectation.Rmd

@@ -7,7 +7,8 @@ vignette: >
   %\VignetteEncoding{UTF-8}
 ---
 
-```{r setup, include = FALSE}
+```{r setup}
+#| include: false
 library(testthat)
 knitr::opts_chunk$set(collapse = TRUE, comment = "#>")