@@ -482,21 +482,64 @@ def test_pytest(mocker):
482482 dangerous_sideffects()
483483```
484484
485- ## Testing Edgecases
485+ ## Extensive Input Testing
486486
487- While writing unit tests, you may be tempted to test edgecases. You may have a
488- critical private function or algorithm, which is not part of the public API (so
489- not a good candidate for External testing) and you are concerned about many
490- edgecases that you want to defend against using tests.
487+ The range of inputs that test cases validate is an important decision.
491488
492- It is perfectly valid to write extensive edgecase testing for private code, but
493- these tests should be kept separate from the unit test suite. Extensive edgecase
494- testing makes tests long, and difficult to read (tests are documentation). They
495- can slow down execution, we want unit tests to run first, fast, and often.
489+ When the need for extensive testing starts to conflict with the readability of test cases
490+ and their usefulness as documentation for users and other developers, the tests should
491+ be re-organized into public-facing (concise, expressive, easily readable),
492+ and technical (complex, extensive) test files.
493+
494+ - To maintain readability:
495+ - tests may need to include fewer inputs
496+ - extensive edgecase testing may be moved into different sections or files
497+ - the code itself may need to be refactored
498+ - Avoid testing invalid input; the range of invalid input is infinite, and
499+ is the programming equivalent of trying to prove a negative.
500+ - Focus on inputs relevant to the code under test, and avoid testing the code in dependencies.
501+ Some integration testing of specific behaviors your code relies on is justifiable.
502+ - [ Fuzz Tests] ( #fuzz-tests ) are the best place to handle extensive and exhaustive input testing.
503+
504+
505+ ### In Public Interface Tests
506+
507+ These are the most appropriate place to test certain invalid inputs and dependencies.
508+ Public Interface tests act like a contract with users; each behavior that is tested is like a promise that
509+ users can rely on, and expect that it will not change without warning (and probably a major version bump).
510+ So any input/output and side-effects included in these tests should be considered * officially supported behavior*
511+ and given careful consideration.
512+
513+ ### In project level integration tests
514+
515+ These are a good place to handle more extensive input testing. Integration tests already tend to be more verbose,
516+ with a lot of setup and teardown, and much more behavior to cover than other kinds of tests.
517+ These are the kinds of tests that should focus on edgecases.
518+
519+
520+ ### In Unit Tests
521+
522+ Unit Tests should focus on the "happy-path" of execution. In most cases one representative
523+ example of the * expected* input is sufficient. The test case should illustrate how the unit
524+ is expected to be used.
525+
526+ Invalid input should only be tested when the unit itself includes logic to handle that invalid input.
527+
528+ for example, this code:
529+ ``` python
530+ def foo (x : int ):
531+ return x + 1
532+ ```
533+ should not test its behavior when passed a string (the type annotation already covers that).
534+
535+ This code should be tested with a string, to cover the exception path.
536+ ``` python
537+ def bar (x ):
538+ if type (x) is str :
539+ raise RuntimeError (' invalid input' )
540+ return x + 1
541+ ```
496542
497- - Place them in separate files from unit tests, to improve readability
498- - [ mark them] ( https://docs.pytest.org/en/stable/how-to/mark.html ) so that they
499- can be run as a separate test suite, after your unit test pass
500543
501544## Additional Types of Test Suites
502545
0 commit comments