Skip to content

Contributions back from cs491tc #30

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 2 commits into
base: gh-pages
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 7 additions & 0 deletions _episodes/01-basics.md
Original file line number Diff line number Diff line change
Expand Up @@ -21,6 +21,13 @@ that mistakes *will* happen and to guard against them. This is called
**defensive programming** and the most common way to do it is to add alarms and
tests into our code so that it checks itself.

Why bother? We need to have confidence in how our code executes even if we do not plan to share it further. Software testing allows us to make sure that:

- the program actually meets our design requirements
- the program responds in expected ways to various inputs (including invalid inputs)
- the program is usable (has a well-defined interface, etc.)
- the program works on the target platform(s)

**Testing** should be a seamless part of scientific software development process.
This is analogous to experiment design in the experimental science world:

Expand Down
21 changes: 17 additions & 4 deletions _episodes/02-assertions.md
Original file line number Diff line number Diff line change
Expand Up @@ -81,18 +81,31 @@ def mean(num_list):
> that arise with floating point arithmetic. Using the assert keyword, how could
> you test whether some value is almost the same as another value?
>
> - My package, mynum, provides the number a.
> - Use the `assert` keyword to check whether the number a is greater than 2.
> - Use the `assert` keyword to check that a is equal to 2 within an error of 0.003.
> - A program provides the number `a`.
> - Use the `assert` keyword to check whether the number `a` is greater than 2.
> - Use the `assert` keyword to check that `a` is equal to 2 within an error of 0.003.
{: .callout}

~~~
from mynum import a
a = 2.002
# greater than 2 assertion here
# 0.003 assertion here
~~~
{: .python}

We can think of assertions as fences set up to keep undesirable output out of our function. Fences are barriers employed to block program execution if the state isn't adequate to the intended task. Assertion fences are crude but effective ways to check input.

>Consider the following function.
>
>If one were to include a fence as the first line of this function, what would be an appropriate fence to avoid a ZeroDivisionError?
{: .callout}

~~~
def mean( num_list ):
return sum( num_list ) / len( num_list )
~~~
{: .python}

## NumPy

The NumPy numerical computing library has a built-in function `assert_allclose`
Expand Down
62 changes: 60 additions & 2 deletions _episodes/04-units.md
Original file line number Diff line number Diff line change
Expand Up @@ -58,9 +58,9 @@ class. Ultimately, the test occurs when an assertion is made, comparing the
observed and expected values. For example, let us test that our mean function
successfully calculates the known value for a simple list.

Before running the next code, save your `mean` function to a file called mean.py in the working directory.
Before running the next code, save your `mean` function to a file called `mean.py` in the working directory.

You can use this code to save to file:
You can use this code for your function:

~~~
def mean(num_list):
Expand Down Expand Up @@ -152,3 +152,61 @@ test_complex() ## Please note that this one might fail. You'll get an error mes


Well, **that** was tedious.

> ## Unit Tests v. Assertion Fences
> - Are unit tests a replacement of assertion fences?
{: .callout}

Broadly, we can divide unit tests into _positive_ tests, those which should succeed or handle a case correctly; and _negative_ tests, those which should fail or reject a condition.

The following code has examples of both positive and negative tests.

~~~
def test_absolute():
# Test integer behavior
assert absolute( 5 ) == 5
assert absolute( 0 ) == 0
assert absolute( -5 ) == 5

# Test float behavior
assert absolute( 5.0 ) == 5.0
assert absolute( 0.0 ) == 0.0
assert absolute( -5.0 ) == 5.0
~~~
{: .python}

> ## Positive Unit Testing
>
> Python implements complex numbers as `complex`, with the form
>
> 0 + 1j # imaginary unit, sqrt(-1)
> 1 + 0j # one, 1
> 1 + 1j # 1 + 1i
>
> Note particularly the use of $j$ for the more common $i$.
>
> Suppose that a function `absolute` is available. Add a test case to `test_absolute` which verifies that `absolute` correctly handles $|1+1j| = \sqrt{2}$.
{: .callout}

Negative unit tests are designed to make a function or module fail. This can include fuzzing or ill-formed input.

Consider again an absolute value function. The negative unit tests for `absolute` should:

- Verify an exception is raised for nonnumeric input.
- Verify correct exceptions are raised for various kinds of input.

> ## Negative Unit Testing
>
> Python implements complex numbers as `complex`, with the form
>
> 0 + 1j # imaginary unit, sqrt(-1)
> 1 + 0j # one, 1
> 1 + 1j # 1 + 1i
>
> Note particularly the use of $j$ for the more common $i$.
>
> Suppose that a function `absolute` is available, but it _does not handle `complex` input_. Add a test case to `test_absolute` which verifies that `absolute` correctly handles $|1+1j| = \sqrt{2}$. In this case, that means that `absolute` should raise a `TypeError`.
>
> Furthermore, consider adding tests to ensure that string input and `list` input fail. What kind of exception should be raised and handled?


27 changes: 27 additions & 0 deletions _episodes/09-tdd.md
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,33 @@ you start writing code, you should be considering how to test that code. The
tests should be written and presented in tandem with the implementation. **Testing
is too important to be an afterthought.**

### Diagnostic Unit Testing

Test-driven development is a powerful tool to both prevent incipient bugs and to identify bugs as they occur. We distinguish different aspects of the programming flaw based on their observational mode:

- A _failure_ refers to the observably incorrect behavior of a program. This can include segmentation faults, erroneous output, and erratic results.

- A _fault_ refers to a discrepancy in code that results in a failure.

- An _error_ is the mistake in human judgment or implementation that caused the fault. Note that many errors are _latent_ or _masked_.

- A _latent_ error is one which will arise under conditions which have not yet been tested.

- A _masked_ error is one whose effect is concealed by another aspect of the program, either another error or an aggregating feature.

- We casually refer to all of these as _bugs_.

- An _exception_ is a manifestation of unexpected behavior which can give rise to a failure, but some languages—notably Python—use exceptions in normal processing as well.

Testing is designed to manifest _failures_ so that _faults_ and _errors_ can be identified and corrected.

So what does _unit testing_ have to do with this? As a philosophy, test-driven development intends to specify program behavior before program composition, which means that you *immediately know* if a program is correct. At least, that is, to the standard to which you composed your unit tests. Good unit tests will check for both latent and manifest bugs. Masked errors tend to be much more difficult to tease out since they can rely on interactions of components which aren't present in the testing harness.

By specifying a desired behavior before composing a software feature, test-driven development (TDD):

1. Clarifies what we intend a feature to accomplish;
2. Provides a reference against which an implementation can be compared.

> ## You Do You
>
> Developers who practice strict TDD will tell you that it is the best thing since
Expand Down