CI improvements

Some quality of life items:

- Make pytest output colorful in CI, so it's easier to tell which test is failing.
- Consolidate testing for PR and commit, i.e. test on all platforms inside PRs.