Skip to content

Add comprehensive BATS black box testing framework#38

Merged
justincampbell merged 14 commits intomainfrom
add-blackbox-tests
Sep 24, 2025
Merged

Add comprehensive BATS black box testing framework#38
justincampbell merged 14 commits intomainfrom
add-blackbox-tests

Conversation

@justincampbell
Copy link
Copy Markdown
Member

@justincampbell justincampbell commented Sep 23, 2025

Summary

  • Fix CLI bug where --directory flag was ignored due to initialization order
  • Add comprehensive BATS testing framework with 56 black box tests
  • Add GitHub Actions CI workflow to run tests automatically

Changes

Bug Fix

  • Fixed --directory flag initialization: Moved client creation from init() to PersistentPreRun so flags are parsed before client instantiation

Testing Framework

  • BATS infrastructure: Added test_helper.bash with utilities for file assertions and hook testing
  • Makefile: Simple make test command with colored output
  • 56 comprehensive tests covering all CLI commands:
    • Core workflow (34 tests): start, finish, clear, cancel, amend, repeat
    • Hook system (9 tests): start/stop hook execution, error handling, edge cases
    • Display commands (10 tests): status output validation, history with --limit
    • Break command (3 tests): timer and hook integration

CI/CD

  • GitHub Actions workflow: Automatically runs full test suite on PRs and main branch
  • Multi-platform testing: Linux and macOS
  • BATS integration: Installs BATS and runs make test

Test Coverage

All tests use temporary directories for isolation and verify CLI behavior through actual command execution and file system assertions. The framework successfully discovered and helped fix the --directory flag bug.

Example Test

@test "finish records actual elapsed time in history" {
    pomodoro start "Work session" -d 30 --ago 10m
    run pomodoro finish
    [ "$status" -eq 0 ]

    assert_file_contains "history" "Work session"
    assert_file_contains "history" "duration=10"
}

🤖 Generated with Claude Code

justincampbell and others added 5 commits September 22, 2025 16:50
@justincampbell @claude
Fix --directory flag initialization bug
f679168 
Move client initialization from init() to PersistentPreRun to ensure
flags are parsed before creating the client. This fixes the --directory
flag which was being ignored because the client was created with an
empty directory string before flag parsing occurred.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@justincampbell @claude
Add comprehensive BATS black box testing framework
1409054 
- Add BATS test infrastructure with test_helper.bash utilities
- Add Makefile with `make test` target for colored test output
- Add 34 focused black box tests covering all core commands:
  - start.bats (7 tests) - command creation and file writing
  - finish.bats (6 tests) - completion and history management
  - clear.bats (4 tests) - current pomodoro removal
  - cancel.bats (5 tests) - cancellation without history
  - amend.bats (6 tests) - current pomodoro modification
  - repeat.bats (6 tests) - last pomodoro repetition

Tests verify CLI behavior through actual command execution and file
system assertions, providing end-to-end validation of the user
experience. All tests use temporary directories for isolation.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@justincampbell @claude
Add comprehensive hook testing with BATS
2d97c93 
- Add hook testing infrastructure to test_helper.bash:
  - create_hook() - creates executable hook scripts
  - assert_hook_executed() - verifies hook execution
  - assert_hook_contains() - checks hook log contents

- Add hooks.bats with 9 tests covering all hook scenarios:
  - start/stop hook execution for all commands
  - non-executable hook error handling
  - missing hook graceful handling
  - hook failure behavior
  - multiple hook sequence execution

Total test coverage now: 43 comprehensive black box tests

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@justincampbell @claude
Add comprehensive tests for remaining commands
b8a9cad 
- Add status.bats (5 tests) - current pomodoro display and state
- Add break.bats (3 tests) - break timer hook execution and validation
- Add history.bats (5 tests) - history display and --limit flag functionality

Complete black box test coverage: 56 tests across all CLI commands
- Core workflow: start, finish, clear, cancel, amend, repeat (34 tests)
- Hook system: comprehensive hook execution testing (9 tests)
- Display commands: status and history output validation (10 tests)
- Break command: timer and hook integration (3 tests)

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@justincampbell @claude
Add GitHub Actions CI workflow for automated testing
d9d0644 
- Run BATS test suite on push and pull requests
- Test on both Ubuntu and macOS platforms
- Install BATS via package manager (apt/brew)
- Execute full test suite with `make test`

Ensures all 56 black box tests pass before merge and catches
regressions across different environments.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
justincampbell and others added 9 commits September 23, 2025 21:40
@justincampbell @claude
Use user's default pomodoro duration from settings
747b3fd 
When no --duration flag is provided, read DefaultPomodoroDuration from
settings file instead of hardcoding 25 minutes. This restores the
intended behavior that was broken by the --directory flag fix.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@justincampbell @claude
Add just-in-time BATS download to Makefile
9af74b6 
Download bats-core automatically if not installed, eliminating the need
for users to install BATS separately. Includes clean target to remove
downloaded BATS installation.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@justincampbell @claude
Build binary for tests with dependency tracking
7a5b332 
- Add binary build target to Makefile that tracks Go source files
- Use compiled binary instead of 'go run' in tests for better performance
- Only rebuild when Go sources or module files change
- Add pomodoro binary to .gitignore
- Update clean target to remove binary

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@justincampbell @claude
Add comprehensive settings tests
3165798 
Test coverage for:
- Default pomodoro duration from settings
- Default break duration from settings
- Explicit duration flags overriding settings
- --directory flag using settings from specified directory
- Multiple settings values parsing
- Fallback to 25 minute default when no settings

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@justincampbell @claude
Add create_settings helper for cleaner test code
00ff41e 
Add create_settings() helper that:
- Accepts multiple setting key=value pairs as arguments
- Optionally takes custom file path as first arg (if contains /)
- Defaults to $TEST_DIR/settings
- Makes multi-line settings more readable

Updated all settings tests to use the new helper.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@justincampbell @claude
Use explicit create_settings_in for custom paths
c1a48fc 
Replace magic path detection with two explicit functions:
- create_settings: writes to $TEST_DIR/settings
- create_settings_in <path>: writes to specified path

This makes the intent clear without relying on / detection.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@justincampbell @claude
Use Makefile BATS installation in CI
35bdb6d 
Remove manual BATS installation from GitHub Actions workflow.
The Makefile now handles downloading BATS automatically, ensuring
consistent versions across local dev and CI environments.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@justincampbell @claude
Fix file endings - add newlines to end of files
ff6a0ff 
Ensure all files end with a single newline character.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@justincampbell @claude
Remove redundant tests and improve test names
c436128 
- Removed duplicate 'cancel empties current file' test
- Removed duplicate 'clear empties current file' test
- Removed duplicate 'finish preserves description' test
- Updated test names to include 'and' for combined assertions

Reduced test count from 63 to 60 while maintaining coverage.

🤖 Generated with [Claude Code](https://claude.ai/code)

Co-Authored-By: Claude <noreply@anthropic.com>
@justincampbell justincampbell merged commit 713037d into main Sep 24, 2025
2 checks passed
@justincampbell justincampbell deleted the add-blackbox-tests branch September 24, 2025 02:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@justincampbell