Skip to content

README.md

Latest commit

 

History

History
507 lines (372 loc) · 10.1 KB

README.md

File metadata and controls

507 lines (372 loc) · 10.1 KB

Testing Suite for 4ndy-bspwm

Comprehensive test suite for the 4ndy-bspwm setup script and configuration.

📋 Table of Contents

🎯 Overview

This test suite validates:

  • ✅ Security: Command injection prevention, proper quoting, no hardcoded secrets
  • ✅ Functionality: All setup steps work correctly
  • ✅ Compatibility: Works on Debian/Ubuntu/Kali Linux
  • ✅ Dependencies: All required packages and files exist
  • ✅ Regressions: Previously fixed bugs don't reappear

Test Statistics

Test File Tests Focus Area
test_setup.bats 80+ Unit tests for setup.sh components
test_security.bats 50+ Security vulnerabilities and best practices
test_integration.bats 30+ End-to-end workflow validation
Total 160+ Comprehensive coverage

🛠️ Installation

Prerequisites

  • Bash 4.0 or higher
  • Git
  • Bats (Bash Automated Testing System)

Installing Bats

Option 1: From Package Manager (Recommended)

# Debian/Ubuntu/Kali
sudo apt update
sudo apt install bats

# Verify installation
bats --version

Option 2: From Source

git clone https://github.com/bats-core/bats-core.git
cd bats-core
sudo ./install.sh /usr/local

Optional Tools

For enhanced testing capabilities:

# ShellCheck (static analysis for shell scripts)
sudo apt install shellcheck

# Python testing tools (for whichSystem.py)
sudo apt install python3-pytest

🚀 Running Tests

Quick Start

Run all tests:

# From the repository root
bats tests/*.bats

Run Specific Test Suites

# Unit tests only
bats tests/test_setup.bats

# Security tests only
bats tests/test_security.bats

# Integration tests only
bats tests/test_integration.bats

Run Individual Tests

# Run a specific test by name
bats tests/test_setup.bats --filter "setup.sh file exists"

# Run tests matching a pattern
bats tests/test_setup.bats --filter "version"

Verbose Output

# Show all test output (including passing tests)
bats tests/test_setup.bats --verbose

# Pretty formatting
bats tests/test_setup.bats --pretty

# Timing information
bats tests/test_setup.bats --timing

📚 Test Categories

1. Unit Tests (test_setup.bats)

Tests individual components of setup.sh.

Categories:

  • Basic checks (file exists, executable, shebang)
  • Security checks (root prevention, signal handlers)
  • Version checks (LSD, Go, OpenJDK)
  • Dependency checks (all packages listed)
  • Installation step order
  • Configuration file handling
  • Permission settings
  • Error handling

Example output:

 ✓ setup.sh file exists
 ✓ setup.sh is executable
 ✓ LSD version is 1.2.0 or higher
 ✓ Go version is 1.23.5 or higher
 ✓ Zsh plugins are included (FIX verification)

2. Security Tests (test_security.bats)

Validates security best practices and vulnerability prevention.

Categories:

  • Command injection prevention
  • Privilege escalation prevention
  • File system safety
  • Network security (HTTPS only)
  • No hardcoded credentials
  • Supply chain security
  • Docker security

Example output:

 ✓ [SECURITY] No unquoted variable expansions
 ✓ [SECURITY] Script refuses to run as root
 ✓ [SECURITY] All downloads use HTTPS
 ✓ [SECURITY] No credentials in script
 ✓ [SECURITY] Exit code check fix prevents race condition

3. Integration Tests (test_integration.bats)

Tests the complete workflow and file operations.

Categories:

  • Repository structure validation
  • Download URL validation (network required)
  • Script flow validation
  • File operations simulation
  • Permission operations
  • Error handling flow
  • Cleanup operations

Example output:

 ✓ [INTEGRATION] All required directories exist
 ✓ [INTEGRATION] Config files can be copied
 ✓ [INTEGRATION] Exit code capture works correctly
 - [INTEGRATION] LSD download URL is reachable (skipped)

🔄 CI/CD Integration

GitHub Actions

Tests run automatically on:

  • Every push to main or master
  • Every pull request
  • Manual workflow dispatch

What runs:

  1. ✅ Bats tests (all 3 suites)
  2. ✅ ShellCheck static analysis
  3. ✅ Bash syntax validation
  4. ✅ Python script tests
  5. ✅ Version URL validation (network check)
  6. ✅ Repository structure check
  7. ✅ Security audit

View results:

  • Go to your repository on GitHub
  • Click "Actions" tab
  • See test results for each commit/PR

Local CI Simulation

Run the same checks that CI runs:

# All Bats tests
bats tests/*.bats

# ShellCheck
shellcheck -S warning setup.sh
find scripts -name "*.sh" -exec shellcheck {} \;

# Syntax check
bash -n setup.sh

# Python syntax
python3 -m py_compile scripts/whichSystem.py

✍️ Writing New Tests

Test Template

@test "Description of what is being tested" {
    # Arrange: Set up test conditions
    expected="value"

    # Act: Perform the action
    result=$(grep "PATTERN" file)

    # Assert: Verify the result
    [ "$result" == "$expected" ]
}

Best Practices

  1. Descriptive names: Use clear, specific test names

    # Good
@test "LSD version is 1.2.0 or higher"

# Bad
@test "check version"

  2. One assertion per test: Keep tests focused

    # Good
@test "setup.sh exists" {
    [ -f "./setup.sh" ]
}

@test "setup.sh is executable" {
    [ -x "./setup.sh" ]
}

# Bad (testing two things)
@test "setup.sh exists and is executable" {
    [ -f "./setup.sh" ]
    [ -x "./setup.sh" ]
}

  3. Use categories: Prefix related tests

    @test "[SECURITY] No hardcoded passwords"
@test "[REGRESSION] Version fix is applied"
@test "[INTEGRATION] File copy works"

  4. Skip when appropriate:

    @test "Network-dependent test" {
    skip "Requires network access - run manually"
    curl -I https://example.com
}

Common Assertions

# File existence
[ -f "file.txt" ]              # File exists
[ -d "directory" ]             # Directory exists
[ -x "script.sh" ]             # File is executable
[ -L "symlink" ]               # Is a symlink

# String comparison
[ "$var" == "value" ]          # Equal
[ "$var" != "value" ]          # Not equal
[[ "$var" =~ pattern ]]        # Regex match

# Numeric comparison
[ "$num" -eq 5 ]               # Equal
[ "$num" -gt 5 ]               # Greater than
[ "$num" -lt 5 ]               # Less than

# Command success
command -v git                 # Command exists
grep -q "pattern" file         # Pattern found (quiet)

Running Your New Tests

# Run just your new test
bats tests/test_setup.bats --filter "your test name"

# Add to the appropriate file
vim tests/test_setup.bats      # Unit tests
vim tests/test_security.bats   # Security tests
vim tests/test_integration.bats # Integration tests

🐛 Troubleshooting

Common Issues

1. Bats not found

$ bats tests/test_setup.bats
bash: bats: command not found

Solution:

sudo apt install bats

2. Test fails with "file not found"

✗ setup.sh file exists
   (in test file tests/test_setup.bats, line 21)

Solution: Run tests from repository root:

cd /path/to/4ndy-bspwm
bats tests/*.bats

3. Permission denied errors

✗ setup.sh is executable
   (in test file tests/test_setup.bats, line 25)

Solution: Make setup.sh executable:

chmod +x setup.sh

4. Python import errors

✗ Python Script Tests
   ModuleNotFoundError: No module named 'whichSystem'

Solution: Tests run from scripts directory:

cd scripts
python3 -c "import whichSystem"

5. Network tests fail

✗ [INTEGRATION] LSD download URL is reachable

Solution: These tests are skipped by default. Run manually:

# Remove skip line from test
sed -i '/skip "Requires network/d' tests/test_integration.bats

# Run with network access
bats tests/test_integration.bats

Getting Help

📊 Test Coverage

Current coverage of setup.sh:

Area Coverage Tests
Security 95% 50+ tests
Installation steps 90% 40+ tests
Error handling 85% 20+ tests
Configuration 90% 30+ tests
Dependencies 100% 15+ tests

🔍 Continuous Improvement

Adding Tests for New Features

When adding new functionality to setup.sh:

  1. Write the test first (TDD approach)

    @test "New feature works correctly" {
    [ -f "new_file.txt" ]
}

  2. Implement the feature

    # Add to setup.sh
touch new_file.txt

  3. Verify test passes

    bats tests/test_setup.bats --filter "New feature"

Regression Testing

When fixing a bug:

  1. Write a test that fails (reproduces the bug)
  2. Fix the bug
  3. Verify test passes
  4. Add [REGRESSION] tag to prevent it from breaking again

Example:

@test "[REGRESSION] Wallpaper directory uses \$HOME not tilde" {
    grep Wallpapers setup.sh | grep 'if \[\[' | grep -q '\$HOME'
}

📈 Future Enhancements

Planned improvements:

  • Add code coverage reporting
  • Add performance benchmarks
  • Add mutation testing
  • Add Docker-based integration tests
  • Add visual regression tests for polybar themes

📝 License

These tests are part of the 4ndy-bspwm project and follow the same license.

🙏 Contributing

To contribute tests:

  1. Fork the repository
  2. Create a feature branch (git checkout -b test/new-feature)
  3. Add your tests
  4. Ensure all tests pass (bats tests/*.bats)
  5. Commit your changes
  6. Push to the branch
  7. Create a Pull Request

Happy Testing! 🧪