Add documentation for secret scanner comparison workflow

Co-authored-by: commjoen <[email protected]>

Author: Copilot

README.md

@@ -475,6 +475,20 @@ As tons of secret detection tools are coming up for both Docker and Git, we are
 Want to know if your tool detects everything? We will keep track of the embedded secrets in [this issue](https://github.com/OWASP/wrongsecrets/issues/201) and have a [branch](https://github.com/OWASP/wrongsecrets/tree/experiment-bed) in which we put additional secrets for your tool to detect.
 The branch will contain a Docker container generation script using which you can eventually test your container secret scanning.
 
+### Automated Secret Scanner Comparison
+
+We now provide an automated GitHub Action workflow that benchmarks multiple secret scanning tools against the WrongSecrets codebase. The [Secret Scanner Comparison workflow](/.github/workflows/scanner-comparison.yml) tests 7 different tools:
+
+- **TruffleHog** - Docker-based secret scanner
+- **git-secrets** - AWS Labs' git hook scanner  
+- **gitleaks** - High-performance Go-based scanner
+- **detect-secrets** - Yelp's enterprise scanner
+- **gittyleaks** - Python-based pattern detector
+- **whispers** - Skyscanner's structured scanner
+- **trufflehog3** - Python version of TruffleHog
+
+The workflow runs weekly and provides a comparison table showing how many secrets each tool detects, helping you understand the relative effectiveness of different secret scanning tools. See [docs/scanner-comparison.md](/docs/scanner-comparison.md) for more details on running and interpreting the results.
+
 ## CTF
 
 We have 3 ways of playing CTFs:

docs/scanner-comparison.md

@@ -0,0 +1,77 @@
+# Secret Scanner Comparison Workflow
+
+## Overview
+
+The `scanner-comparison.yml` workflow provides a comprehensive benchmark comparing multiple secret scanning tools on the OWASP WrongSecrets repository. This helps understand the relative effectiveness of different secret detection tools.
+
+## Supported Scanners
+
+The workflow currently tests 7 different secret scanning tools:
+
+1. **TruffleHog** - Docker-based secret scanner with verified results
+2. **git-secrets** - AWS Labs' git hook for preventing secrets in commits
+3. **gitleaks** - High-performance secret scanner with JSON output
+4. **detect-secrets** - Yelp's enterprise secret scanner with baseline functionality
+5. **gittyleaks** - Python-based secret scanner with broad pattern detection
+6. **whispers** - Skyscanner's structured secret detection tool
+7. **trufflehog3** - Python version of TruffleHog with additional features
+
+## Running the Workflow
+
+### Manual Execution
+```bash
+# Trigger manually via GitHub Actions UI
+# Go to Actions > Secret Scanner Comparison Benchmark > Run workflow
+```
+
+### Scheduled Execution
+The workflow automatically runs every Sunday at 02:00 UTC to provide regular benchmarking.
+
+## Output Format
+
+The workflow generates a summary report showing:
+
+```
+| Scanner | Secrets Found |
+|---------|---------------|
+| TruffleHog | X |
+| git-secrets | X |
+| gitleaks | X |
+| detect-secrets | X |
+| gittyleaks | X |
+| whispers | X |
+| trufflehog3 | X |
+```
+
+## Expected Results
+
+Based on manual testing, the tools typically find:
+- **detect-secrets**: ~98 potential secrets
+- **gitleaks**: ~106 secrets
+- **gittyleaks**: ~137 findings
+
+Results may vary as the repository evolves and tools update their detection patterns.
+
+## Error Handling
+
+The workflow includes robust error handling:
+- Tools that fail to install will show 0 results
+- Network timeouts are handled gracefully
+- Each scanner job runs independently in parallel
+- Failed scans don't block the summary report
+
+## Tool Installation Notes
+
+- **git-secrets**: Compiled from source (no official GitHub Action)
+- **gitleaks**: Downloaded as binary from GitHub releases
+- **Python tools**: Installed via pip with timeout handling
+- **TruffleHog**: Uses official Docker image
+
+## Interpreting Results
+
+Different tools have varying approaches:
+- Some focus on verified/confirmed secrets
+- Others detect potential patterns that may be false positives
+- Count differences are expected and help understand tool characteristics
+
+This benchmark helps users choose appropriate tools for their security scanning needs based on detection coverage and accuracy requirements.