GitGuardian to GHAS Dismissed & Resolved Incident Importer

A Python tool that automatically closes GitHub Advanced Security (GHAS) secret scanning alerts by matching them with previously triaged false positives from GitGuardian exports.

Overview

This tool solves the problem of having to re-triage thousands of security alerts when migrating from GitGuardian to GitHub Advanced Security. It matches GHAS secret scanning alerts with dismissed or resolved GitGuardian incidents by comparing secret types and values, then automatically closes matching alerts.

Features

• Automated Alert Closure: Automatically closes GHAS secret scanning alerts that match GitGuardian false positives
• Multi-Secret Type Support: Handles various secret types including AWS keys, GitHub tokens, private keys, database connection strings, and more
• Robust Matching: Implements sophisticated matching algorithms for different secret formats
• GitGuardian CSV Integration: Processes GitGuardian CSV exports with "Show Secrets" option enabled
• Comprehensive Logging: Detailed logging for audit trails and debugging
• Batch Processing: Can process alerts across entire GitHub organizations

Supported Secret Types

The tool currently supports matching for the following secret types:

• AWS Credentials: Access keys, secret keys, session tokens
• Private Keys: RSA, OpenSSH, PGP private keys
• Authentication: HTTP Basic/Bearer authentication headers
• API Keys: GitHub personal access tokens, Google API keys, JFrog tokens
• Database Connections: PostgreSQL connection strings
• Cloud Services: Google Cloud service account credentials, Databricks tokens
• Communication: Slack API tokens and webhook URLs
• Generic Secrets: High entropy secrets, passwords, bearer tokens

Prerequisites

• Python 3.13+
• GitHub token with security_events:write scope for the target organization (Secret Scanning Alerts r/w for fine-grained)
• GitGuardian token with Members:read and Incidents:read permissions
• GitGuardian CSV export with "Show Secrets" option enabled

Installation

1. Clone the repository:

git clone <repository-url>
cd gitguardian-fp-importer

2. Compile and install dependencies:

# Compile requirements from .in files
./compile-requirements.sh

# Install production dependencies
pip install -r requirements.txt

# Or install development dependencies (includes production deps)
pip install -r requirements-dev.txt

Dependency Management

This project uses pip-tools for dependency management:

• requirements.in: High-level production dependencies
• requirements-dev.in: Development dependencies (includes production)
• compile-requirements.sh: Script to generate locked requirements files

To update dependencies:

# Edit requirements.in or requirements-dev.in
# Then recompile
./compile-requirements.sh

Configuration

Set the required environment variables:

 export GITHUB_TOKEN="your_github_token_here"
 export GITGUARDIAN_TOKEN="your_gitguardian_token_here"

Usage

Basic Usage

Process a single repository: (add --dismiss-alerts to actually close the GHAS alerts)

python update_ghas_secret_alerts_from_gitguardian_data.py \
  --gitguardian-csv-file gitguardian_export.csv \
  --gitguardian-api-url https://api.gitguardian.com/v1 \
  --org your-org \
  --repo your-repo 

Organization-wide Processing

Process all repositories in an organization:

python update_ghas_secret_alerts_from_gitguardian_data.py \
  --gitguardian-csv-file gitguardian_export.csv \
  --gitguardian-api-url https://api.gitguardian.com/v1 \
  --org your-org \

Command Line Options

Required Arguments

• --org: GitHub organization name
• --gitguardian-csv-file: Path to GitGuardian CSV export file with secret info
• --gitguardian-api-url: GitGuardian API URL (e.g., https://api.gitguardian.com/v1)

Optional Arguments

• --repo: Specific repository name (processes all repos in org if omitted)
• --dismiss-alerts: Actually dismiss alerts in GHAS (default is dry-run mode)
• --secret-type: Filter alerts by specific secret type (e.g., aws_secret_access_key)
• --matching-info-csv-output: CSV output file about which alerts matched
• --close-info-csv-output: CSV output file with information about alerts being closed
• --debug: Enable debug logging
• --debug-log-secrets: Enable debug logging of secrets (WARNING: outputs secrets to console)
• --alert: Specific secret alert number to process (useful for testing)
• --include-closed-alerts: Query closed GHAS alerts (useful for testing)

GitGuardian CSV Export Setup

1. Navigate to the GitGuardian Incidents page
2. Apply filters for "Ignored" or "Resolved" incidents
3. Select "Individual secret incidents without details about multiple occurrences"
4. Important: Enable "Show Secrets" option
5. Export to CSV

How It Works

1. Load GitGuardian Data: Parses the CSV export and organizes incidents by detector type
2. Fetch GHAS Alerts: Retrieves open secret scanning alerts from GitHub
3. Secret Matching: Uses specialized mappers to compare secrets:
   • Simple Mapping: Direct value comparison for API keys and tokens
   • Base64 Decoding: Handles authentication headers and encoded secrets
   • Key Normalization: Strips metadata and formatting from private keys
   • URL Parsing: Extracts components from database connection strings
4. Alert Closure: Marks matching alerts as false positives with detailed reasoning

Architecture

Core Components

• GitGuardianIncidentRow: Represents a single incident from the CSV export
• GitGuardianIncidentDictionary: Organizes incidents by detector type
• AbstractGitGuardianMapper: Base class for secret matching strategies
• Specialized Mappers: Handle different secret formats and edge cases

Key Files

• update_ghas_secret_alerts_from_gitguardian_data.py: Main application logic
• gitguardian_mappers.py: Secret matching mapper classes and logic
• crypto_utils.py: Utilities for handling malformed private keys and cryptographic data
• gitguardian_incident_info.py: Helper for GitGuardian API interactions

Limitations

• Requires GitGuardian CSV export with secrets visible
• Some private key formats may not match due to GitGuardian encoding issues
• Limited to secret types with implemented mappers
• Cannot process secrets that GitGuardian failed to detect

Contributing

1. Fork the repository
2. Create a feature branch
3. Perform testing (automated tests don't exist but we would welcome some!)
4. Submit a pull request

Adding New Secret Types

To add support for a new secret type:

1. Create a new mapper class inheriting from AbstractGitGuardianMapper in gitguardian_mappers.py
2. Implement the secret_present_in_row method
3. Add the mapping to the create_ghas_to_gitguardian_converters() function

Logging

The tool provides comprehensive logging:

• INFO: High-level progress and statistics
• DEBUG: Detailed matching attempts and API calls
• WARNING: Potential issues or unsupported secret types
• ERROR: Critical failures

Security Considerations

• Store API tokens securely using environment variables
• Limit GitHub token permissions to minimum required scope
• Review logs for sensitive data before sharing
• The tool runs in dry-run mode by default - use --dismiss-alerts to actually close alerts
• Never use --debug-log-secrets in production or shared environments as it outputs secrets to logs

License

This project is licensed under the Apache 2.0 License - see the LICENSE file for details.

Support

For issues or questions:

1. Check the existing issues on GitHub
2. Review the research documentation
3. Enable debug logging for troubleshooting
4. Create a new issue with detailed information