Merge pull request #12 from theztefan/main

CallMeGreg · web-flow · commit 3d03d857d05f · 2024-08-19T12:13:48.000-04:00
Adding Secret Scanning out-of-the-box validity field in output
diff --git a/.DS_Store b/.DS_Store
diff --git a/.gitignore b/.gitignore
@@ -1,4 +1,6 @@
 /gh-secret-scanning
 /gh-secret-scanning.exe
 release-command.md
-Secret Scanning Report*.csv
+secret*.csv
+Secret*.csv
+.DS_Store
diff --git a/README.md b/README.md
@@ -1,88 +1,102 @@
-# Overview
+# gh-secret-scanning - The GitHub Secret Scanning CLI Extension
+
 This project is a GitHub CLI (`gh`) extension that provides commands for interacting with secret scanning alerts.
 
 This extension helps GitHub Advanced Security (GHAS) customers prioritize remediation of their secret scanning alerts by identifying and focusing on those that are confirmed active first.
 
-While this extension works for Enterprise Cloud (GHEC) customers, it is primarily intended for GitHub Enterprise Server (GHES) customers who do not have access to the [GitHub.com secret scanning validity check feature](https://docs.github.com/en/enterprise-cloud@latest/code-security/secret-scanning/managing-alerts-from-secret-scanning#validating-partner-patterns).
+While this extension works for Enterprise Cloud (GHEC) customers, it is primarily intended for GitHub Enterprise Server (GHES) customers who do not have access to the [GitHub.com secret scanning validity check feature](https://docs.github.com/en/enterprise-cloud@latest/code-security/secret-scanning/managing-alerts-from-secret-scanning/evaluating-alerts#checking-a-secrets-validity). Validity check on GHES is available as of `>=3.12` but currently limited to GitHub Personal Access Tokens (PAT).
 
 Primary features include:
+
 - Listing secret scanning alerts for an enterprise, organization, or repository
 - Verifying if secret scanning alerts are still active
+  - Expand the out-of-the-box secret scanning validity checks capabilities with custom validators
 - Opening issues in repos that contain valid secrets
 
-# Supported Token Types
+## Supported Token Types
+
 - GitHub Personal Access Tokens (GHES + GHEC)
 - Slack API Tokens
 
-# Pre-requisites
+## Pre-requisites
+
 - [GitHub CLI](https://github.com/cli/cli#installation)
 - [GHES 3.7+](https://docs.github.com/en/enterprise-server@3.7/admin/all-releases#releases-of-github-enterprise-server) or [GHEC](https://docs.github.com/en/enterprise-cloud@latest/admin/overview/about-github-enterprise-cloud)
 - [GHAS](https://docs.github.com/en/enterprise-cloud@latest/get-started/learning-about-github/about-github-advanced-security)
 
-# Installation
-```
+## Installation
+
+```bash
 gh extension install CallMeGreg/gh-secret-scanning
 ```
 
-# Usage
+## Usage
+
 Authenticate with your GitHub Enterprise Server or GitHub Enterprise Cloud account:
-```
+
+```bash
 gh auth login
 ```
 
-## Alerts subcommand
+### Alerts subcommand
+
 Target either an enterprise, organization, or repository by specifying the `-e`, `-o`, or `-r` flags respectively. _Exactly one selection from these three flags is required._
 
-```
+```bash
 gh secret-scanning alerts -e <enterprise>
 ```
 
-```
+```bash
 gh secret-scanning alerts -o <organization>
 ```
 
-```
+```bash
 gh secret-scanning alerts -r <repository>
 ```
 
 Optionally add flags to specify a GHES server, limit the number of secrets processed, filter for a specific secret provider, display the secret values, generate a csv report, include extra fields, and more:
-```
+
+```bash
 gh secret-scanning alerts -e github --url my-github-server.com --limit 10 --provider slack --show-secret --csv --verbose
 ```
 
-## Verify subcommand
+### Verify subcommand
+
 Target either an enterprise, organization, or repository by specifying the `-e`, `-o`, or `-r` flags respectively. _Exactly one selection from these three flags is required._
 
-```
+```bash
 gh secret-scanning verify -e <enterprise>
 ```
 
-```
+```bash
 gh secret-scanning verify -o <organization>
 ```
 
-```
+```bash
 gh secret-scanning verify -r <repository>
 ```
 
 Optionally add flags to specify a GHES server, limit the number of secrets processed, filter for a specific secret provider, display the secret values, generate a csv report, include extra fields, and more:
-```
+
+```bash
 gh secret-scanning verify -e github --url my-github-server.com --limit 10 --provider slack --show-secret --csv --verbose
 ```
 
 Also, optionally create an issue in any repository that contains a valid secret by using the `--create-issues` (`-i`) flag:
-```
+
+```bash
 gh secret-scanning verify -e github --url my-github-server.com --create-issues
 ```
 
+### Help
 
-## Help
 See available commands and flags by running:
-```
+
+```bash
 gh secret-scanning -h
 ```
 
-```
+```bash
 Interact with secret scanning alerts for a GHEC or GHES 3.7+ enterprise, organization, or repository
 
 Usage:
@@ -109,8 +123,8 @@ Flags:
 Use "secret-scanning [command] --help" for more information about a command.
 ```
 
-# Demo
+## Demo
+
 This example first lists the alerts for an organization with the `alerts` subcommand, and then verifies the secrets with the `verify` subcommand. The `--csv` flag is used to generate a csv report of the results, and the `--create-issues` flag is used to create issues in any repository that contains a valid secret.
 
 https://github.com/CallMeGreg/gh-secret-scanning/assets/110078080/fa8d7b08-1a2c-4522-ae96-5c3aab60107d
-
diff --git a/cmd/common.go b/cmd/common.go
@@ -47,6 +47,7 @@ type Alert struct {
 	Push_protection_bypassed    bool       `json:"push_protection_bypassed"`
 	Push_protection_bypassed_at string     `json:"push_protection_bypassed_at"`
 	Push_protection_bypassed_by User       `json:"push_protection_bypassed_by"`
+	Validity_github             string     `json:"validity"`
 	Validity_boolean            bool       `json:"validity_boolean"`
 	Validity_response_code      string     `json:"validity_response_code"`
 	Validity_endpoint           string     `json:"validity_endpoint"`
@@ -228,22 +229,14 @@ func addRepoFullNameToAlerts(alerts []Alert) []Alert {
 
 func getSecretTypeParameter() (secret_type_param string) {
 	// if provider was specified, only return the secret types for that provider:
+	secret_type_param = "all"
 	if provider != "" {
 		// get keys for SupportedProviders[provider] map:
 		var secret_types []string
 		for key := range SupportedProviders[provider] {
 			secret_types = append(secret_types, key)
 		}
 		secret_type_param = strings.Join(secret_types, ",")
-	} else {
-		// get keys for every SupportedProvider key:
-		var secret_types []string
-		for _, supported_provider := range SupportedProviders {
-			for key := range supported_provider {
-				secret_types = append(secret_types, key)
-			}
-		}
-		secret_type_param = strings.Join(secret_types, ",")
 	}
 	return secret_type_param
 }
@@ -272,6 +265,7 @@ func prettyPrintAlerts(alerts []Alert, validity_check bool) (err error) {
 		t.AddField("ID", tableprinter.WithColor(Green), tableprinter.WithTruncate(nil))
 		t.AddField("State", tableprinter.WithColor(Green), tableprinter.WithTruncate(nil))
 		t.AddField("Secret Type", tableprinter.WithColor(Green), tableprinter.WithTruncate(nil))
+		t.AddField("Validity GitHub", tableprinter.WithColor(Green), tableprinter.WithTruncate(nil))
 		if secret {
 			t.AddField("Secret", tableprinter.WithColor(Green), tableprinter.WithTruncate(nil))
 		}
@@ -304,6 +298,7 @@ func prettyPrintAlerts(alerts []Alert, validity_check bool) (err error) {
 			t.AddField(strconv.Itoa(alert.Number), tableprinter.WithColor(color), tableprinter.WithTruncate(nil))
 			t.AddField(alert.State, tableprinter.WithColor(color), tableprinter.WithTruncate(nil))
 			t.AddField(alert.Secret_type, tableprinter.WithColor(color), tableprinter.WithTruncate(nil))
+			t.AddField(alert.Validity_github, tableprinter.WithColor(color), tableprinter.WithTruncate(nil))
 			if secret {
 				t.AddField(alert.Secret, tableprinter.WithColor(color), tableprinter.WithTruncate(nil))
 			}
@@ -344,8 +339,9 @@ func generateCSVReport(alerts []Alert, scope string, validity_check bool) (err e
 	counter := 0
 	// get current date & time:
 	now := time.Now()
-	timestamp := now.Format("2006-01-02 15-04-05")
-	filename := "Secret Scanning Report - " + scope + " - " + timestamp + ".csv"
+	// Format the time as YYYYMMDD-HHMMSS
+	timestamp := now.Format("20060102-150405")
+	filename := "secretscanningreport-" + scope + "-" + timestamp + ".csv"
 	// Create a CSV file
 	file, err := os.Create(filename)
 	if err != nil {
@@ -357,7 +353,7 @@ func generateCSVReport(alerts []Alert, scope string, validity_check bool) (err e
 	writer := csv.NewWriter(file)
 	defer writer.Flush()
 	// Write headers to CSV file
-	headers := []string{"Repository", "ID", "State", "Secret Type"}
+	headers := []string{"Repository", "ID", "State", "Secret Type", "GitHub Validity"}
 	if secret {
 		headers = append(headers, "Secret")
 	}
@@ -371,7 +367,7 @@ func generateCSVReport(alerts []Alert, scope string, validity_check bool) (err e
 	// Write data to CSV file
 	for counter < len(alerts) && counter < limit {
 		alert := alerts[counter]
-		row := []string{alert.Repository.Full_name, strconv.Itoa(alert.Number), alert.State, alert.Secret_type}
+		row := []string{alert.Repository.Full_name, strconv.Itoa(alert.Number), alert.State, alert.Secret_type, alert.Validity_github}
 		if secret {
 			row = append(row, alert.Secret)
 		}
@@ -393,9 +389,24 @@ func generateCSVReport(alerts []Alert, scope string, validity_check bool) (err e
 }
 
 func verifyAlerts(alerts []Alert) (alertsOutput []Alert, err error) {
+	// Print Supported providers for reference when verbose flag is enabled
+	if verbose {
+		fmt.Println(Blue("Supported Providers:"))
+		for provider, secretTypes := range SupportedProviders {
+			for secretType := range secretTypes {
+				fmt.Printf("- %s - %s\n", provider, secretType)
+			}
+		}
+	}
 	for i, alert := range alerts {
 		// verify that the alert is valid by making a request to its validation endpoint:
 		provider := strings.Split(alert.Secret_type, "_")[0]
+
+		// Skip alert if provider is not supported
+		if _, ok := SupportedProviders[provider]; !ok {
+			continue
+		}
+
 		secret_type := alert.Secret_type
 		secret_validation_method := SupportedProviders[provider][secret_type]["HttpMethod"]
 		secret_validation_content_type := SupportedProviders[provider][secret_type]["ContentType"]
diff --git a/cmd/verify.go b/cmd/verify.go
@@ -59,8 +59,10 @@ func runVerify(cmd *cobra.Command, args []string) (err error) {
 	values.Set("per_page", per_page)
 	// if provider was specified, filter results for just that provider. Otherwise, target all supported providers:
 	secret_type := getSecretTypeParameter()
-	values.Set("secret_type", secret_type)
-	parsedURL.RawQuery = values.Encode()
+	if secret_type != "all" {
+		values.Set("secret_type", secret_type)
+		parsedURL.RawQuery = values.Encode()
+	}
 
 	// update the request path
 	requestPath = parsedURL.String()
diff --git a/go.mod b/go.mod
@@ -1,6 +1,6 @@
 module github.com/CallMeGreg/gh-secret-scanning
 
-go 1.19
+go 1.23
 
 require (
 	github.com/cli/go-gh v1.2.1
diff --git a/go.sum b/go.sum
@@ -11,12 +11,15 @@ github.com/cli/shurcooL-graphql v0.0.4/go.mod h1:3waN4u02FiZivIV+p1y4d0Jo1jc6BVi
 github.com/cpuguy83/go-md2man/v2 v2.0.3/go.mod h1:tgQtvFlXSQOSOSIRvRPT7W67SCa46tRHOmNcaadrF8o=
 github.com/creack/pty v1.1.9/go.mod h1:oKZEueFk5CKHvIhNR5MUki03XCEU+Q6VDXinZuGJ33E=
 github.com/davecgh/go-spew v1.1.1 h1:vj9j/u1bqnvCEfJOwUhtlOARqs3+rkHYY13jYWTU97c=
+github.com/davecgh/go-spew v1.1.1/go.mod h1:J7Y8YcW2NihsgmVo/mv3lAwl/skON4iLHjSsI+c5H38=
 github.com/h2non/parth v0.0.0-20190131123155-b4df798d6542 h1:2VTzZjLZBgl62/EtslCrtky5vbi9dd7HrQPQIx6wqiw=
+github.com/h2non/parth v0.0.0-20190131123155-b4df798d6542/go.mod h1:Ow0tF8D4Kplbc8s8sSb3V2oUCygFHVp8gC3Dn6U4MNI=
 github.com/henvic/httpretty v0.0.6 h1:JdzGzKZBajBfnvlMALXXMVQWxWMF/ofTy8C3/OSUTxs=
 github.com/henvic/httpretty v0.0.6/go.mod h1:X38wLjWXHkXT7r2+uK8LjCMne9rsuNaBLJ+5cU2/Pmo=
 github.com/inconshreveable/mousetrap v1.1.0 h1:wN+x4NVGpMsO7ErUn/mUI3vEoE6Jt13X2s0bqwp9tc8=
 github.com/inconshreveable/mousetrap v1.1.0/go.mod h1:vpF70FUmC8bwa3OWnCshd2FqLfsEA9PFc4w1p2J65bw=
 github.com/kr/pretty v0.1.0 h1:L/CwN0zerZDmRFUapSPitk6f+Q3+0za1rQkzVuMiMFI=
+github.com/kr/pretty v0.1.0/go.mod h1:dAy3ld7l9f0ibDNOQOHHMYYIIbhfbHSm3C4ZsoJORNo=
 github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
 github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
 github.com/lucasb-eyer/go-colorful v1.2.0 h1:1nnpGOrhyZZuNyfu1QjKiUICQ74+3FNCN69Aj6K7nkY=
@@ -32,6 +35,7 @@ github.com/muesli/reflow v0.3.0/go.mod h1:pbwTDkVPibjO2kyvBQRBxTWEEGDGq0FlB1BIKt
 github.com/muesli/termenv v0.13.0 h1:wK20DRpJdDX8b7Ek2QfhvqhRQFZ237RGRO0RQ/Iqdy0=
 github.com/muesli/termenv v0.13.0/go.mod h1:sP1+uffeLaEYpyOTb8pLCUctGcGLnoFjSn4YJK5e2bc=
 github.com/pmezard/go-difflib v1.0.0 h1:4DBwDE0NGyQoBHbLQYPwSUPoCMWR5BEzIk/f1lZbAQM=
+github.com/pmezard/go-difflib v1.0.0/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
 github.com/rivo/uniseg v0.1.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
 github.com/rivo/uniseg v0.2.0/go.mod h1:J6wj4VEh+S6ZtnVlnTBMWIodfgj8LQOQFoIToxlJtxc=
 github.com/rivo/uniseg v0.4.4 h1:8TfxU8dW6PdqD27gjM8MVNuicgxIjxpm4K7x4jp8sis=
@@ -42,6 +46,7 @@ github.com/spf13/cobra v1.8.0/go.mod h1:WXLWApfZ71AjXPya3WOlMsY9yMs7YeiHhFVlvLyh
 github.com/spf13/pflag v1.0.5 h1:iy+VFUOCP1a+8yFto/drg2CJ5u0yRoB7fZw3DKv/JXA=
 github.com/spf13/pflag v1.0.5/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
 github.com/stretchr/testify v1.7.0 h1:nwc3DEeHmmLAfoZucVR881uASk0Mfjw8xYJ99tb5CcY=
+github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
 github.com/thlib/go-timezone-local v0.0.0-20210907160436-ef149e42d28e h1:BuzhfgfWQbX0dWzYzT1zsORLnHRv3bcRcsaUk0VmXA8=
 github.com/thlib/go-timezone-local v0.0.0-20210907160436-ef149e42d28e/go.mod h1:/Tnicc6m/lsJE0irFMA0LfIwTBo4QP7A8IfyIv4zZKI=
 golang.org/x/sys v0.0.0-20210831042530-f4d43177bf5e/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
@@ -55,6 +60,8 @@ golang.org/x/text v0.13.0 h1:ablQoSUd0tRdKxZewP80B+BaqeKJuVhuRxj/dkrun3k=
 golang.org/x/text v0.13.0/go.mod h1:TvPlkZtksWOMsz7fbANvkp4WM8x/WCo/om8BMLbz+aE=
 gopkg.in/check.v1 v0.0.0-20161208181325-20d25e280405/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127 h1:qIbj1fsPNlZgppZ+VLlY7N33q108Sa+fhmuc+sWQYwY=
+gopkg.in/check.v1 v1.0.0-20180628173108-788fd7840127/go.mod h1:Co6ibVJAznAaIkqp8huTwlJQCZ016jof/cbN4VW5Yz0=
 gopkg.in/h2non/gock.v1 v1.1.2 h1:jBbHXgGBK/AoPVfJh5x4r/WxIrElvbLel8TCZkkZJoY=
+gopkg.in/h2non/gock.v1 v1.1.2/go.mod h1:n7UGz/ckNChHiK05rDoiC4MYSunEC/lyaUm2WWaDva0=
 gopkg.in/yaml.v3 v3.0.1 h1:fxVm/GzAzEWqLHuvctI91KS9hhNmmWOoWu0XTYJS7CA=
 gopkg.in/yaml.v3 v3.0.1/go.mod h1:K4uyk7z7BCEPqu6E+C64Yfv1cQ7kz7rIZviUmN+EgEM=
