Skip to content

Commit 8a46a37

Browse files
michelkaporinmichelkaporin
authored
Merge pull request #143 from dylan-asos/feat/failureThreshold
feat: supporting failOnThreshold
2 parents 3936051 + 1725a15 commit 8a46a37

11 files changed

+304
-53
lines changed

‎.github/CONTRIBUTING.md

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,38 @@ Unit tests can be run via `npm run test:unit` command.
99

1010
To run the code, a GitHub PR against `develop` should be raised with the committed code to the branch PR. The PR runs deployment script with deploy to development environment. The script builds the code that's added as part of your change and installs it in Azure DevOps organization as an extension that can be added to run a pipeline.
1111

12+
### Local debugging
13+
14+
A number of environment variable are required for debugging, here's an example launch config for `VSCode` that sets mandatory parameters such as `AGENT_TEMPDIRECTORY`, `INPUT_failOnIssues` and `INPUT_authToken`
15+
16+
```
17+
{
18+
"version": "0.2.0",
19+
"configurations": [
20+
{
21+
"type": "node",
22+
"request": "launch",
23+
"name": "Launch Program",
24+
"program": "${workspaceFolder}/snykTask/src/index.ts",
25+
"env": {
26+
"AGENT_TEMPDIRECTORY": "some/temp/path",
27+
"INPUT_failOnIssues": "true",
28+
"INPUT_authToken": "your-auth-token-guid-from-portal",
29+
"INPUT_targetFile": "path-to-visual-studio-solution.sln",
30+
"INPUT_organization": "your-org-guid-from-portal",
31+
"INPUT_monitorWhen": "never",
32+
"INPUT_severityThreshold": "low",
33+
"INPUT_failOnThreshold": "critical",
34+
"NODE_OPTIONS": null
35+
},
36+
"outFiles": [
37+
"${workspaceFolder}/**/*.js"
38+
]
39+
}
40+
]
41+
}
42+
```
43+
1244
## Release
1345
The release process is fully-automated: all you need to do is create a PR to merge `develop` into `master` and call the PR `Merge develop into master for release`.
1446

@@ -48,6 +80,8 @@ To release a major you need to add `BREAKING CHANGE: ` to the start of the body
4880

4981
Ensure that your code adheres to the included `.eslintrc` config by running `npm run test:checks`.
5082

83+
Fix any `prettier` violations reported before pushing by running `npm run format`
84+
5185
## Sending pull requests
5286

5387
- add tests for newly added code (and try to mirror directory and file structure if possible) or fixes

‎README.md

Lines changed: 29 additions & 15 deletions
Original file line numberDiff line numberDiff line change
@@ -19,21 +19,22 @@ This extension requires that Node.js and npm be installed on the build agent. Th
1919

2020
## Task Parameters
2121

22-
| Parameter | Description | Required | Default | Type |
23-
| ------------------------- | ------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------- | ------------- | --------------------------------------------------------------------------------- |
24-
| serviceConnectionEndpoint | The Azure DevOps service connection endpoint where your Snyk API token is defined. Define this within your Azure DevOps project settings / S | no | none | String / Azure Service Connection Endpoint of type SnykAuth / Snyk Authentication |
25-
| testType | Used by the task UI only | no | "application" | string: "app" or "container" |
26-
| dockerImageName | The name of the container image to test. | yes, if container image test | none | string |
27-
| dockerfilePath | The path to the Dockerfile corresponding to the `dockerImageName` | yes, if container image test | none | string |
28-
| targetFile | Applicable to application type tests ony. The path to the manifest file to be used by Snyk. Should only be provided if non-standard. | no | none | string |
29-
| severityThreshold | The severity-threshold to use when testing. By default, issues of all severity types will be found. | no | "low" | string: "low" or "medium" or "high" or "critical" |
30-
| monitorWhen | When to run `snyk monitor`. Valid options are `always` (default), `noIssuesFound`, and `never`. If set, this option overrides the value of `monitorOnBuild`. | no | "always" | boolean |
31-
| failOnIssues | This specifies if builds should be failed or continued based on issues found by Snyk. | yes | true | boolean |
32-
| projectName | A custom name for the Snyk project to be created on snyk.io | no | none | string |
33-
| organization | Name of the Snyk organisation name, under which this project should be tested and monitored | no | none | string |
34-
| testDirectory | Alternate working directory. For example, if you want to test a manifest file in a directory other than the root of your repo, you would put in relative path to that directory. | no | none | string |
35-
| ignoreUnknownCA | Use to ignore unknown or self-signed certificates. This might be useful in for self-hosted build agents with unusual network configurations or for Snyk on-prem installs configured with a self-signed certificate. | no | false | boolean |
36-
| additionalArguments | Additional Snyk CLI arguments to be passed in. Refer to the Snyk CLI help page for information on additional arguments. | no | none | string |
22+
| Parameter | Description | Required | Default | Type |
23+
| ------------------------- | -------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | ---------------------------- | ------------- | --------------------------------------------------------------------------------- |
24+
| serviceConnectionEndpoint | The Azure DevOps service connection endpoint where your Snyk API token is defined. Define this within your Azure DevOps project settings / S | no | none | String / Azure Service Connection Endpoint of type SnykAuth / Snyk Authentication |
25+
| testType | Used by the task UI only | no | "application" | string: "app" or "container" |
26+
| dockerImageName | The name of the container image to test. | yes, if container image test | none | string |
27+
| dockerfilePath | The path to the Dockerfile corresponding to the `dockerImageName` | yes, if container image test | none | string |
28+
| targetFile | Applicable to application type tests ony. The path to the manifest file to be used by Snyk. Should only be provided if non-standard. | no | none | string |
29+
| severityThreshold | The severity-threshold to use when testing and reporting. By default, issues of all severity types will be found. | no | "low" | string: "low" or "medium" or "high" or "critical" |
30+
| failOnThreshold | The `severityThreshold` parameter is used to control the interaction with the Snyk CLI and reporting vulnerabilities. The `failOnThreshold` gives you additional control over build failure behaviour. For example, with `failOnIssues` set to `true` and `failOnThreshold` to `critical`, all issues would be reported on but _only_ critical issues would cause a build failure. See Usage Examples for more information | no | "low" | string: "low" or "medium" or "high" or "critical" |
31+
| monitorWhen | When to run `snyk monitor`. Valid options are `always` (default), `noIssuesFound`, and `never`. If set, this option overrides the value of `monitorOnBuild`. | no | "always" | boolean |
32+
| failOnIssues | This specifies if builds should be failed or continued based on issues found by Snyk. Combine with `failOnThreshold` to control which severity of issues causes the build to fail | yes | true | boolean |
33+
| projectName | A custom name for the Snyk project to be created on snyk.io | no | none | string |
34+
| organization | Name of the Snyk organisation name, under which this project should be tested and monitored | no | none | string |
35+
| testDirectory | Alternate working directory. For example, if you want to test a manifest file in a directory other than the root of your repo, you would put in relative path to that directory. | no | none | string |
36+
| ignoreUnknownCA | Use to ignore unknown or self-signed certificates. This might be useful in for self-hosted build agents with unusual network configurations or for Snyk on-prem installs configured with a self-signed certificate. | no | false | boolean |
37+
| additionalArguments | Additional Snyk CLI arguments to be passed in. Refer to the Snyk CLI help page for information on additional arguments. | no | none | string |
3738

3839
## Usage Examples
3940

@@ -66,6 +67,19 @@ Here's a full example:
6667
monitorWhen: 'always'
6768
```
6869

70+
An example that specifies a value for `severityThreshold` as medium and configures `failOnThreshold` to critical. This configuration would _only fail_ the build when critical issues are found, but all issues detected at medium, high and critical would be reported back to your snyk project for analysis
71+
72+
```
73+
- task: SnykSecurityScan@1
74+
inputs:
75+
serviceConnectionEndpoint: 'mySnykToken'
76+
testType: 'app'
77+
severityThreshold: 'medium'
78+
failOnIssues: true
79+
failOnThreshold: 'critical'
80+
monitorWhen: 'always'
81+
```
82+
6983
### Simple Container Image Testing Example
7084

7185
```

‎snykTask/src/__tests__/task-lib-severity-threshold.test.ts

Lines changed: 0 additions & 20 deletions
This file was deleted.

‎snykTask/src/__tests__/task-lib.test.ts

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -14,6 +14,7 @@ import {
1414
formatDate,
1515
attachReport,
1616
removeRegexFromFile,
17+
doVulnerabilitiesExistForFailureThreshold,
1718
} from '../task-lib';
1819
import { TaskArgs } from '../task-args';
1920

@@ -45,6 +46,47 @@ test('getOptionsToExecuteSnyk builds IExecOptions like we need it', () => {
4546
expect(options.ignoreReturnCode).toBe(true);
4647
});
4748

49+
test('finds vulnerabilities greater than medium threshold in single-project results', () => {
50+
const fixturePath =
51+
'snykTask/test/fixtures/single-project-high-vulnerabilities.json';
52+
const itemsFound = doVulnerabilitiesExistForFailureThreshold(
53+
fixturePath,
54+
'medium',
55+
);
56+
57+
expect(itemsFound).toBe(true);
58+
});
59+
60+
test('finds vulnerabilities greater than medium threshold in multi-project results', () => {
61+
const fixturePath = 'snykTask/test/fixtures/high-vulnerabilities.json';
62+
const itemsFound = doVulnerabilitiesExistForFailureThreshold(
63+
fixturePath,
64+
'medium',
65+
);
66+
67+
expect(itemsFound).toBe(true);
68+
});
69+
70+
test('defaults to found when file does not exist', () => {
71+
const fixturePath = 'snykTask/test/fixtures/does-not-exist.json';
72+
const itemsFound = doVulnerabilitiesExistForFailureThreshold(
73+
fixturePath,
74+
'medium',
75+
);
76+
77+
expect(itemsFound).toBe(true);
78+
});
79+
80+
test('does not match vulnerabilities lower than high threshold', () => {
81+
const fixturePath = 'snykTask/test/fixtures/low-vulnerabilities.json';
82+
const itemsFound = doVulnerabilitiesExistForFailureThreshold(
83+
fixturePath,
84+
'high',
85+
);
86+
87+
expect(itemsFound).toBe(false);
88+
});
89+
4890
test('getOptionsToExecuteSnykCLICommand builds IExecOptions like we need it', () => {
4991
const taskNameForAnalytics = 'AZURE_PIPELINES';
5092
const version = '1.2.3';

0 commit comments

Comments
 (0)