Skip to content

Commit 6056deb

Browse files
teodora-sanduteodora-sandu
authored
chore: improve local dev and ci/cd (#34)
1 parent 24a35e8 commit 6056deb

File tree

20 files changed

+209
-64
lines changed

20 files changed

+209
-64
lines changed

‎.circleci/config.yml

Lines changed: 22 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -18,11 +18,14 @@ jobs:
1818
name: Install tools
1919
command: make tools
2020
- run: make format
21+
- run: make generate
2122
- run:
22-
name: "Ensure go.mod is tidy"
23+
name: Run go mod tidy"
24+
command: go mod tidy
25+
- run:
26+
name: Check if there are any changes
2327
command: |
24-
go mod tidy
25-
git diff --exit-code go.mod go.sum
28+
git status --porcelain=v1 | tee /dev/stderr | wc -l | grep -qE '^ *0 *$$'
2629
- run: make lint
2730
unit_test:
2831
executor: default
@@ -44,6 +47,16 @@ jobs:
4447
- run:
4548
name: Run smoke tests
4649
command: make smoke-test
50+
contract_test:
51+
executor: default
52+
steps:
53+
- checkout
54+
- run:
55+
name: Install tools
56+
command: make tools
57+
- run:
58+
name: Run smoke tests
59+
command: make contract-test
4760
build:
4861
executor: default
4962
steps:
@@ -73,6 +86,7 @@ workflows:
7386
context: code-client-go
7487
- lint-and-format:
7588
name: Lint & Format
89+
context: code-client-go
7690
- unit_test:
7791
name: Unit tests
7892
requires:
@@ -83,9 +97,14 @@ workflows:
8397
- code-client-go-smoke-tests-token
8498
requires:
8599
- Unit tests
100+
- contract_test:
101+
name: Contract tests
102+
requires:
103+
- Unit tests
86104
- build:
87105
name: Build
88106
requires:
89107
- Smoke tests
108+
- Contract tests
90109
- Security Scans
91110
- Scan repository for secrets

‎.github/pull_request_template.md

Lines changed: 11 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,11 @@
1+
### Description
2+
3+
_Provide description of this PR and changes, if linked Jira ticket doesn't cover it in full._
4+
5+
### Checklist
6+
7+
- [ ] Tests added and all succeed
8+
- [ ] Linted
9+
- [ ] README.md updated, if user-facing
10+
11+
🚨After having merged, please update the `snyk-ls` and CLI go.mod to pull in latest client.

‎.github/workflows/release.yml

Lines changed: 21 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,8 +5,29 @@ on:
55
- 'main'
66

77
jobs:
8+
unit-tests:
9+
name: unit tests
10+
runs-on: ubuntu-latest
11+
steps:
12+
- uses: actions/checkout@v3
13+
14+
- name: Set up Go
15+
uses: actions/setup-go@v3
16+
with:
17+
go-version-file: './go.mod'
18+
19+
- name: Lint
20+
run: |
21+
make lint
22+
23+
- name: Test
24+
run: |
25+
make test
26+
827
build:
928
runs-on: ubuntu-latest
29+
needs:
30+
- unit-tests
1031
steps:
1132
- uses: actions/checkout@v3
1233
with:

‎CONTRIBUTING.md

Lines changed: 38 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,4 @@
1+
12
# Contributing
23

34
> This guide is for internal Snyk contributors with write access to this repository. If you are an external contributor, before working on any contributions, please first [contact support](https://support.snyk.io) to discuss the issue or feature request with us.
@@ -43,14 +44,21 @@ In order to update the clients you need access to some private Snyk repositories
4344
export GITHUB_PAT=<GITHUB_PAT>
4445
```
4546

46-
Also make sure to have `python3` installed.
47+
Also make sure to have `python3` installed and dependencies:
48+
```sh
49+
brew install python
50+
python3 -m pip install yq --break-system-packages
51+
python3 -m pip install pycurl --break-system-packages
52+
```
4753

4854
Then run `make download-apis` to download the latest code. If you want to download a new version, you can also do that.
4955

5056
Then generate the client by running `make generate`.
5157

5258
## Running tests
5359

60+
### Unit tests
61+
5462
To run the tests run:
5563

5664
```sh
@@ -59,11 +67,37 @@ make test
5967

6068
If writing unit tests, use the mocks generated by [GoMock](https://github.com/golang/mock) by running `make generate`.
6169

62-
If writing `pact`, integration, or smoke tests, use the test implementations in [./internal/util/testutil](./internal/util/testutil).
70+
If you've changed any of the interfaces you may need to re-run `make generate` to generate the mocks again.
6371

64-
The organisation used by the smoke tests is `ide-consistent-ignores-test` in [https://app.dev.snyk.io](https://app.dev.snyk.io) and we are authenticating using a service account api key.
72+
### Contract tests
6573

66-
If you've changed any of the interfaces you may need to re-run `make generate` to generate the mocks again.
74+
To run the tests run:
75+
76+
```sh
77+
make contract-test
78+
```
79+
80+
These tests use `pact` and the name of each test should start with `TestPact`. The contracts end up in [https://snyk-dev.pactflow.io/](https://snyk-dev.pactflow.io/).
81+
82+
If writing contract tests, use the test implementations in [./internal/util/testutil](./internal/util/testutil) and
83+
make sure that at the top of each file is the following line `//go:build CONTRACT`.
84+
85+
These tests are used then in the CI/CD as a gate for the PR.
86+
87+
### Smoke tests
88+
89+
To run the tests make sure to set the `SMOKE_TEST_TOKEN` environment variable to an API key for an org that has the
90+
consistent ignores feature flag enabled.
91+
92+
Then run:
93+
94+
```sh
95+
make smoke-test
96+
```
97+
98+
If writing smoke tests, do not use the test implementations and make sure that at the top of each file is the following line `//go:build smoke`.
99+
100+
The organisation used by the smoke tests is `ide-consistent-ignores-test` in [https://app.dev.snyk.io](https://app.dev.snyk.io) and we are authenticating using a service account api key.
67101

68102
## Code ownership
69103

‎Makefile

Lines changed: 30 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -56,13 +56,32 @@ testv:
5656
@echo "Testing verbosely..."
5757
@go test -v
5858

59+
.PHONY: contract-test
60+
contract-test: $(TOOLS_BIN)/pact
61+
@echo "Contract testing..."
62+
@go test -tags=CONTRACT ./...
63+
5964
.PHONY: smoke-test
6065
smoke-test:
61-
@go test -run="Test_SmokeScan"
66+
@echo "Smoke testing..."
67+
@go test -tags=SMOKE
6268

6369
.PHONY: generate
64-
generate: $(TOOLS_BIN)/go/mockgen $(TOOLS_BIN)/go/oapi-codegen
65-
@go generate ./...
70+
generate:
71+
ifdef CI
72+
$(MAKE) generate-mocks
73+
else
74+
$(MAKE) generate-mocks
75+
$(MAKE) generate-apis
76+
endif
77+
78+
.PHONY: generate-mocks
79+
generate-mocks: $(TOOLS_BIN)/go/mockgen
80+
@go generate -tags MOCK ./...
81+
82+
.PHONY: generate-apis
83+
generate-apis: $(TOOLS_BIN)/go/oapi-codegen download-apis
84+
@go generate -tags API,!MOCK ./...
6685

6786
.PHONY: download-apis
6887
download-apis: download-workspace-api download-orchestration-api
@@ -82,6 +101,13 @@ help:
82101
@echo "$(LOG_PREFIX) lint"
83102
@echo "$(LOG_PREFIX) build"
84103
@echo "$(LOG_PREFIX) test"
85-
@echo "$(LOG_PREFIX) testv Test versbosely"
104+
@echo "$(LOG_PREFIX) testv Test verbosely"
105+
@echo "$(LOG_PREFIX) smoke-test"
106+
@echo "$(LOG_PREFIX) generate"
107+
@echo "$(LOG_PREFIX) generate-mocks"
108+
@echo "$(LOG_PREFIX) generate-apis"
109+
@echo "$(LOG_PREFIX) download-apis"
110+
@echo "$(LOG_PREFIX) download-workspace-api"
111+
@echo "$(LOG_PREFIX) download-orchestration-api"
86112
@echo "$(LOG_PREFIX) GOOS Specify Operating System to compile for (see golang GOOS, default=$(GOOS))"
87113
@echo "$(LOG_PREFIX) GOARCH Specify Architecture to compile for (see golang GOARCH, default=$(GOARCH))"

‎internal/bundle/bundle_manager_test.go

Lines changed: 10 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ func Test_Create(t *testing.T) {
4242
ctrl := gomock.NewController(t)
4343
mockSpan := mocks.NewMockSpan(ctrl)
4444
mockSpan.EXPECT().Context().AnyTimes()
45-
mockSnykCodeClient := deepcodeMocks.NewMockDeepcode(ctrl)
45+
mockSnykCodeClient := deepcodeMocks.NewMockDeepcodeClient(ctrl)
4646
mockSnykCodeClient.EXPECT().GetFilters(gomock.Any()).Return(deepcode.FiltersResponse{
4747
ConfigFiles: []string{},
4848
Extensions: []string{".java"},
@@ -77,7 +77,7 @@ func Test_Create(t *testing.T) {
7777
ctrl := gomock.NewController(t)
7878
mockSpan := mocks.NewMockSpan(ctrl)
7979
mockSpan.EXPECT().Context().AnyTimes()
80-
mockSnykCodeClient := deepcodeMocks.NewMockDeepcode(ctrl)
80+
mockSnykCodeClient := deepcodeMocks.NewMockDeepcodeClient(ctrl)
8181
mockSnykCodeClient.EXPECT().GetFilters(gomock.Any()).Return(deepcode.FiltersResponse{
8282
ConfigFiles: []string{},
8383
Extensions: []string{".java"},
@@ -110,7 +110,7 @@ func Test_Create(t *testing.T) {
110110
ctrl := gomock.NewController(t)
111111
mockSpan := mocks.NewMockSpan(ctrl)
112112
mockSpan.EXPECT().Context().AnyTimes()
113-
mockSnykCodeClient := deepcodeMocks.NewMockDeepcode(ctrl)
113+
mockSnykCodeClient := deepcodeMocks.NewMockDeepcodeClient(ctrl)
114114
mockSnykCodeClient.EXPECT().GetFilters(gomock.Any()).Return(deepcode.FiltersResponse{
115115
ConfigFiles: []string{},
116116
Extensions: []string{".java"},
@@ -147,7 +147,7 @@ func Test_Create(t *testing.T) {
147147
ctrl := gomock.NewController(t)
148148
mockSpan := mocks.NewMockSpan(ctrl)
149149
mockSpan.EXPECT().Context().AnyTimes()
150-
mockSnykCodeClient := deepcodeMocks.NewMockDeepcode(ctrl)
150+
mockSnykCodeClient := deepcodeMocks.NewMockDeepcodeClient(ctrl)
151151
mockSnykCodeClient.EXPECT().GetFilters(gomock.Any()).Return(deepcode.FiltersResponse{
152152
ConfigFiles: []string{},
153153
Extensions: []string{".java"},
@@ -181,7 +181,7 @@ func Test_Create(t *testing.T) {
181181
ctrl := gomock.NewController(t)
182182
mockSpan := mocks.NewMockSpan(ctrl)
183183
mockSpan.EXPECT().Context().AnyTimes()
184-
mockSnykCodeClient := deepcodeMocks.NewMockDeepcode(ctrl)
184+
mockSnykCodeClient := deepcodeMocks.NewMockDeepcodeClient(ctrl)
185185
mockSnykCodeClient.EXPECT().GetFilters(gomock.Any()).Return(deepcode.FiltersResponse{
186186
ConfigFiles: []string{".test"},
187187
Extensions: []string{},
@@ -214,7 +214,7 @@ func Test_Create(t *testing.T) {
214214
ctrl := gomock.NewController(t)
215215
mockSpan := mocks.NewMockSpan(ctrl)
216216
mockSpan.EXPECT().Context().AnyTimes()
217-
mockSnykCodeClient := deepcodeMocks.NewMockDeepcode(ctrl)
217+
mockSnykCodeClient := deepcodeMocks.NewMockDeepcodeClient(ctrl)
218218
mockSnykCodeClient.EXPECT().GetFilters(gomock.Any()).Return(deepcode.FiltersResponse{
219219
ConfigFiles: []string{},
220220
Extensions: []string{".java"},
@@ -272,7 +272,7 @@ func Test_Upload(t *testing.T) {
272272
ctrl := gomock.NewController(t)
273273
mockSpan := mocks.NewMockSpan(ctrl)
274274
mockSpan.EXPECT().Context().AnyTimes()
275-
mockSnykCodeClient := deepcodeMocks.NewMockDeepcode(ctrl)
275+
mockSnykCodeClient := deepcodeMocks.NewMockDeepcodeClient(ctrl)
276276
mockSnykCodeClient.EXPECT().ExtendBundle(gomock.Any(), "bundleHash", gomock.Len(1), []string{}).Times(1)
277277
mockInstrumentor := mocks.NewMockInstrumentor(ctrl)
278278
mockInstrumentor.EXPECT().StartSpan(gomock.Any(), gomock.Any()).Return(mockSpan).Times(2)
@@ -296,7 +296,7 @@ func Test_Upload(t *testing.T) {
296296
ctrl := gomock.NewController(t)
297297
mockSpan := mocks.NewMockSpan(ctrl)
298298
mockSpan.EXPECT().Context().AnyTimes()
299-
mockSnykCodeClient := deepcodeMocks.NewMockDeepcode(ctrl)
299+
mockSnykCodeClient := deepcodeMocks.NewMockDeepcodeClient(ctrl)
300300
mockSnykCodeClient.EXPECT().ExtendBundle(gomock.Any(), "bundleHash", gomock.Len(3), []string{}).Return("newBundleHash", []string{}, nil).Times(1)
301301
mockSnykCodeClient.EXPECT().ExtendBundle(gomock.Any(), "newBundleHash", gomock.Len(2), []string{}).Return("newerBundleHash", []string{}, nil).Times(1)
302302
mockInstrumentor := mocks.NewMockInstrumentor(ctrl)
@@ -341,7 +341,7 @@ func createTempFileInDir(t *testing.T, name string, size int, temporaryDir strin
341341

342342
func Test_IsSupported_Extensions(t *testing.T) {
343343
ctrl := gomock.NewController(t)
344-
mockSnykCodeClient := deepcodeMocks.NewMockDeepcode(ctrl)
344+
mockSnykCodeClient := deepcodeMocks.NewMockDeepcodeClient(ctrl)
345345
mockSnykCodeClient.EXPECT().GetFilters(gomock.Any()).Return(deepcode.FiltersResponse{
346346
ConfigFiles: []string{},
347347
Extensions: []string{".java"},
@@ -380,7 +380,7 @@ func Test_IsSupported_ConfigFiles(t *testing.T) {
380380
}
381381

382382
ctrl := gomock.NewController(t)
383-
mockSnykCodeClient := deepcodeMocks.NewMockDeepcode(ctrl)
383+
mockSnykCodeClient := deepcodeMocks.NewMockDeepcodeClient(ctrl)
384384
mockSnykCodeClient.EXPECT().GetFilters(gomock.Any()).Return(deepcode.FiltersResponse{
385385
ConfigFiles: configFilesFromFiltersEndpoint,
386386
Extensions: []string{},

‎internal/bundle/bundle_test.go

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -42,7 +42,7 @@ func Test_UploadBatch(t *testing.T) {
4242

4343
t.Run("when no documents - creates nothing", func(t *testing.T) {
4444
ctrl := gomock.NewController(t)
45-
mockSnykCodeClient := deepcodeMocks.NewMockDeepcode(ctrl)
45+
mockSnykCodeClient := deepcodeMocks.NewMockDeepcodeClient(ctrl)
4646

4747
mockSpan := mocks.NewMockSpan(ctrl)
4848
mockSpan.EXPECT().Context().AnyTimes()
@@ -59,7 +59,7 @@ func Test_UploadBatch(t *testing.T) {
5959

6060
t.Run("when no bundles - creates new deepCodeBundle and sets hash", func(t *testing.T) {
6161
ctrl := gomock.NewController(t)
62-
mockSnykCodeClient := deepcodeMocks.NewMockDeepcode(ctrl)
62+
mockSnykCodeClient := deepcodeMocks.NewMockDeepcodeClient(ctrl)
6363
mockSnykCodeClient.EXPECT().ExtendBundle(gomock.Any(), "testBundleHash", map[string]deepcode.BundleFile{
6464
"file": {},
6565
}, []string{}).Return("testBundleHash", []string{}, nil)
@@ -78,7 +78,7 @@ func Test_UploadBatch(t *testing.T) {
7878

7979
t.Run("when existing bundles - extends deepCodeBundle and updates hash", func(t *testing.T) {
8080
ctrl := gomock.NewController(t)
81-
mockSnykCodeClient := deepcodeMocks.NewMockDeepcode(ctrl)
81+
mockSnykCodeClient := deepcodeMocks.NewMockDeepcodeClient(ctrl)
8282
mockSnykCodeClient.EXPECT().ExtendBundle(gomock.Any(), "testBundleHash", map[string]deepcode.BundleFile{
8383
"another": {},
8484
"file": {},

‎internal/deepcode/client_pact_test.go

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,3 +1,5 @@
1+
//go:build CONTRACT
2+
13
/*
24
* © 2022-2024 Snyk Limited
35
*

0 commit comments

Comments
 (0)