v0.11.0

New Features

- (#119) RASP: add Shell Injection protection support. This protection is currently dynamically applied to `os.StartProcess()` which is the only entry point of the Go standard library to execute a process.  This protection can be configured at https://my.sqreen.com/application/goto/modules/rasp/details/shi.

- (#119) RASP: add Local File Inclusion protection support. This protection is currently dynamically applied to `os.Open()` which is the only entry point of the Go standard library to open a file for reading. This protection can be configured at https://my.sqreen.com/application/goto/modules/rasp/details/lfi.

- (#120) RASP: add Server-Side Request Forgery protection support. This protection is currently dynamically applied to `net/http.(*Client).do()` which is the only entry point of the Go standard library to perform an HTTP request. This protection can be configured at https://my.sqreen.com/application/goto/modules/rasp/details/ssrf.

- (#125) RASP: enable SQL Injection protection for every MySQL, Oracle, SQLite and PostgreSQL drivers listed in the Go language wiki page https://github.com/golang/go/wiki/SQLDrivers.

- (#115) RASP: store Sqreen's request protection context into the Goroutine Local Storage (GLS). Therefore, Sqreen can now protect every Go function without requiring the request Go context (eg. both `QueryContext()` and `Query()` can be now protected against SQL injections). For now, this protection context is only available in the goroutine handling the request, and sub-goroutines are not protected. Further support will be added very soon to remove this limitation.

- (#121) Add IP denylist support: block every request performed by an IP address of the denylist. Every usage of whitelist and blacklist in the agent was also removed when possible. The IP denylist can be configured at https://my.sqreen.com/application/goto/settings/denylist.

- (#122) Add path passlist support: requests performed on those paths are not monitored nor protected by Sqreen. The Path passlist can be configured at https://my.sqreen.com/application/goto/settings/passlist.

- (#123) Export the error type returned by Sqreen protections when blocking in the new SDK package `github.com/sqreen/go-agent/sdk/types` in order to avoid retrying blocked function calls (eg. avoid retrying a blocked SQL query). It must be used along with `errors.As()` to detect such cases. Read more at https://godoc.org/github.com/sqreen/go-agent/sdk/types.

- (#124) Allow to "quickly" remove the agent from a program by only removing it from the source code without disabling the program instrumentation. This is made possible by making the instrumentation fully autonomous to avoid compilation errors.

Fixes

- Gin Middleware: fix the HTTP status code monitoring that was possibly changed by Gin after having been already written.

Internal Changes

- (#126) Cache request value lookups, mainly to accelerate the In-App WAF when lots of rulesets are enabled.

- (#117) Simpler Go vendoring support implementation.

- (#113) Significant JavaScript performance improvements by changing the virtual machine to `github.com/dop251/goja`.

- (#114) Add Goroutine Local Storage (GLS) support through static instrumentation of the Go runtime.

Author: Julio Guerra

.github/workflows/ci-example-apps.yaml

@@ -0,0 +1,25 @@
+name: Example App Builds
+on:
+  push:
+  pull_request:
+  schedule:
+    - cron: "0 0 * * *"
+jobs:
+  docker:
+    name: Docker Examples
+    strategy:
+      matrix:
+        example: [ alpine, debian, scratch ]
+        go-version: [ rc, 1.14, 1.13, 1.12]
+        do-vendoring: [ true, false ]
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v2
+      - name: Setup go
+        uses: actions/setup-go@v2
+        if: ${{ matrix.do-vendoring }}
+      - run: go mod vendor
+        name: Vendor the dependencies
+        if: ${{ matrix.do-vendoring }}
+      - run: docker build -f ${{ matrix.example }}/Dockerfile -t hello-sqreen:${{ matrix.example }} --build-arg GO_VERSION=${{ matrix.go-version }} .
+        working-directory: examples/docker

CHANGELOG.md

@@ -1,3 +1,40 @@
+# v0.11.0
+
+## New Features
+
+- (#119) RASP: add Shell Injection protection support. This protection is currently dynamically applied to `os.StartProcess()` which is the only entry point of the Go standard library to execute a process.  This protection can be configured at https://my.sqreen.com/application/goto/modules/rasp/details/shi.
+
+- (#119) RASP: add Local File Inclusion protection support. This protection is currently dynamically applied to `os.Open()` which is the only entry point of the Go standard library to open a file for reading. This protection can be configured at https://my.sqreen.com/application/goto/modules/rasp/details/lfi.
+
+- (#120) RASP: add Server-Side Request Forgery protection support. This protection is currently dynamically applied to `net/http.(*Client).do()` which is the only entry point of the Go standard library to perform an HTTP request. This protection can be configured at https://my.sqreen.com/application/goto/modules/rasp/details/ssrf.
+
+- (#125) RASP: enable SQL Injection protection for every MySQL, Oracle, SQLite and PostgreSQL drivers listed in the Go language wiki page https://github.com/golang/go/wiki/SQLDrivers.
+
+- (#115) RASP: store Sqreen's request protection context into the Goroutine Local Storage (GLS). Therefore, Sqreen can now protect every Go function without requiring the request Go context (eg. both `QueryContext()` and `Query()` can be now protected against SQL injections). For now, this protection context is only available in the goroutine handling the request, and sub-goroutines are not protected. Further support will be added very soon to remove this limitation.
+
+- (#121) Add IP denylist support: block every request performed by an IP address of the denylist. Every usage of whitelist and blacklist in the agent was also removed when possible. The IP denylist can be configured at https://my.sqreen.com/application/goto/settings/denylist.
+
+- (#122) Add path passlist support: requests performed on those paths are not monitored nor protected by Sqreen. The Path passlist can be configured at https://my.sqreen.com/application/goto/settings/passlist.
+
+- (#123) Export the error type returned by Sqreen protections when blocking in the new SDK package `github.com/sqreen/go-agent/sdk/types` in order to avoid retrying blocked function calls (eg. avoid retrying a blocked SQL query). It must be used along with `errors.As()` to detect such cases. Read more at https://godoc.org/github.com/sqreen/go-agent/sdk/types.
+
+- (#124) Allow to "quickly" remove the agent from a program by only removing it from the source code without disabling the program instrumentation. This is made possible by making the instrumentation fully autonomous to avoid compilation errors.
+
+## Fix
+
+- Gin Middleware: fix the HTTP status code monitoring that was possibly changed by Gin after having been already written.
+
+## Internal Changes
+
+- (#126) Cache request value lookups, mainly to accelerate the In-App WAF when lots of rulesets are enabled.
+
+- (#117) Simpler Go vendoring support implementation.
+
+- (#113) Significant JavaScript performance improvements by changing the virtual machine to `github.com/dop251/goja`.
+
+- (#114) Add Goroutine Local Storage (GLS) support through static instrumentation of the Go runtime.
+
+
 # v0.10.1
 
 ## Fix

azure-pipelines.yml

@@ -1,3 +1,12 @@
+schedules:
+  # Nightly run
+  - cron: "0 0 * * *"
+    displayName: Daily midnight run
+    always: true
+    branches:
+      include:
+        - master
+
 trigger:
   - master
   - dev
@@ -41,11 +50,23 @@ jobs:
         inputs:
           version: $(GOVERSION)
       - task: Go@0
+        displayName: Build gotestsum
+        name: build_gotestsum
+        inputs:
+          command: 'build'
+          arguments: '-v gotest.tools/gotestsum'
+      - bash: |
+          go env
+          go build -v gotest.tools/gotestsum
+          ./gotestsum --junitfile report.xml ./...
+        name: go_test
         displayName: go test
-        continueOnError: true
+      - task: PublishTestResults@2
+        condition: succeededOrFailed()
         inputs:
-          command: 'test'
-          arguments: '-v ./...'
+          testRunner: JUnit
+          testResultsFiles: $(System.DefaultWorkingDirectory)/**/report.xml
+          platform: windows
 
   - job: MacOS
     pool:
@@ -54,11 +75,13 @@ jobs:
       - script: |
           go env
           clang -v
-          go build -v github.com/jstemmer/go-junit-report
-          go test -v ./... 2>&1 | ./go-junit-report > report.xml
+          go build -v gotest.tools/gotestsum
+          ./gotestsum --junitfile report.xml ./...
         name: go_test
         displayName: go test
       - task: PublishTestResults@2
+        condition: succeededOrFailed()
         inputs:
           testRunner: JUnit
           testResultsFiles: $(System.DefaultWorkingDirectory)/**/report.xml
+          platform: macos

examples/docker/README.md

@@ -23,7 +23,7 @@ protected by Sqreen.
 Build the docker image and tag it with the image name `hello-sqreen` by doing:
 
 ```console
-examples/docker $ docker build -t hello-sqreen:debian -f debian/Dockerfile .
+examples/docker $ docker build -t hello-sqreen -f debian/Dockerfile .
 ```
 
 #### Building the Alpine docker image example
@@ -47,7 +47,7 @@ by Sqreen.
 Build the docker image and tag it with the image name `hello-sqreen` by doing:
 
 ```console
-examples/docker $ docker build -t hello-sqreen -f alpine/Dockerfile .
+examples/docker $ docker build -t hello-sqreen -f scratch/Dockerfile .
 ```
 
 ### Running the docker image

examples/docker/alpine/Dockerfile

@@ -2,7 +2,8 @@
 # creating a final alpine docker image.
 
 # Build docker image
-FROM golang:1 AS build
+ARG GO_VERSION=1
+FROM golang:$GO_VERSION AS build
 # Workdir out of the GOPATH to enable the Go modules mode.
 WORKDIR /app
 COPY . .

examples/docker/debian/Dockerfile

@@ -2,7 +2,8 @@
 # creating a final debian docker image.
 
 # Build docker image
-FROM golang:1 AS build
+ARG GO_VERSION=1
+FROM golang:$GO_VERSION AS build
 # Workdir out of the GOPATH to enable the Go modules mode.
 WORKDIR /app
 COPY . .

examples/docker/go.mod

@@ -2,4 +2,4 @@ module sqreen-hello-http
 
 go 1.12
 
-require github.com/sqreen/go-agent v0.9.3
+require github.com/sqreen/go-agent latest

examples/docker/scratch/Dockerfile

@@ -3,7 +3,8 @@
 # image is absolutely empty and more files need to be copied in that case.
 
 # Build docker image
-FROM golang:1 AS build
+ARG GO_VERSION=1
+FROM golang:$GO_VERSION AS build
 # Workdir out of the GOPATH to enable the Go modules mode.
 WORKDIR /app
 COPY . .
@@ -18,7 +19,7 @@ RUN go build -v -a -toolexec $PWD/sqreen-instrumentation-tool -o hello-sqreen .
 # Now prepare a directory with the shared libraries the compiled program file
 # requires by using ldd:
 # 1. Install binutils for ldd
-RUN apt update && apt install binutils ca-certificates
+RUN apt update && apt install -y binutils ca-certificates
 # 2. Use ldd to list the shared libraries and copy them into deps/
 RUN ldd hello-sqreen | tr -s '[:blank:]' '\n' | grep '^/' | \
     xargs -I % sh -c 'mkdir -p $(dirname deps%); cp % deps%;'

examples/gcp-app-engine/go.mod

@@ -2,4 +2,4 @@ module sqreen-hello-http
 
 go 1.12
 
-require github.com/sqreen/go-agent v0.9.3
+require github.com/sqreen/go-agent latest

examples/heroku/go.mod

@@ -2,4 +2,4 @@ module sqreen-hello-http
 
 go 1.12
 
-require github.com/sqreen/go-agent v0.9.3
+require github.com/sqreen/go-agent latest