DevSecOps CICD pipeline demo using several technologies such as:
- Openshift Pipelines
- Openshift GitOps
- Openshift Advanced Cluster Security for Kubernetes
- Openshift Container Registry
- SonarQube
- Nexus
- JUnit
- Gogs
- Git Webhook
- Gatling
- Zap Proxy
On every push to the spring-petclinic git repository on Gogs git server, the following steps are executed within the Tekton pipeline:
- Code is cloned from Gogs git server and the unit-tests are run
- Dependency report from the source code is generated and uploaded to the report server repository.
- Unit tests are executed and in parallel the code is analyzed by Sonarqube for anti-patterns.
- Application is packaged as a JAR and released to Sonatype Nexus snapshot repository
- A container image is built in DEV environment using S2I, and pushed to OpenShift internal registry, and tagged with spring-petclinic:[branch]-[commit-sha] and spring-petclinic:latest
Advanced Cluster Management for Kubernetes controls clusters and applications from a single console, with built-in security policies.
Using roxctl and ACS API, we integrated in our pipeline several additional security steps into our DevSecOps pipeline:
- Image Scanning using ACS Scanner of the image generated and pushed in step 4.
- Image Check of the build-time violations of the different security policies defined in ACS
- Checks build-time and deploy-time violations of security policies in ACS of the YAML deployment files used for deploy our application.
NOTE: these 3 steps are executed in parallel for saving time in our DevSecOps pipeline.
- Kubernetes kustomization files updated with the latest image [commit-sha] in the overlays for dev. This will ensure that our Application are deployed using the specific built image in this pipeline.
Argo CD continuously monitor the configurations stored in the Git repository and uses Kustomize to overlay environment specific configurations when deploying the application to DEV and STAGE environments.
- The ArgoCD applications syncs the manifests in our gogs git repositories, and applies the changes automatically into the namespaces defined:
and deploys every manifest that is defined in the branch/repo of our application:
Once our application is deployed, we need to ensure of our application is stable and performant and also that nobody can hack our application easily.
- Our CI in Openshift Pipelines waits until the ArgoCD app is fully sync and our app and all the resources are deployed
- The performance tests are cloned into our pipeline workspace
- The pentesting is executed using the web scanner OWASP Zap Proxy using a baseline in order to check the possible vulnerabilities, and a Zap Proxy report is uploaded to the report server repository.
- In parallel the performance tests are executed using the load test Gatling and a performance report is uploaded to the report server repository.
ACS can be integrated with several Notifier for notify if certain events happened in the clusters managed. In our case, we integrated with Slack in order to receive notifications when some Policies are violated in order to have more useful information:
These policies notification can be enabled by each system policy enabled in our system, so you can create your own notification baseline in order to have only the proper information received in your systems.
NOTE: By now the integration is manual. WIP to automate it.
In this demo, we can control the security policies applied into our pipelines, scanning the images and analysing the different deployments templates used for deploy our applications.
We can enforce the different Security Policies in ACS, failing our CI pipelines if a violation of this policy appears in each step of our DevSecOps pipelines (steps 6,7,8).
This Security Policies can be defined at BUILD level (during the build/push of the image), or at DEPLOYMENT level (preventing to deploy the application).
For example this Security Policy, checks if a RH Package Manager (dnf,yum) is installed in your Image, and will FAIL the pipeline if detects that the image built contains any RH Package Manager:
This ensures that we have the total control of our pipelines, and no image is pushed into your registry or deployed in your system that surpases the Security Policies defined.
- Openshift Cluster 4.7+
- Ansible 2.7+
ansible-galaxy collection install community.kubernetes
Fully automated deployment and integration of every resource and tool needed for this demo.
./install.sh
IMPORTANT: this is a working progress, be aware that some parts could not work as expected. PR are welcome :)
NOTE: you want to test the full end2end install using Ansible? Check out the develop branch!
Check the resources deployed for this demo with:
./status.sh
- Gogs git server (username/password: gogs/gogs)
- Sonatype Nexus (username/password: admin/admin123)
- SonarQube (username/password: admin/admin)
- Argo CD (username/password: admin/[Login with OAuth using Dex])
- ACS (username/password: admin/stackrox)
- Repository Server (username/password: reports/reports)
cd ..
./demo.sh start
NOTE: This pipeline will fail if you don't disable the "Fixable CVSS >= 7" policy enforcement behaviour of ACS. This is expected to demonstrate the failure when a violation of the system policy occurs.
This repo is heavily based in the CICD repository and work of Siamak Sadeghianfar. Kudos to Siamak!
Big thanks also to Rodrigo Alvares that helped with their wisdom and knowledge in this demo.
- Add documentation about triggers
- Add better branching with GitHub Flow model
- Update images for the infra (nexus, gogs, etc) with the latest versions
- Use Nexus Operator
- Use Quay Operator and Clair