Simplify / fix tests, CI

Author: dtrudg

.circleci/config.yml

@@ -8,13 +8,31 @@ workflows:
 jobs:
   build-and-test: 
     machine:
-      image: ubuntu-2004:202104-01
+      image: ubuntu-2004:current
     steps:
       - checkout
+      - run: 
+          name: install deps
+          command: |
+            sudo apt-get update
+            sudo apt-get -y install cryptsetup-bin squashfs-tools
+      - run: 
+          name: install singularity
+          command: |
+            wget https://github.com/sylabs/singularity/releases/download/v3.9.7/singularity-ce_3.9.7-focal_amd64.deb
+            sudo apt -y install ./singularity-ce_3.9.7-focal_amd64.deb
+            rm singularity-ce_3.9.7-focal_amd64.deb
       - run:
           name: install
-          command: python3 setup.py install
+          command: python3 setup.py install --user
+      - run:
+          name: start clair
+          command: |
+            docker pull arminc/clair-db:2021-06-14
+            docker run -d --name clair-db arminc/clair-db:2021-06-14
+            sleep 5
+            docker pull arminc/clair-local-scan:v2.1.7_5125fde67edee46cb058a3feee7164af9645e07d
+            docker run -p 6060:6060 --link clair-db:postgres -d --name clair arminc/clair-local-scan:v2.1.7_5125fde67edee46cb058a3feee7164af9645e07d
       - run:
           name: test
-          command: build_scripts/docker_local_tests.sh
-
+          command: python3 setup.py test

Dockerfile

@@ -2,9 +2,11 @@
 
 [![CircleCI](https://circleci.com/gh/dtrudg/clair-singularity/tree/master.svg?style=svg)](https://circleci.com/gh/dtrudg/clair-singularity/tree/master)
 
-__Scan [SingularityCE](http://sylabs.io/singularity/) container images for security vulnerabilities
+__Scan [Singularity](http://sylabs.io/singularity/) container images for security vulnerabilities
 using [CoreOS Clair](https://github.com/coreos/clair).__
 
+![screenshot](screenshot.png)
+
 The [CoreOS Clair vulnerability scanner](https://github.com/coreos/clair) is a useful tool able to scan docker and other container
 formats for security vulnerabilities. It obtains up-to-date lists of vulnerabilities for various
 platforms (namespaces) from public databases.
@@ -25,7 +27,7 @@ Based on experiments detailed [in this Gist](https://gist.github.com/dctrud/4797
 
 __IMPORTANT NOTES__
 
-CI tests usage with SingularityCE 3.9.7
+Funtionality was last tested using SingularityCE 3.9.7.
 
 This tool should be considered proof of concept, not heavily tested. Use at your own risk. 
 
@@ -39,7 +41,7 @@ Clair instance, or that only a trusted Clair instance can retrieve images from t
 To use clair-singularity you will need a _Linux_ host with:
 
   * Python 3.5 or greater installed
-  * Singularity 3+ installed (tested with 3.8.0) and the singularity executable in your `PATH`
+  * SingularityCE 3+ installed (tested with 3.9.7) and the singularity executable in your `PATH`
   * A Clair instance running somewhere, that is able to access the machine you will run 
   clair-singularity on. It's easiest to accomplish this using docker to run a local Clair instance as below.
 
@@ -77,15 +79,10 @@ python setup.py install
 
 __Clair on same machine__
 
-
-If you are running `clair-singularity` locally (outside of docker), and clair
-within docker, you need to tell `clair-singularity` to serve images on the main
-IP of your host, so that dockerized clair can access them.
-
 To scan a singularity image, using a clair instance running under local docker, on
 port 6060:
 
-    clair-singularity myimage.img
+    clair-singularity myimage.sif
 
 /If your hostname is not resolvable to a non-localhost IP of your machine, accessible to
 docker containers, you must specify the IP with `--bind-ip`/
@@ -105,20 +102,3 @@ By default, clair-singularity gives a simplified text report on STDOUT. To obtai
 report returned by Clair use the `--jsoon-output` option.
 
     clair-singularity --json-output myimage.img
-
-## Development / Testing
-
-Tests can are run in 3 different ways:
-
-__Local - no access to Clair__
-
-Runs all tests that don't depend on access to a Clair server, using the local Python.
-
-    $ build_scripts/noclair_local_tests.sh
-
-__Local - dockerized with Clair__
-
-Starts a Clair service with local docker, builds clair-singularity into a docker container, with Python 3.5, and
-runs tests in this docker container.
-
-    $ build_scripts/docker_local_tests.sh

README.md

@@ -97,7 +97,7 @@ def format_report_text(report):
                     vuln['Link'] + "\n" + vuln['Description']
                 ])
 
-    print("Image contains %d features/packages total.\n", len(features))
+    print("Image contains %d features/packages total.\n" % len(features))
     print("Found %d vulnerabilities in %d features/packages:\n" % (len(vulns)-1, vulnFeatures))
 
     if vulnFeatures > 0:

build_scripts/docker_local_tests.sh

@@ -7,7 +7,7 @@
 
 setup(
     name='clair_singularity',
-    version='0.3.0',
+    version='0.4.0',
     url='https://github.com/dctrud/clair-singularity',
     author='David Trudgian',
     author_email='[email protected]',

build_scripts/noclair_local_tests.sh

@@ -2,7 +2,7 @@
 
 from clair_singularity.clair import check_clair, post_layer, get_report
 
-API_URL = 'http://clair:6060/v1/'
+API_URL = 'http://localhost:6060/v1/'
 
 @pytest.mark.needs_clair
 def test_check_clair():

clair_singularity/clair.py

@@ -24,7 +24,7 @@ def test_help(runner):
 def test_full_json(runner, testimage):
     result = runner.invoke(cli,
                            ['--json-output', '--bind-ip', MY_IP, '--bind-port', '8081', '--clair-uri',
-                            'http://clair:6060', testimage])
+                            'http://localhost:6060', testimage])
     output = json.loads(result.output)
 
     # There are 62 features in the container scan, and 18 have vulnerabilities
@@ -41,7 +41,7 @@ def test_full_json(runner, testimage):
 @pytest.mark.needs_clair
 def test_full_text(runner, testimage):
     result = runner.invoke(cli, ['--bind-ip', MY_IP, '--bind-port', '8082', '--clair-uri',
-                                 'http://clair:6060', testimage])
+                                 'http://localhost:6060', testimage])
     # Check we do have some CVEs we expect reported here
     assert 'coreutils' in result.output
     assert 'CVE' in result.output