Skip to content

Commit 1997f50

Browse files
tamasvajktamasvajk
authored
Merge pull request github#5832 from tamasvajk/feature/csv-coverage-report
Java: github action for CSV coverage report
2 parents d05f524 + 70b3066 commit 1997f50

File tree

8 files changed

+568
-0
lines changed

8 files changed

+568
-0
lines changed

‎.github/workflows/csv-coverage.yml

Lines changed: 77 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,77 @@
1+
name: Build/check CSV flow coverage report
2+
3+
on:
4+
workflow_dispatch:
5+
inputs:
6+
qlModelShaOverride:
7+
description: 'github/codeql repo SHA used for looking up the CSV models'
8+
required: false
9+
push:
10+
branches:
11+
- main
12+
- 'rc/**'
13+
pull_request:
14+
paths:
15+
- '.github/workflows/csv-coverage.yml'
16+
- '*/ql/src/**/*.ql'
17+
- '*/ql/src/**/*.qll'
18+
- 'misc/scripts/library-coverage/*.py'
19+
# input data files
20+
- '*/documentation/library-coverage/cwe-sink.csv'
21+
- '*/documentation/library-coverage/frameworks.csv'
22+
# coverage report files
23+
- '*/documentation/library-coverage/flow-model-coverage.csv'
24+
- '*/documentation/library-coverage/flow-model-coverage.rst'
25+
26+
jobs:
27+
build:
28+
29+
runs-on: ubuntu-latest
30+
31+
steps:
32+
- name: Clone self (github/codeql)
33+
uses: actions/checkout@v2
34+
with:
35+
path: script
36+
- name: Clone self (github/codeql) at a given SHA for analysis
37+
if: github.event.inputs.qlModelShaOverride != ''
38+
uses: actions/checkout@v2
39+
with:
40+
path: codeqlModels
41+
ref: github.event.inputs.qlModelShaOverride
42+
- name: Clone self (github/codeql) for analysis
43+
if: github.event.inputs.qlModelShaOverride == ''
44+
uses: actions/checkout@v2
45+
with:
46+
path: codeqlModels
47+
- name: Set up Python 3.8
48+
uses: actions/setup-python@v2
49+
with:
50+
python-version: 3.8
51+
- name: Download CodeQL CLI
52+
uses: dsaltares/fetch-gh-release-asset@aa37ae5c44d3c9820bc12fe675e8670ecd93bd1c
53+
with:
54+
repo: "github/codeql-cli-binaries"
55+
version: "latest"
56+
file: "codeql-linux64.zip"
57+
token: ${{ secrets.GITHUB_TOKEN }}
58+
- name: Unzip CodeQL CLI
59+
run: unzip -d codeql-cli codeql-linux64.zip
60+
- name: Build modeled package list
61+
run: |
62+
PATH="$PATH:codeql-cli/codeql" python script/misc/scripts/library-coverage/generate-report.py ci codeqlModels script
63+
- name: Upload CSV package list
64+
uses: actions/upload-artifact@v2
65+
with:
66+
name: csv-flow-model-coverage
67+
path: flow-model-coverage-*.csv
68+
- name: Upload RST package list
69+
uses: actions/upload-artifact@v2
70+
with:
71+
name: rst-flow-model-coverage
72+
path: flow-model-coverage-*.rst
73+
- name: Check coverage files
74+
if: github.event.pull_request
75+
run: |
76+
python script/misc/scripts/library-coverage/compare-files.py codeqlModels
77+

‎java/documentation/library-coverage/cwe-sink.csv

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
CWE,Sink identifier,Label
2+
CWE‑089,sql,SQL injection
3+
CWE‑022,create-file,Path injection
4+
CWE‑036,url-open-stream,Path traversal
5+
CWE‑094,bean-validation,Code injection
6+
CWE‑319,open-url,Cleartext transmission
7+
CWE‑079,xss,Cross-site scripting
8+
CWE‑090,ldap,LDAP injection

‎java/documentation/library-coverage/flow-model-coverage.csv

Lines changed: 42 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,42 @@
1+
package,sink,source,summary,sink:bean-validation,sink:create-file,sink:header-splitting,sink:ldap,sink:open-url,sink:set-hostname-verifier,sink:url-open-stream,sink:xpath,sink:xss,source:remote,summary:taint,summary:value
2+
android.util,,16,,,,,,,,,,,16,,
3+
android.webkit,3,2,,,,,,,,,,3,2,,
4+
com.esotericsoftware.kryo.io,,,1,,,,,,,,,,,1,
5+
com.esotericsoftware.kryo5.io,,,1,,,,,,,,,,,1,
6+
com.fasterxml.jackson.databind,,,2,,,,,,,,,,,2,
7+
com.google.common.base,,,28,,,,,,,,,,,22,6
8+
com.google.common.io,6,,69,,,,,,,6,,,,68,1
9+
com.unboundid.ldap.sdk,17,,,,,,17,,,,,,,,
10+
java.beans,,,1,,,,,,,,,,,1,
11+
java.io,3,,20,,3,,,,,,,,,20,
12+
java.lang,,,1,,,,,,,,,,,1,
13+
java.net,2,3,4,,,,,2,,,,,3,4,
14+
java.nio,10,,2,,10,,,,,,,,,2,
15+
java.util,,,13,,,,,,,,,,,13,
16+
javax.naming.directory,1,,,,,,1,,,,,,,,
17+
javax.net.ssl,2,,,,,,,,2,,,,,,
18+
javax.servlet,4,21,2,,,3,,,,,,1,21,2,
19+
javax.validation,1,1,,1,,,,,,,,,1,,
20+
javax.ws.rs.core,1,,,,,1,,,,,,,,,
21+
javax.xml.transform.sax,,,4,,,,,,,,,,,4,
22+
javax.xml.transform.stream,,,2,,,,,,,,,,,2,
23+
javax.xml.xpath,3,,,,,,,,,,3,,,,
24+
org.apache.commons.codec,,,2,,,,,,,,,,,2,
25+
org.apache.commons.io,,,22,,,,,,,,,,,22,
26+
org.apache.commons.lang3,,,313,,,,,,,,,,,299,14
27+
org.apache.commons.text,,,203,,,,,,,,,,,203,
28+
org.apache.directory.ldap.client.api,1,,,,,,1,,,,,,,,
29+
org.apache.hc.core5.function,,,1,,,,,,,,,,,1,
30+
org.apache.hc.core5.http,1,2,39,,,,,,,,,1,2,39,
31+
org.apache.hc.core5.net,,,2,,,,,,,,,,,2,
32+
org.apache.hc.core5.util,,,22,,,,,,,,,,,18,4
33+
org.apache.http,2,3,66,,,,,,,,,2,3,59,7
34+
org.dom4j,20,,,,,,,,,,20,,,,
35+
org.springframework.ldap.core,14,,,,,,14,,,,,,,,
36+
org.springframework.security.web.savedrequest,,6,,,,,,,,,,,6,,
37+
org.springframework.web.client,,3,,,,,,,,,,,3,,
38+
org.springframework.web.context.request,,8,,,,,,,,,,,8,,
39+
org.springframework.web.multipart,,12,,,,,,,,,,,12,,
40+
org.xml.sax,,,1,,,,,,,,,,,1,
41+
org.xmlpull.v1,,3,,,,,,,,,,,3,,
42+
play.mvc,,4,,,,,,,,,,,4,,

‎java/documentation/library-coverage/flow-model-coverage.rst

Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
Java framework & library support
2+
================================
3+
4+
.. csv-table::
5+
:header-rows: 1
6+
:class: fullWidthTable
7+
:widths: auto
8+
9+
Framework / library,Package,Remote flow sources,Taint & value steps,Sinks (total),`CWE‑022` :sub:`Path injection`,`CWE‑036` :sub:`Path traversal`,`CWE‑079` :sub:`Cross-site scripting`,`CWE‑089` :sub:`SQL injection`,`CWE‑090` :sub:`LDAP injection`,`CWE‑094` :sub:`Code injection`,`CWE‑319` :sub:`Cleartext transmission`
10+
Android,``android.*``,18,,3,,,3,,,,
11+
Apache,``org.apache.*``,5,648,4,,,3,,1,,
12+
`Apache Commons IO <https://commons.apache.org/proper/commons-io/>`_,``org.apache.commons.io``,,22,,,,,,,,
13+
Google,``com.google.common.*``,,97,6,,6,,,,,
14+
Java Standard Library,``java.*``,3,41,15,13,,,,,,2
15+
Java extensions,``javax.*``,22,8,12,,,1,,1,1,
16+
`Spring <https://spring.io/>`_,``org.springframework.*``,29,,14,,,,,14,,
17+
Others,"``com.esotericsoftware.kryo.io``, ``com.esotericsoftware.kryo5.io``, ``com.fasterxml.jackson.databind``, ``com.unboundid.ldap.sdk``, ``org.dom4j``, ``org.xml.sax``, ``org.xmlpull.v1``, ``play.mvc``",7,5,37,,,,,17,,
18+
Totals,,84,821,91,13,6,7,,33,1,2
19+

‎java/documentation/library-coverage/frameworks.csv

Lines changed: 8 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,8 @@
1+
Framework name,URL,Package prefix
2+
Java Standard Library,,java.*
3+
Google,,com.google.common.*
4+
Apache,,org.apache.*
5+
Apache Commons IO,https://commons.apache.org/proper/commons-io/,org.apache.commons.io
6+
Android,,android.*
7+
Spring,https://spring.io/,org.springframework.*
8+
Java extensions,,javax.*

‎misc/scripts/library-coverage/compare-files.py

Lines changed: 54 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,54 @@
1+
import sys
2+
import os
3+
import settings
4+
import difflib
5+
6+
"""
7+
This script compares the generated CSV coverage files with the ones in the codebase.
8+
"""
9+
10+
11+
def check_file_exists(file):
12+
if not os.path.exists(file):
13+
print("Expected file '" + file + "' doesn't exist.", file=sys.stderr)
14+
sys.exit(1)
15+
16+
17+
def ignore_line_ending(ch):
18+
return difflib.IS_CHARACTER_JUNK(ch, ws=" \r\n")
19+
20+
21+
def compare_files(file1, file2):
22+
has_differences = False
23+
diff = difflib.ndiff(open(file1).readlines(),
24+
open(file2).readlines(), None, ignore_line_ending)
25+
for line in diff:
26+
if line.startswith("+") or line.startswith("-"):
27+
print(line, end="", file=sys.stderr)
28+
has_differences = True
29+
30+
if has_differences:
31+
print("Error: The generated file doesn't match the one in the codebase. Please check and fix file '" +
32+
file1 + "'.", file=sys.stderr)
33+
sys.exit(1)
34+
35+
36+
languages = ['java']
37+
38+
for lang in languages:
39+
repo_output_rst = settings.repo_output_rst.format(language=lang)
40+
repo_output_csv = settings.repo_output_csv.format(language=lang)
41+
42+
generated_output_rst = settings.generated_output_rst.format(language=lang)
43+
generated_output_csv = settings.generated_output_csv.format(language=lang)
44+
45+
check_file_exists(repo_output_rst)
46+
check_file_exists(repo_output_csv)
47+
check_file_exists(generated_output_rst)
48+
check_file_exists(generated_output_csv)
49+
50+
compare_files(repo_output_rst, generated_output_rst)
51+
compare_files(repo_output_csv, generated_output_csv)
52+
53+
print("The generated files for '" + lang +
54+
"' match the ones in the codebase.")

0 commit comments

Comments
 (0)