|
| 1 | +<!DOCTYPE html> |
| 2 | +<html lang="en"> |
| 3 | + <head> |
| 4 | + <meta charset="utf-8"> |
| 5 | + <meta http-equiv="X-UA-Compatible" content="IE=edge"> |
| 6 | + <meta name="viewport" content="width=device-width, initial-scale=1"> |
| 7 | + <!-- The above 3 meta tags *must* come first in the head; any other head content must come *after* these tags --> |
| 8 | + <meta name="description" content=""> |
| 9 | + <meta name="author" content=""> |
| 10 | + <link rel="icon" href="../../favicon.ico"> |
| 11 | + |
| 12 | + <title>Commercial Tools Scorecard</title> |
| 13 | + |
| 14 | + <!-- Bootstrap core CSS --> |
| 15 | + <link href="content/css/bootstrap.min.css" rel="stylesheet"> |
| 16 | + |
| 17 | + <!-- Custom styles for this template --> |
| 18 | + <link href="content/dashboard.css" rel="stylesheet"> |
| 19 | + |
| 20 | + <!-- Just for debugging purposes. Don't actually copy these 2 lines! --> |
| 21 | + <!--[if lt IE 9]><script src="../../assets/js/ie8-responsive-file-warning.js"></script><![endif]--> |
| 22 | + <script src="content/js/ie-emulation-modes-warning.js"></script> |
| 23 | + |
| 24 | + <!-- HTML5 shim and Respond.js for IE8 support of HTML5 elements and media queries --> |
| 25 | + <!--[if lt IE 9]> |
| 26 | + <script src="https://oss.maxcdn.com/html5shiv/3.7.2/html5shiv.min.js"></script> |
| 27 | + <script src="https://oss.maxcdn.com/respond/1.4.2/respond.min.js"></script> |
| 28 | + <![endif]--> |
| 29 | + </head> |
| 30 | + |
| 31 | + <body> |
| 32 | + |
| 33 | + <nav class="navbar navbar-inverse navbar-fixed-top"> |
| 34 | + <div class="container-fluid"> |
| 35 | + <div class="navbar-header"> |
| 36 | + <a class="navbar-brand" href="OWASP_Benchmark_Home.html">OWASP Benchmark v1.1,1.2beta</a> |
| 37 | + </div> |
| 38 | + <div id="navbar" class="navbar-collapse collapse"> |
| 39 | + <ul class="nav navbar-nav navbar-right"> |
| 40 | + <li><a href="OWASP_Benchmark_Home.html">Home</a></li> |
| 41 | + <li class="dropdown"> |
| 42 | + <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Tools<span class="caret"></span></a> |
| 43 | + <ul class="dropdown-menu"> |
| 44 | +<li><a href="Benchmark_v1.2beta_Scorecard_for_FBwFindSecBugs.html">FBwFindSecBugs</a></li> |
| 45 | +<li><a href="Benchmark_v1.2beta_Scorecard_for_FindBugs.html">FindBugs</a></li> |
| 46 | +<li><a href="Benchmark_v1.2beta_Scorecard_for_OWASP_ZAP.html">OWASP ZAP</a></li> |
| 47 | +<li><a href="Benchmark_v1.2beta_Scorecard_for_PMD.html">PMD</a></li> |
| 48 | +<li><a href="Benchmark_v1.2beta_Scorecard_for_SonarQube.html">SonarQube</a></li> |
| 49 | +<li><a href="Benchmark_v1.1,1.2beta_Scorecard_for_Commercial_Tools.html">Commercial Average</a></li> |
| 50 | + |
| 51 | + </ul> |
| 52 | + </li> |
| 53 | + <li class="dropdown"> |
| 54 | + <a href="#" class="dropdown-toggle" data-toggle="dropdown" role="button" aria-haspopup="true" aria-expanded="false">Vulnerabilities<span class="caret"></span></a> |
| 55 | + <ul class="dropdown-menu"> |
| 56 | + <li><a href="Benchmark_v1.1,1.2beta_Scorecard_for_Command_Injection.html">Command Injection</a></li> |
| 57 | + <li><a href="Benchmark_v1.1,1.2beta_Scorecard_for_Cross-Site_Scripting.html">Cross-Site Scripting</a></li> |
| 58 | + <li><a href="Benchmark_v1.1,1.2beta_Scorecard_for_Insecure_Cookie.html">Insecure Cookie</a></li> |
| 59 | + <li><a href="Benchmark_v1.1,1.2beta_Scorecard_for_LDAP_Injection.html">LDAP Injection</a></li> |
| 60 | + <li><a href="Benchmark_v1.1,1.2beta_Scorecard_for_Path_Traversal.html">Path Traversal</a></li> |
| 61 | + <li><a href="Benchmark_v1.1,1.2beta_Scorecard_for_SQL_Injection.html">SQL Injection</a></li> |
| 62 | + <li><a href="Benchmark_v1.1,1.2beta_Scorecard_for_Trust_Boundary_Violation.html">Trust Boundary Violation</a></li> |
| 63 | + <li><a href="Benchmark_v1.1,1.2beta_Scorecard_for_Weak_Encryption_Algorithm.html">Weak Encryption Algorithm</a></li> |
| 64 | + <li><a href="Benchmark_v1.1,1.2beta_Scorecard_for_Weak_Hash_Algorithm.html">Weak Hash Algorithm</a></li> |
| 65 | + <li><a href="Benchmark_v1.1,1.2beta_Scorecard_for_Weak_Random_Number.html">Weak Random Number</a></li> |
| 66 | + <li><a href="Benchmark_v1.1,1.2beta_Scorecard_for_XPath_Injection.html">XPath Injection</a></li> |
| 67 | + |
| 68 | + </ul> |
| 69 | + </li> |
| 70 | + <li><a href="OWASP_Benchmark_Guide.html">Guide</a></li> |
| 71 | + </ul> |
| 72 | + </div> |
| 73 | + </div> |
| 74 | + </nav> |
| 75 | + |
| 76 | + <div class="container"> |
| 77 | + |
| 78 | + <div class="starter-template"> |
| 79 | + |
| 80 | +<div>empty</div> |
| 81 | +<div>empty</div> |
| 82 | + |
| 83 | +<h3>OWASP Benchmark Scorecard for Commercial Tools</h3> |
| 84 | + |
| 85 | +<p>The OWASP Benchmark is a test suite designed to evaluate the speed, coverage, and accuracy of automated vulnerability detection tools. Without the ability to measure these tools, |
| 86 | +it is difficult to understand their strengths and weaknesses, and compare them to each other. The Benchmark contains thousands of test cases that are fully runnable and exploitable. |
| 87 | +The following is the scorecard showing how well the commercial tools collectively performed against version 1.1,1.2beta of the Benchmark. For each vulnerability |
| 88 | +it shows the lowest, average, and highest scores across all the commercial tools included in this scorecard calculation.</p> |
| 89 | + |
| 90 | +<p>For more information, please visit the <a href="https://www.owasp.org/index.php/Benchmark">OWASP Benchmark Project Site</a>. |
| 91 | + |
| 92 | +<p> |
| 93 | +<p> |
| 94 | + |
| 95 | +<h2>Average Scores Per Vulnerability for Commercial Tools</h2> |
| 96 | +<table class="table"> |
| 97 | +<tr><th>Vulnerability Category</th><th>Low Tool Type</th><th>Low Score</th><th>Ave Score</th><th>High Score</th><th>High Tool Type</th></tr> |
| 98 | +<tr><td>Command Injection</td><td>SAST</td><td class="danger">0</td><td>16</td><td class="danger">25</td><td>SAST</td></tr> |
| 99 | +<tr><td>Cross-Site Scripting</td><td>SAST</td><td class="danger">9</td><td>22</td><td class="danger">39</td><td>SAST</td></tr> |
| 100 | +<tr><td>Insecure Cookie</td><td>SAST</td><td class="danger">0</td><td>33</td><td class="success">100</td><td>SAST</td></tr> |
| 101 | +<tr><td>LDAP Injection</td><td>SAST</td><td class="danger">0</td><td>17</td><td class="success">54</td><td>SAST</td></tr> |
| 102 | +<tr><td>Path Traversal</td><td>SAST</td><td class="danger">1</td><td>18</td><td class="danger">30</td><td>SAST</td></tr> |
| 103 | +<tr><td>SQL Injection</td><td>SAST</td><td class="danger">9</td><td>24</td><td class="danger">34</td><td>SAST</td></tr> |
| 104 | +<tr><td>Trust Boundary Violation</td><td>SAST</td><td class="danger">0</td><td>8</td><td class="danger">16</td><td>SAST</td></tr> |
| 105 | +<tr><td>Weak Encryption Algorithm</td><td>SAST</td><td class="danger">0</td><td>39</td><td class="success">74</td><td>SAST</td></tr> |
| 106 | +<tr><td>Weak Hash Algorithm</td><td>SAST</td><td class="danger">0</td><td>42</td><td class="success">77</td><td>SAST</td></tr> |
| 107 | +<tr><td>Weak Random Number</td><td>SAST</td><td class="danger">0</td><td>36</td><td class="success">90</td><td>SAST</td></tr> |
| 108 | +<tr><td>XPath Injection</td><td>SAST</td><td class="danger">0</td><td>27</td><td class="success">59</td><td>SAST</td></tr> |
| 109 | +<tr><td>Average across all categories for 6 tools</td><td></td><td>1.7</td><td>25.6</td><td>54.4</td><td></td></tr> |
| 110 | + |
| 111 | + |
| 112 | +<p> |
| 113 | +<p> |
| 114 | + |
| 115 | +<h2>Key</h2> |
| 116 | +<table class="table"> |
| 117 | +<tr> |
| 118 | +<th>Tool Type</th> |
| 119 | +<td>SAST - Static Application Security Testing. DAST - Dynamic Application Security Testing. |
| 120 | +IAST - Interactive Application Security Testing. These terms were coined by Gartner.</td> |
| 121 | +</tr> |
| 122 | + |
| 123 | +<tr> |
| 124 | +<th>True Positive (TP)</th> |
| 125 | +<td>Tests with real vulnerabilities that were correctly reported as vulnerable by the tool.</td> |
| 126 | +</tr> |
| 127 | + |
| 128 | +<tr> |
| 129 | +<th>False Negative (FN)</th> |
| 130 | +<td>Tests with real vulnerabilities that were not correctly reported as vulnerable by the tool.</td> |
| 131 | +</tr> |
| 132 | + |
| 133 | +<tr> |
| 134 | +<th>True Negative (TN)</th> |
| 135 | +<td>Tests with fake vulnerabilities that were correctly not reported as vulnerable by the tool.</td> |
| 136 | +</tr> |
| 137 | + |
| 138 | +<tr> |
| 139 | +<th>False Positive (FP)</th> |
| 140 | +<td>Tests with fake vulnerabilities that were incorrectly reported as vulnerable by the tool.</td> |
| 141 | +</tr> |
| 142 | + |
| 143 | +<tr> |
| 144 | +<th>True Positive Rate (TPR) = TP / ( TP + FN )</th> |
| 145 | +<td>The rate at which the tool correctly reports real vulnerabilities. Also referred to as Recall, as defined at |
| 146 | +<a href="https://en.wikipedia.org/wiki/Precision_and_recall">Wikipedia</a>.</td> |
| 147 | +</tr> |
| 148 | + |
| 149 | +<tr> |
| 150 | +<th>False Positive Rate (FPR) = FP / ( FP + TN )</th> |
| 151 | +<td>The rate at which the tool incorrectly reports fake vulnerabilities as real.</td> |
| 152 | +</tr> |
| 153 | + |
| 154 | +<tr> |
| 155 | +<th>Score = TPR - FPR</th> |
| 156 | +<td>Normalized distance from the random guess line.</td> |
| 157 | +</tr> |
| 158 | + |
| 159 | +</table> |
| 160 | + |
| 161 | + </div> |
| 162 | + |
| 163 | + </div><!-- /.container --> |
| 164 | + |
| 165 | + <!-- Bootstrap core JavaScript |
| 166 | + ================================================== --> |
| 167 | + <!-- jQuery (necessary for Bootstrap's JavaScript plugins) --> |
| 168 | + <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.11.3/jquery.min.js"></script> |
| 169 | + <!-- Include all compiled plugins (below), or include individual files as needed --> |
| 170 | + <script src="content/js/bootstrap.min.js"></script> |
| 171 | + </body> |
| 172 | +</html> |
| 173 | + |
0 commit comments