Merge pull request #7616 from github/henrymercer/js-atm-add-query-help

JS: Add query help for ML-powered queries

Author: henrymercer

javascript/ql/experimental/adaptivethreatmodeling/src/NosqlInjectionATM.md

@@ -0,0 +1,50 @@
+# NoSQL database query built from user-controlled sources (experimental)
+This is an experimental query. Experimental queries generate alerts using machine learning. They might include more false positives but they will improve over time.
+
+If a database query (such as a SQL or NoSQL query) is built from user-provided data without sufficient sanitization, a malicious user may be able to run malicious database queries.
+
+
+## Recommendation
+Most database connector libraries offer a way of safely embedding untrusted data into a query by means of query parameters or prepared statements.
+
+For NoSQL queries, make use of an operator like MongoDB's `$eq` to ensure that untrusted data is interpreted as a literal value and not as a query object.
+
+
+## Example
+In the following example, assume the function `handler` is an HTTP request handler in a web application, whose parameter `req` contains the request object.
+
+The handler constructs two copies of the same SQL query involving user input taken from the request object, once unsafely using string concatenation, and once safely using query parameters.
+
+In the first case, the query string `query1` is built by directly concatenating a user-supplied request parameter with some string literals. The parameter may include quote characters, so this code is vulnerable to a SQL injection attack.
+
+In the second case, the parameter is embedded into the query string `query2` using query parameters. In this example, we use the API offered by the `pg` Postgres database connector library, but other libraries offer similar features. This version is immune to injection attacks.
+
+
+```javascript
+const app = require("express")(),
+      pg = require("pg"),
+      pool = new pg.Pool(config);
+
+app.get("search", function handler(req, res) {
+  // BAD: the category might have SQL special characters in it
+  var query1 =
+    "SELECT ITEM,PRICE FROM PRODUCT WHERE ITEM_CATEGORY='" +
+    req.params.category +
+    "' ORDER BY PRICE";
+  pool.query(query1, [], function(err, results) {
+    // process results
+  });
+
+  // GOOD: use parameters
+  var query2 =
+    "SELECT ITEM,PRICE FROM PRODUCT WHERE ITEM_CATEGORY=$1" + " ORDER BY PRICE";
+  pool.query(query2, [req.params.category], function(err, results) {
+    // process results
+  });
+});
+
+```
+
+## References
+* Wikipedia: [SQL injection](https://en.wikipedia.org/wiki/SQL_injection).
+* MongoDB: [$eq operator](https://docs.mongodb.com/manual/reference/operator/query/eq).

javascript/ql/experimental/adaptivethreatmodeling/src/SqlInjectionATM.md

@@ -0,0 +1,50 @@
+# SQL database query built from user-controlled sources (experimental)
+This is an experimental query. Experimental queries generate alerts using machine learning. They might include more false positives but they will improve over time.
+
+If a database query (such as a SQL or NoSQL query) is built from user-provided data without sufficient sanitization, a malicious user may be able to run malicious database queries.
+
+
+## Recommendation
+Most database connector libraries offer a way of safely embedding untrusted data into a query by means of query parameters or prepared statements.
+
+For NoSQL queries, make use of an operator like MongoDB's `$eq` to ensure that untrusted data is interpreted as a literal value and not as a query object.
+
+
+## Example
+In the following example, assume the function `handler` is an HTTP request handler in a web application, whose parameter `req` contains the request object.
+
+The handler constructs two copies of the same SQL query involving user input taken from the request object, once unsafely using string concatenation, and once safely using query parameters.
+
+In the first case, the query string `query1` is built by directly concatenating a user-supplied request parameter with some string literals. The parameter may include quote characters, so this code is vulnerable to a SQL injection attack.
+
+In the second case, the parameter is embedded into the query string `query2` using query parameters. In this example, we use the API offered by the `pg` Postgres database connector library, but other libraries offer similar features. This version is immune to injection attacks.
+
+
+```javascript
+const app = require("express")(),
+      pg = require("pg"),
+      pool = new pg.Pool(config);
+
+app.get("search", function handler(req, res) {
+  // BAD: the category might have SQL special characters in it
+  var query1 =
+    "SELECT ITEM,PRICE FROM PRODUCT WHERE ITEM_CATEGORY='" +
+    req.params.category +
+    "' ORDER BY PRICE";
+  pool.query(query1, [], function(err, results) {
+    // process results
+  });
+
+  // GOOD: use parameters
+  var query2 =
+    "SELECT ITEM,PRICE FROM PRODUCT WHERE ITEM_CATEGORY=$1" + " ORDER BY PRICE";
+  pool.query(query2, [req.params.category], function(err, results) {
+    // process results
+  });
+});
+
+```
+
+## References
+* Wikipedia: [SQL injection](https://en.wikipedia.org/wiki/SQL_injection).
+* MongoDB: [$eq operator](https://docs.mongodb.com/manual/reference/operator/query/eq).

javascript/ql/experimental/adaptivethreatmodeling/src/TaintedPathATM.md

@@ -0,0 +1,42 @@
+# Uncontrolled data used in path expression (experimental)
+This is an experimental query. Experimental queries generate alerts using machine learning. They might include more false positives but they will improve over time.
+
+Accessing files using paths constructed from user-controlled data can allow an attacker to access unexpected resources. This can result in sensitive information being revealed or deleted, or an attacker being able to influence behavior by modifying unexpected files.
+
+
+## Recommendation
+Validate user input before using it to construct a file path, either using an off-the-shelf library like the `sanitize-filename` npm package, or by performing custom validation.
+
+Ideally, follow these rules:
+
+* Do not allow more than a single "." character.
+* Do not allow directory separators such as "/" or "\\" (depending on the file system).
+* Do not rely on simply replacing problematic sequences such as "../". For example, after applying this filter to ".../...//", the resulting string would still be "../".
+* Use a whitelist of known good patterns.
+
+## Example
+In the first example, a file name is read from an HTTP request and then used to access a file. However, a malicious user could enter a file name which is an absolute path, such as `"/etc/passwd"`.
+
+In the second example, it appears that the user is restricted to opening a file within the `"user"` home directory. However, a malicious user could enter a file name containing special characters. For example, the string `"../../etc/passwd"` will result in the code reading the file located at `"/home/user/../../etc/passwd"`, which is the system's password file. This file would then be sent back to the user, giving them access to all the system's passwords.
+
+
+```javascript
+var fs = require('fs'),
+    http = require('http'),
+    url = require('url');
+
+var server = http.createServer(function(req, res) {
+  let path = url.parse(req.url, true).query.path;
+
+  // BAD: This could read any file on the file system
+  res.write(fs.readFileSync(path));
+
+  // BAD: This could still read any file on the file system
+  res.write(fs.readFileSync("/home/user/" + path));
+});
+
+```
+
+## References
+* OWASP: [Path Traversal](https://owasp.org/www-community/attacks/Path_Traversal).
+* npm: [sanitize-filename](https://www.npmjs.com/package/sanitize-filename) package.

javascript/ql/experimental/adaptivethreatmodeling/src/XssATM.md

@@ -0,0 +1,32 @@
+# Client-side cross-site scripting (experimental)
+This is an experimental query. Experimental queries generate alerts using machine learning. They might include more false positives but they will improve over time.
+
+Directly writing user input (for example, a URL query parameter) to a webpage without properly sanitizing the input first, allows for a cross-site scripting vulnerability.
+
+This kind of vulnerability is also called *DOM-based* cross-site scripting, to distinguish it from other types of cross-site scripting.
+
+
+## Recommendation
+To guard against cross-site scripting, consider using contextual output encoding/escaping before writing user input to the page, or one of the other solutions that are mentioned in the references.
+
+
+## Example
+The following example shows part of the page URL being written directly to the document, leaving the website vulnerable to cross-site scripting.
+
+
+```javascript
+function setLanguageOptions() {
+    var href = document.location.href,
+        deflt = href.substring(href.indexOf("default=")+8);
+    document.write("<OPTION value=1>"+deflt+"</OPTION>");
+    document.write("<OPTION value=2>English</OPTION>");
+}
+
+```
+
+## References
+* OWASP: [DOM based XSS Prevention Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/DOM_based_XSS_Prevention_Cheat_Sheet.html).
+* OWASP: [XSS (Cross Site Scripting) Prevention Cheat Sheet](https://cheatsheetseries.owasp.org/cheatsheets/Cross_Site_Scripting_Prevention_Cheat_Sheet.html).
+* OWASP [DOM Based XSS](https://www.owasp.org/index.php/DOM_Based_XSS).
+* OWASP [Types of Cross-Site Scripting](https://www.owasp.org/index.php/Types_of_Cross-Site_Scripting).
+* Wikipedia: [Cross-site scripting](http://en.wikipedia.org/wiki/Cross-site_scripting).

javascript/ql/experimental/adaptivethreatmodeling/src/codeql-suites/javascript-atm-code-scanning.qls

@@ -1,8 +1,2 @@
 - description: ATM boosted Code Scanning queries for JavaScript
 - queries: .
-- include:
-    id: 
-    - adaptive-threat-modeling/js/nosql-injection
-    - adaptive-threat-modeling/js/sql-injection
-    - adaptive-threat-modeling/js/path-injection
-    - adaptive-threat-modeling/js/xss