Skip to content

Commit 01a6c5e

Browse files
joefarebrotherjoefarebrother
authored
Merge pull request #16446 from joefarebrother/shared-sensitive-heuristics
Ruby/Python/JS/Swift: Add category of Private information to shared sensitive data heuristics
2 parents 60ee7fb + da93a08 commit 01a6c5e

File tree

10 files changed

+270
-10
lines changed

10 files changed

+270
-10
lines changed

‎javascript/ql/lib/change-notes/2024-05-09-sensitive-heuristics.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
---
2+
category: minorAnalysis
3+
---
4+
* Additional heuristics for a new sensitive data classification for private information (e.g. credit card numbers) have been added to the shared `SensitiveDataHeuristics.qll` library. This may result in additional results for queries that use sensitive data such as `js/clear-text-storage-sensitive-data` and `js/clear-text-logging`.

‎javascript/ql/lib/semmle/javascript/security/internal/SensitiveDataHeuristics.qll

Lines changed: 42 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,13 +14,14 @@
1414
* - id: a user name or other account information;
1515
* - password: a password or authorization key;
1616
* - certificate: a certificate.
17+
* - private: private data such as credit card numbers
1718
*
1819
* While classifications are represented as strings, this should not be relied upon.
1920
* Instead, use the predicates in `SensitiveDataClassification::` to work with
2021
* classifications.
2122
*/
2223
class SensitiveDataClassification extends string {
23-
SensitiveDataClassification() { this in ["secret", "id", "password", "certificate"] }
24+
SensitiveDataClassification() { this in ["secret", "id", "password", "certificate", "private"] }
2425
}
2526

2627
/**
@@ -38,6 +39,9 @@ module SensitiveDataClassification {
3839

3940
/** Gets the classification for certificates. */
4041
SensitiveDataClassification certificate() { result = "certificate" }
42+
43+
/** Gets the classification for private data. */
44+
SensitiveDataClassification private() { result = "private" }
4145
}
4246

4347
/**
@@ -77,6 +81,40 @@ module HeuristicNames {
7781
*/
7882
string maybeCertificate() { result = "(?is).*(cert)(?!.*(format|name|ification)).*" }
7983

84+
/**
85+
* Gets a regular expression that identifies strings that may indicate the presence of
86+
* private data.
87+
*/
88+
string maybePrivate() {
89+
result =
90+
"(?is).*(" +
91+
// Inspired by the list on https://cwe.mitre.org/data/definitions/359.html
92+
// Government identifiers, such as Social Security Numbers
93+
"social.?security|employer.?identification|national.?insurance|resident.?id|" +
94+
"passport.?(num|no)|([_-]|\\b)ssn([_-]|\\b)|" +
95+
// Contact information, such as home addresses
96+
"post.?code|zip.?code|home.?addr|" +
97+
// and telephone numbers
98+
"(mob(ile)?|home).?(num|no|tel|phone)|(tel|fax|phone).?(num|no)|telephone|" +
99+
"emergency.?contact|" +
100+
// Geographic location - where the user is (or was)
101+
"latitude|longitude|nationality|" +
102+
// Financial data - such as credit card numbers, salary, bank accounts, and debts
103+
"(credit|debit|bank|visa).?(card|num|no|acc(ou)?nt)|acc(ou)?nt.?(no|num|credit)|" +
104+
"salary|billing|credit.?(rating|score)|([_-]|\\b)ccn([_-]|\\b)|" +
105+
// Communications - e-mail addresses, private e-mail messages, SMS text messages, chat logs, etc.
106+
// "e(mail|_mail)|" + // this seems too noisy
107+
// Health - medical conditions, insurance status, prescription records
108+
"birth.?da(te|y)|da(te|y).?(of.?)?birth|" +
109+
"medical|(health|care).?plan|healthkit|appointment|prescription|" +
110+
"blood.?(type|alcohol|glucose|pressure)|heart.?(rate|rhythm)|body.?(mass|fat)|" +
111+
"menstrua|pregnan|insulin|inhaler|" +
112+
// Relationships - work and family
113+
"employ(er|ee)|spouse|maiden.?name" +
114+
// ---
115+
").*"
116+
}
117+
80118
/**
81119
* Gets a regular expression that identifies strings that may indicate the presence
82120
* of sensitive data, with `classification` describing the kind of sensitive data involved.
@@ -90,6 +128,9 @@ module HeuristicNames {
90128
or
91129
result = maybeCertificate() and
92130
classification = SensitiveDataClassification::certificate()
131+
or
132+
result = maybePrivate() and
133+
classification = SensitiveDataClassification::private()
93134
}
94135

95136
/**

‎python/ql/lib/change-notes/2024-05-09-sensitive-heuristics.md

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,4 @@
1+
---
2+
category: minorAnalysis
3+
---
4+
* Additional heuristics for a new sensitive data classification for private information (e.g. credit card numbers) have been added to the shared `SensitiveDataHeuristics.qll` library. This may result in additional results for queries that use sensitive data such as `py/clear-text-storage-sensitive-data` and `py/clear-text-logging-sensitive-data`.

‎python/ql/lib/semmle/python/security/internal/SensitiveDataHeuristics.qll

Lines changed: 42 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -14,13 +14,14 @@
1414
* - id: a user name or other account information;
1515
* - password: a password or authorization key;
1616
* - certificate: a certificate.
17+
* - private: private data such as credit card numbers
1718
*
1819
* While classifications are represented as strings, this should not be relied upon.
1920
* Instead, use the predicates in `SensitiveDataClassification::` to work with
2021
* classifications.
2122
*/
2223
class SensitiveDataClassification extends string {
23-
SensitiveDataClassification() { this in ["secret", "id", "password", "certificate"] }
24+
SensitiveDataClassification() { this in ["secret", "id", "password", "certificate", "private"] }
2425
}
2526

2627
/**
@@ -38,6 +39,9 @@ module SensitiveDataClassification {
3839

3940
/** Gets the classification for certificates. */
4041
SensitiveDataClassification certificate() { result = "certificate" }
42+
43+
/** Gets the classification for private data. */
44+
SensitiveDataClassification private() { result = "private" }
4145
}
4246

4347
/**
@@ -77,6 +81,40 @@ module HeuristicNames {
7781
*/
7882
string maybeCertificate() { result = "(?is).*(cert)(?!.*(format|name|ification)).*" }
7983

84+
/**
85+
* Gets a regular expression that identifies strings that may indicate the presence of
86+
* private data.
87+
*/
88+
string maybePrivate() {
89+
result =
90+
"(?is).*(" +
91+
// Inspired by the list on https://cwe.mitre.org/data/definitions/359.html
92+
// Government identifiers, such as Social Security Numbers
93+
"social.?security|employer.?identification|national.?insurance|resident.?id|" +
94+
"passport.?(num|no)|([_-]|\\b)ssn([_-]|\\b)|" +
95+
// Contact information, such as home addresses
96+
"post.?code|zip.?code|home.?addr|" +
97+
// and telephone numbers
98+
"(mob(ile)?|home).?(num|no|tel|phone)|(tel|fax|phone).?(num|no)|telephone|" +
99+
"emergency.?contact|" +
100+
// Geographic location - where the user is (or was)
101+
"latitude|longitude|nationality|" +
102+
// Financial data - such as credit card numbers, salary, bank accounts, and debts
103+
"(credit|debit|bank|visa).?(card|num|no|acc(ou)?nt)|acc(ou)?nt.?(no|num|credit)|" +
104+
"salary|billing|credit.?(rating|score)|([_-]|\\b)ccn([_-]|\\b)|" +
105+
// Communications - e-mail addresses, private e-mail messages, SMS text messages, chat logs, etc.
106+
// "e(mail|_mail)|" + // this seems too noisy
107+
// Health - medical conditions, insurance status, prescription records
108+
"birth.?da(te|y)|da(te|y).?(of.?)?birth|" +
109+
"medical|(health|care).?plan|healthkit|appointment|prescription|" +
110+
"blood.?(type|alcohol|glucose|pressure)|heart.?(rate|rhythm)|body.?(mass|fat)|" +
111+
"menstrua|pregnan|insulin|inhaler|" +
112+
// Relationships - work and family
113+
"employ(er|ee)|spouse|maiden.?name" +
114+
// ---
115+
").*"
116+
}
117+
80118
/**
81119
* Gets a regular expression that identifies strings that may indicate the presence
82120
* of sensitive data, with `classification` describing the kind of sensitive data involved.
@@ -90,6 +128,9 @@ module HeuristicNames {
90128
or
91129
result = maybeCertificate() and
92130
classification = SensitiveDataClassification::certificate()
131+
or
132+
result = maybePrivate() and
133+
classification = SensitiveDataClassification::private()
93134
}
94135

95136
/**

‎python/ql/test/query-tests/Security/CWE-312-CleartextLogging/CleartextLogging.expected

Lines changed: 58 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -7,8 +7,21 @@ edges
77
| test.py:19:16:19:29 | ControlFlowNode for get_password() | test.py:19:5:19:12 | ControlFlowNode for password | provenance | |
88
| test.py:44:5:44:5 | ControlFlowNode for x | test.py:45:11:45:11 | ControlFlowNode for x | provenance | |
99
| test.py:44:9:44:25 | ControlFlowNode for Attribute() | test.py:44:5:44:5 | ControlFlowNode for x | provenance | |
10-
| test.py:70:5:70:10 | ControlFlowNode for config | test.py:74:11:74:31 | ControlFlowNode for Subscript | provenance | |
11-
| test.py:72:21:72:37 | ControlFlowNode for Attribute | test.py:70:5:70:10 | ControlFlowNode for config | provenance | |
10+
| test.py:48:14:48:35 | ControlFlowNode for social_security_number | test.py:49:15:49:36 | ControlFlowNode for social_security_number | provenance | |
11+
| test.py:48:38:48:40 | ControlFlowNode for ssn | test.py:50:15:50:17 | ControlFlowNode for ssn | provenance | |
12+
| test.py:48:54:48:63 | ControlFlowNode for passportNo | test.py:52:15:52:24 | ControlFlowNode for passportNo | provenance | |
13+
| test.py:54:34:54:45 | ControlFlowNode for home_address | test.py:57:15:57:26 | ControlFlowNode for home_address | provenance | |
14+
| test.py:59:14:59:26 | ControlFlowNode for user_latitude | test.py:60:15:60:27 | ControlFlowNode for user_latitude | provenance | |
15+
| test.py:59:29:59:42 | ControlFlowNode for user_longitude | test.py:61:15:61:28 | ControlFlowNode for user_longitude | provenance | |
16+
| test.py:63:14:63:26 | ControlFlowNode for mobile_number | test.py:64:15:64:27 | ControlFlowNode for mobile_number | provenance | |
17+
| test.py:63:29:63:35 | ControlFlowNode for phoneNo | test.py:65:15:65:21 | ControlFlowNode for phoneNo | provenance | |
18+
| test.py:67:14:67:23 | ControlFlowNode for creditcard | test.py:68:15:68:24 | ControlFlowNode for creditcard | provenance | |
19+
| test.py:67:26:67:35 | ControlFlowNode for debit_card | test.py:69:15:69:24 | ControlFlowNode for debit_card | provenance | |
20+
| test.py:67:38:67:48 | ControlFlowNode for bank_number | test.py:70:15:70:25 | ControlFlowNode for bank_number | provenance | |
21+
| test.py:67:76:67:78 | ControlFlowNode for ccn | test.py:73:15:73:17 | ControlFlowNode for ccn | provenance | |
22+
| test.py:67:81:67:88 | ControlFlowNode for user_ccn | test.py:74:15:74:22 | ControlFlowNode for user_ccn | provenance | |
23+
| test.py:101:5:101:10 | ControlFlowNode for config | test.py:105:11:105:31 | ControlFlowNode for Subscript | provenance | |
24+
| test.py:103:21:103:37 | ControlFlowNode for Attribute | test.py:101:5:101:10 | ControlFlowNode for config | provenance | |
1225
nodes
1326
| test.py:19:5:19:12 | ControlFlowNode for password | semmle.label | ControlFlowNode for password |
1427
| test.py:19:16:19:29 | ControlFlowNode for get_password() | semmle.label | ControlFlowNode for get_password() |
@@ -24,9 +37,35 @@ nodes
2437
| test.py:44:5:44:5 | ControlFlowNode for x | semmle.label | ControlFlowNode for x |
2538
| test.py:44:9:44:25 | ControlFlowNode for Attribute() | semmle.label | ControlFlowNode for Attribute() |
2639
| test.py:45:11:45:11 | ControlFlowNode for x | semmle.label | ControlFlowNode for x |
27-
| test.py:70:5:70:10 | ControlFlowNode for config | semmle.label | ControlFlowNode for config |
28-
| test.py:72:21:72:37 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute |
29-
| test.py:74:11:74:31 | ControlFlowNode for Subscript | semmle.label | ControlFlowNode for Subscript |
40+
| test.py:48:14:48:35 | ControlFlowNode for social_security_number | semmle.label | ControlFlowNode for social_security_number |
41+
| test.py:48:38:48:40 | ControlFlowNode for ssn | semmle.label | ControlFlowNode for ssn |
42+
| test.py:48:54:48:63 | ControlFlowNode for passportNo | semmle.label | ControlFlowNode for passportNo |
43+
| test.py:49:15:49:36 | ControlFlowNode for social_security_number | semmle.label | ControlFlowNode for social_security_number |
44+
| test.py:50:15:50:17 | ControlFlowNode for ssn | semmle.label | ControlFlowNode for ssn |
45+
| test.py:52:15:52:24 | ControlFlowNode for passportNo | semmle.label | ControlFlowNode for passportNo |
46+
| test.py:54:34:54:45 | ControlFlowNode for home_address | semmle.label | ControlFlowNode for home_address |
47+
| test.py:57:15:57:26 | ControlFlowNode for home_address | semmle.label | ControlFlowNode for home_address |
48+
| test.py:59:14:59:26 | ControlFlowNode for user_latitude | semmle.label | ControlFlowNode for user_latitude |
49+
| test.py:59:29:59:42 | ControlFlowNode for user_longitude | semmle.label | ControlFlowNode for user_longitude |
50+
| test.py:60:15:60:27 | ControlFlowNode for user_latitude | semmle.label | ControlFlowNode for user_latitude |
51+
| test.py:61:15:61:28 | ControlFlowNode for user_longitude | semmle.label | ControlFlowNode for user_longitude |
52+
| test.py:63:14:63:26 | ControlFlowNode for mobile_number | semmle.label | ControlFlowNode for mobile_number |
53+
| test.py:63:29:63:35 | ControlFlowNode for phoneNo | semmle.label | ControlFlowNode for phoneNo |
54+
| test.py:64:15:64:27 | ControlFlowNode for mobile_number | semmle.label | ControlFlowNode for mobile_number |
55+
| test.py:65:15:65:21 | ControlFlowNode for phoneNo | semmle.label | ControlFlowNode for phoneNo |
56+
| test.py:67:14:67:23 | ControlFlowNode for creditcard | semmle.label | ControlFlowNode for creditcard |
57+
| test.py:67:26:67:35 | ControlFlowNode for debit_card | semmle.label | ControlFlowNode for debit_card |
58+
| test.py:67:38:67:48 | ControlFlowNode for bank_number | semmle.label | ControlFlowNode for bank_number |
59+
| test.py:67:76:67:78 | ControlFlowNode for ccn | semmle.label | ControlFlowNode for ccn |
60+
| test.py:67:81:67:88 | ControlFlowNode for user_ccn | semmle.label | ControlFlowNode for user_ccn |
61+
| test.py:68:15:68:24 | ControlFlowNode for creditcard | semmle.label | ControlFlowNode for creditcard |
62+
| test.py:69:15:69:24 | ControlFlowNode for debit_card | semmle.label | ControlFlowNode for debit_card |
63+
| test.py:70:15:70:25 | ControlFlowNode for bank_number | semmle.label | ControlFlowNode for bank_number |
64+
| test.py:73:15:73:17 | ControlFlowNode for ccn | semmle.label | ControlFlowNode for ccn |
65+
| test.py:74:15:74:22 | ControlFlowNode for user_ccn | semmle.label | ControlFlowNode for user_ccn |
66+
| test.py:101:5:101:10 | ControlFlowNode for config | semmle.label | ControlFlowNode for config |
67+
| test.py:103:21:103:37 | ControlFlowNode for Attribute | semmle.label | ControlFlowNode for Attribute |
68+
| test.py:105:11:105:31 | ControlFlowNode for Subscript | semmle.label | ControlFlowNode for Subscript |
3069
subpaths
3170
#select
3271
| test.py:20:48:20:55 | ControlFlowNode for password | test.py:19:16:19:29 | ControlFlowNode for get_password() | test.py:20:48:20:55 | ControlFlowNode for password | This expression logs $@ as clear text. | test.py:19:16:19:29 | ControlFlowNode for get_password() | sensitive data (password) |
@@ -39,4 +78,17 @@ subpaths
3978
| test.py:39:22:39:35 | ControlFlowNode for get_password() | test.py:39:22:39:35 | ControlFlowNode for get_password() | test.py:39:22:39:35 | ControlFlowNode for get_password() | This expression logs $@ as clear text. | test.py:39:22:39:35 | ControlFlowNode for get_password() | sensitive data (password) |
4079
| test.py:40:22:40:35 | ControlFlowNode for get_password() | test.py:40:22:40:35 | ControlFlowNode for get_password() | test.py:40:22:40:35 | ControlFlowNode for get_password() | This expression logs $@ as clear text. | test.py:40:22:40:35 | ControlFlowNode for get_password() | sensitive data (password) |
4180
| test.py:45:11:45:11 | ControlFlowNode for x | test.py:44:9:44:25 | ControlFlowNode for Attribute() | test.py:45:11:45:11 | ControlFlowNode for x | This expression logs $@ as clear text. | test.py:44:9:44:25 | ControlFlowNode for Attribute() | sensitive data (password) |
42-
| test.py:74:11:74:31 | ControlFlowNode for Subscript | test.py:72:21:72:37 | ControlFlowNode for Attribute | test.py:74:11:74:31 | ControlFlowNode for Subscript | This expression logs $@ as clear text. | test.py:72:21:72:37 | ControlFlowNode for Attribute | sensitive data (password) |
81+
| test.py:49:15:49:36 | ControlFlowNode for social_security_number | test.py:48:14:48:35 | ControlFlowNode for social_security_number | test.py:49:15:49:36 | ControlFlowNode for social_security_number | This expression logs $@ as clear text. | test.py:48:14:48:35 | ControlFlowNode for social_security_number | sensitive data (private) |
82+
| test.py:50:15:50:17 | ControlFlowNode for ssn | test.py:48:38:48:40 | ControlFlowNode for ssn | test.py:50:15:50:17 | ControlFlowNode for ssn | This expression logs $@ as clear text. | test.py:48:38:48:40 | ControlFlowNode for ssn | sensitive data (private) |
83+
| test.py:52:15:52:24 | ControlFlowNode for passportNo | test.py:48:54:48:63 | ControlFlowNode for passportNo | test.py:52:15:52:24 | ControlFlowNode for passportNo | This expression logs $@ as clear text. | test.py:48:54:48:63 | ControlFlowNode for passportNo | sensitive data (private) |
84+
| test.py:57:15:57:26 | ControlFlowNode for home_address | test.py:54:34:54:45 | ControlFlowNode for home_address | test.py:57:15:57:26 | ControlFlowNode for home_address | This expression logs $@ as clear text. | test.py:54:34:54:45 | ControlFlowNode for home_address | sensitive data (private) |
85+
| test.py:60:15:60:27 | ControlFlowNode for user_latitude | test.py:59:14:59:26 | ControlFlowNode for user_latitude | test.py:60:15:60:27 | ControlFlowNode for user_latitude | This expression logs $@ as clear text. | test.py:59:14:59:26 | ControlFlowNode for user_latitude | sensitive data (private) |
86+
| test.py:61:15:61:28 | ControlFlowNode for user_longitude | test.py:59:29:59:42 | ControlFlowNode for user_longitude | test.py:61:15:61:28 | ControlFlowNode for user_longitude | This expression logs $@ as clear text. | test.py:59:29:59:42 | ControlFlowNode for user_longitude | sensitive data (private) |
87+
| test.py:64:15:64:27 | ControlFlowNode for mobile_number | test.py:63:14:63:26 | ControlFlowNode for mobile_number | test.py:64:15:64:27 | ControlFlowNode for mobile_number | This expression logs $@ as clear text. | test.py:63:14:63:26 | ControlFlowNode for mobile_number | sensitive data (private) |
88+
| test.py:65:15:65:21 | ControlFlowNode for phoneNo | test.py:63:29:63:35 | ControlFlowNode for phoneNo | test.py:65:15:65:21 | ControlFlowNode for phoneNo | This expression logs $@ as clear text. | test.py:63:29:63:35 | ControlFlowNode for phoneNo | sensitive data (private) |
89+
| test.py:68:15:68:24 | ControlFlowNode for creditcard | test.py:67:14:67:23 | ControlFlowNode for creditcard | test.py:68:15:68:24 | ControlFlowNode for creditcard | This expression logs $@ as clear text. | test.py:67:14:67:23 | ControlFlowNode for creditcard | sensitive data (private) |
90+
| test.py:69:15:69:24 | ControlFlowNode for debit_card | test.py:67:26:67:35 | ControlFlowNode for debit_card | test.py:69:15:69:24 | ControlFlowNode for debit_card | This expression logs $@ as clear text. | test.py:67:26:67:35 | ControlFlowNode for debit_card | sensitive data (private) |
91+
| test.py:70:15:70:25 | ControlFlowNode for bank_number | test.py:67:38:67:48 | ControlFlowNode for bank_number | test.py:70:15:70:25 | ControlFlowNode for bank_number | This expression logs $@ as clear text. | test.py:67:38:67:48 | ControlFlowNode for bank_number | sensitive data (private) |
92+
| test.py:73:15:73:17 | ControlFlowNode for ccn | test.py:67:76:67:78 | ControlFlowNode for ccn | test.py:73:15:73:17 | ControlFlowNode for ccn | This expression logs $@ as clear text. | test.py:67:76:67:78 | ControlFlowNode for ccn | sensitive data (private) |
93+
| test.py:74:15:74:22 | ControlFlowNode for user_ccn | test.py:67:81:67:88 | ControlFlowNode for user_ccn | test.py:74:15:74:22 | ControlFlowNode for user_ccn | This expression logs $@ as clear text. | test.py:67:81:67:88 | ControlFlowNode for user_ccn | sensitive data (private) |
94+
| test.py:105:11:105:31 | ControlFlowNode for Subscript | test.py:103:21:103:37 | ControlFlowNode for Attribute | test.py:105:11:105:31 | ControlFlowNode for Subscript | This expression logs $@ as clear text. | test.py:103:21:103:37 | ControlFlowNode for Attribute | sensitive data (password) |

0 commit comments

Comments
 (0)