Skip to content

Commit 0e31439

Browse files
committed
JS: Simplify aggregation of tokens into entity strings
Change the cutoff logic from `count` to `strictcount`, since we know it only applies to a non-empty set of results. Use a single `strictconcat` aggregate to combine tokens in order of location, instead of computing a `rank` followed by a `concat`. Strictness introduces a slight change of behaviour because missing tokens will now result in no results from the predicate rather than an empty feature string.
1 parent 2a3b5fc commit 0e31439

File tree

1 file changed

+9
-12
lines changed
  • javascript/ql/experimental/adaptivethreatmodeling/lib/experimental/adaptivethreatmodeling

1 file changed

+9
-12
lines changed

javascript/ql/experimental/adaptivethreatmodeling/lib/experimental/adaptivethreatmodeling/EndpointFeatures.qll

Lines changed: 9 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -127,22 +127,19 @@ module FunctionBodies {
127127
// If a function has more than 256 body subtokens, then featurize it as absent. This
128128
// approximates the behavior of the classifer on non-generic body features where large body
129129
// features are replaced by the absent token.
130-
if count(DatabaseFeatures::AstNode node, string token | bodyTokens(entity, node, token)) > 256
130+
if
131+
strictcount(DatabaseFeatures::AstNode node, string token | bodyTokens(entity, node, token)) >
132+
256
131133
then result = ""
132134
else
133135
result =
134-
concat(int i, string rankedToken |
135-
rankedToken =
136-
rank[i](DatabaseFeatures::AstNode node, string token, Location l |
137-
bodyTokens(entity, node, token) and l = node.getLocation()
138-
|
139-
token
140-
order by
141-
l.getFile().getAbsolutePath(), l.getStartLine(), l.getStartColumn(), l.getEndLine(),
142-
l.getEndColumn(), token
143-
)
136+
strictconcat(DatabaseFeatures::AstNode node, string token, Location l |
137+
bodyTokens(entity, node, token) and l = node.getLocation()
144138
|
145-
rankedToken, " " order by i
139+
token, " "
140+
order by
141+
l.getFile().getAbsolutePath(), l.getStartLine(), l.getStartColumn(), l.getEndLine(),
142+
l.getEndColumn(), token
146143
)
147144
}
148145
}

0 commit comments

Comments
 (0)