Skip to content

Commit 4a29095

Browse files
authored
Python: Fix bad join order in TPythonTuple
TL;DR: Something introduced the following bad join order: ``` (227s) Tuple counts for dom#TObject::TPythonTuple#ff/2@i2#8f58670w after 3m46s: 25000 ~0% {2} r1 = SCAN PointsToContext::PointsToContext::appliesToScope_dispred#ff#prev_delta OUTPUT In.1, In.0 'context' 24000 ~1% {2} r2 = JOIN r1 WITH @py_scope#f ON FIRST 1 OUTPUT Lhs.1 'context', Lhs.0 1076876712 ~6% {3} r3 = JOIN r2 WITH Flow::TupleNode#class#f CARTESIAN PRODUCT OUTPUT Rhs.0, Lhs.0 'context', Lhs.1 870129666 ~0% {3} r4 = JOIN r3 WITH Flow::ControlFlowNode::isLoad_dispred#f ON FIRST 1 OUTPUT Lhs.1 'context', Lhs.2, Lhs.0 'origin' 870129000 ~0% {3} r5 = r4 AND NOT dom#TObject::TPythonTuple#ff#prev(Lhs.2 'origin', Lhs.0 'context') 870129000 ~1% {3} r6 = SCAN r5 OUTPUT In.2 'origin', In.1, In.0 'context' 9000 ~0% {2} r7 = JOIN r6 WITH Flow::ControlFlowNode::getScope_dispred#ff ON FIRST 2 OUTPUT Lhs.0 'origin', Lhs.2 'context' return r7 ``` (...the above being the tuple counts _at the point when I cancelled the query_!) Rewriting the code to force a join between `TupleNode#class` and `getScope` results in the following join orders: ``` (0s) Tuple counts for TObject::scope_loads_tuplenode#ff/2@b3cf0bo5 after 13ms: 37369 ~3% {1} r1 = JOIN Flow::TupleNode#class#f WITH Flow::ControlFlowNode::isLoad_dispred#f ON FIRST 1 OUTPUT Lhs.0 'origin' 37369 ~3% {2} r2 = JOIN r1 WITH Flow::ControlFlowNode::getScope_dispred#ff ON FIRST 1 OUTPUT Rhs.1 's', Lhs.0 'origin' return r2 ``` and ``` (78s) Tuple counts for dom#TObject::TPythonTuple#ff/2@i53#121c440w after 6ms: 34736 ~3% {2} r1 = SCAN PointsToContext::PointsToContext::appliesToScope_dispred#ff#prev_delta OUTPUT In.1, In.0 'context' 7370 ~5% {2} r2 = JOIN r1 WITH TObject::scope_loads_tuplenode#ff ON FIRST 1 OUTPUT Lhs.1 'context', Rhs.1 'origin' 7370 ~5% {2} r3 = r2 AND NOT dom#TObject::TPythonTuple#ff#prev(Lhs.1 'origin', Lhs.0 'context') 7370 ~1% {2} r4 = SCAN r3 OUTPUT In.1 'origin', In.0 'context' return r4 ``` the latter being the largest iteration of `dom#TPythonTuple` throughout the log. No other major performance issues were observed.
1 parent 56ac990 commit 4a29095

File tree

1 file changed

+11
-2
lines changed

1 file changed

+11
-2
lines changed

python/ql/lib/semmle/python/objects/TObject.qll

Lines changed: 11 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -150,8 +150,10 @@ newtype TObject =
150150
TBuiltinTuple(Builtin bltn) { bltn.getClass() = Builtin::special("tuple") } or
151151
/** Represents a tuple in the Python source */
152152
TPythonTuple(TupleNode origin, PointsToContext context) {
153-
origin.isLoad() and
154-
context.appliesTo(origin)
153+
exists(Scope s |
154+
context.appliesToScope(s) and
155+
scope_loads_tuplenode(s, origin)
156+
)
155157
} or
156158
/** Varargs tuple */
157159
TVarargsTuple(CallNode call, PointsToContext context, int offset, int length) {
@@ -201,6 +203,13 @@ newtype TObject =
201203
Expressions::subscriptPartsPointsTo(_, _, generic, index)
202204
}
203205

206+
/** Join-order helper for TPythonTuple */
207+
pragma[nomagic]
208+
private predicate scope_loads_tuplenode(Scope s, TupleNode origin) {
209+
origin.isLoad() and
210+
origin.getScope() = s
211+
}
212+
204213
/** Holds if the object `t` is a type. */
205214
predicate isType(ObjectInternal t) {
206215
t.isClass() = true

0 commit comments

Comments
 (0)