Skip to content
Open
Show file tree
Hide file tree
Changes from 6 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions go/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -54,9 +54,9 @@ ql/lib/go.dbscheme.stats: ql/lib/go.dbscheme build/stats/src.stamp extractor
codeql dataset measure -o $@ build/stats/database/db-go

test: all build/testdb/check-upgrade-path
codeql test run -j0 ql/test --search-path .. --check-diff-informed --consistency-queries ql/test/consistency --compilation-cache=$(cache) --dynamic-join-order-mode=$(rtjo) --check-databases --fail-on-trap-errors --check-undefined-labels --check-unused-labels --check-repeated-labels --check-redefined-labels --check-use-before-definition
codeql test run -j0 ql/test --search-path .. --check-diff-informed --consistency-queries ql/consistency-queries --compilation-cache=$(cache) --dynamic-join-order-mode=$(rtjo) --check-databases --fail-on-trap-errors --check-undefined-labels --check-unused-labels --check-repeated-labels --check-redefined-labels --check-use-before-definition
# use GOOS=linux because GOOS=darwin GOARCH=386 is no longer supported
env GOOS=linux GOARCH=386 codeql$(EXE) test run -j0 ql/test/query-tests/Security/CWE-681 --search-path .. --check-diff-informed --consistency-queries ql/test/consistency --compilation-cache=$(cache) --dynamic-join-order-mode=$(rtjo)
env GOOS=linux GOARCH=386 codeql$(EXE) test run -j0 ql/test/query-tests/Security/CWE-681 --search-path .. --check-diff-informed --consistency-queries ql/consistency-queries --compilation-cache=$(cache) --dynamic-join-order-mode=$(rtjo)
cd extractor; $(BAZEL) test ...
bash extractor-smoke-test/test.sh || (echo "Extractor smoke test FAILED"; exit 1)

Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
---
category: breaking
---
* The query `go/unexpected-frontend-error` has been moved from the `codeql/go-queries` query to the `codeql-go-consistency-queries` query pack.
Original file line number Diff line number Diff line change
Expand Up @@ -9,6 +9,20 @@ private import TaintTrackingImplSpecific
private import codeql.dataflow.internal.DataFlowImplConsistency
private import semmle.go.dataflow.internal.DataFlowNodes

private module Input implements InputSig<Location, Impl::GoDataFlow> { }
private module Input implements InputSig<Location, Impl::GoDataFlow> {
predicate missingLocationExclude(DataFlow::Node n) {
n instanceof DataFlow::GlobalFunctionNode or n instanceof Private::FlowSummaryNode
}

predicate uniqueNodeLocationExclude(DataFlow::Node n) { missingLocationExclude(n) }

predicate localFlowIsLocalExclude(DataFlow::Node n1, DataFlow::Node n2) {
n1 instanceof DataFlow::FunctionNode and exists(n2)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thats a large CP. Also, can you elaborate what such steps are for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've fixed the CP by adding the constraint simpleLocalFlowStep(n1, n2, _), which is in the only place that this predicate is used.

I didn't immediatley know why we have these steps, but after thinking for a little bit I think I have figured it out. In go you can assign functions to variables and pass them around. It isn't that common but we should deal with it, at least locally. So we use local data flow to find possible callees - for example if you click through Function.getACall() you will get to this line. It's very useful to have steps from function definitions to uses of them as part of the local flow relation so that we can do this without having to use global flow.

}

predicate argHasPostUpdateExclude(DataFlow::ArgumentNode n) {
not DataFlow::insnHasPostUpdateNode(n.asInstruction())
}
}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Did you forget to add the DataFlowConsistency.ql file?

Copy link
Contributor Author

@owen-mc owen-mc Nov 26, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It already exists, so you won't see it in this PR. I made it a while ago but didn't set up CI to run it because there were so many results due to not using use-use flow at the time.


module Consistency = MakeConsistency<Location, Impl::GoDataFlow, GoTaintTracking, Input>;
1 change: 0 additions & 1 deletion go/ql/test/consistency/UnexpectedFrontendErrors.expected

This file was deleted.

1 change: 1 addition & 0 deletions go/ql/test/consistency/dummy.expected
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
| This is a dummy query which is only needed so that the consistency tests will run |
1 change: 1 addition & 0 deletions go/ql/test/consistency/dummy.ql
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
select "This is a dummy query which is only needed so that the consistency tests will run"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand why this is needed; consistency queries are supposed to run on all QL tests.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This test folder exists just to check that UnexpectedFrontendErrors.ql, which used to be the only consistency query, is working, so there are no other .ql files in the folder, and by moving UnexpectedFrontendErrors.ql to be in the same folder as the data flow consistency queries I was leaving the folder without a test query file. I tried without adding another test query and it said it found no tests. I guess it needs a .ql or .qlref file to exist so that it will build a db and run a test, and then run the consistency queries on the db as well.

Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
identityLocalStep
| main.go:46:18:46:18 | n | Node steps to itself |
| main.go:47:3:47:3 | c | Node steps to itself |
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@
reverseRead
| timing.go:15:18:15:20 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| timing.go:28:18:28:20 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| timing.go:41:18:41:20 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| timing.go:53:18:53:20 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
reverseRead
| ImproperLdapAuth.go:18:18:18:20 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| ImproperLdapAuth.go:39:18:39:20 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| ImproperLdapAuth.go:64:18:64:20 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
reverseRead
| go-jose.v3.go:16:17:16:17 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| golang-jwt-v5.go:22:17:22:17 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@
reverseRead
| DivideByZero.go:10:12:10:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| DivideByZero.go:17:12:17:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| DivideByZero.go:24:12:24:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| DivideByZero.go:31:12:31:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| DivideByZero.go:38:12:38:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| DivideByZero.go:45:12:45:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| DivideByZero.go:54:12:54:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| DivideByZero.go:63:12:63:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| DivideByZero.go:72:12:72:12 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
identityLocalStep
| DatabaseCallInLoop.go:9:3:9:4 | db | Node steps to itself |
| test.go:21:12:21:13 | db | Node steps to itself |
| test.go:25:15:25:16 | db | Node steps to itself |
Original file line number Diff line number Diff line change
@@ -0,0 +1,58 @@
reverseRead
| test.go:60:15:60:21 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:61:24:61:30 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:62:13:62:19 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:63:17:63:23 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:64:8:64:14 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:65:12:65:18 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:66:8:66:14 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:67:12:67:18 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:68:17:68:23 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:69:21:69:27 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:70:13:70:19 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:71:17:71:23 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:72:16:72:22 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:73:20:73:26 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:74:7:74:13 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:75:11:75:17 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:76:9:76:15 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:77:13:77:19 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:78:18:78:24 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:79:22:79:28 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:80:5:80:11 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:81:9:81:15 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:82:7:82:13 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:83:11:83:17 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:84:15:84:21 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:85:16:85:22 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:86:20:86:26 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:87:16:87:22 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:88:20:88:26 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:89:17:89:23 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:90:21:90:27 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:91:15:91:21 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:92:19:92:25 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:93:5:93:11 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| test.go:94:9:94:15 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
identityLocalStep
| test.go:114:37:114:38 | rc | Node steps to itself |
| test.go:198:27:198:29 | out | Node steps to itself |
| test.go:224:27:224:29 | out | Node steps to itself |
| test.go:249:27:249:29 | out | Node steps to itself |
| test.go:274:27:274:29 | out | Node steps to itself |
| test.go:299:27:299:29 | out | Node steps to itself |
| test.go:324:26:324:28 | out | Node steps to itself |
| test.go:349:26:349:28 | out | Node steps to itself |
| test.go:375:28:375:30 | out | Node steps to itself |
| test.go:403:28:403:30 | out | Node steps to itself |
| test.go:431:24:431:26 | out | Node steps to itself |
| test.go:463:26:463:28 | out | Node steps to itself |
| test.go:490:26:490:28 | out | Node steps to itself |
| test.go:517:27:517:29 | out | Node steps to itself |
| test.go:546:26:546:28 | out | Node steps to itself |
| test.go:571:26:571:28 | out | Node steps to itself |
| test.go:601:24:601:26 | out | Node steps to itself |
| test.go:614:15:614:21 | tarRead | Node steps to itself |
| test.go:622:3:622:7 | files | Node steps to itself |
| test.go:637:10:637:16 | tarRead | Node steps to itself |
| test.go:647:10:647:16 | tarRead | Node steps to itself |
Original file line number Diff line number Diff line change
@@ -0,0 +1,13 @@
reverseRead
| SensitiveConditionBypassBad.go:7:5:7:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:16:5:16:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:25:5:25:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:34:5:34:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:41:5:41:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:41:35:41:35 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:49:5:49:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:56:5:56:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:63:5:63:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:70:5:70:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:77:5:77:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:84:5:84:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
reverseRead
| ConditionalBypassBad.go:9:5:9:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| ConditionalBypassGood.go:9:5:9:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:9:5:9:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:16:5:16:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:16:41:16:41 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| condition.go:23:5:23:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
reverseRead
| builtin.go:115:31:115:31 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| builtin.go:124:32:124:32 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| builtin.go:133:54:133:54 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| builtin.go:142:55:142:55 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| new-tests.go:62:31:62:33 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| new-tests.go:78:18:78:20 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| new-tests.go:81:37:81:39 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
reverseRead
| CorsMisconfiguration.go:52:14:52:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:59:14:59:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:66:17:66:19 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:74:14:74:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:81:14:81:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:88:14:88:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:101:14:101:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:112:14:112:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:126:15:126:17 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:141:14:141:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:156:14:156:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:170:14:170:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:194:17:194:19 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| CorsMisconfiguration.go:206:14:206:16 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
identityLocalStep
| CorsMisconfiguration.go:208:13:208:18 | origin | Node steps to itself |
| CorsMisconfiguration.go:235:6:235:8 | try | Node steps to itself |
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
reverseRead
| WrongUsageOfUnsafe.go:34:40:34:47 | harmless | Origin of readStep is missing a PostUpdateNode. |
| WrongUsageOfUnsafe.go:55:40:55:47 | harmless | Origin of readStep is missing a PostUpdateNode. |
| WrongUsageOfUnsafe.go:77:43:77:50 | harmless | Origin of readStep is missing a PostUpdateNode. |
| WrongUsageOfUnsafe.go:111:47:111:54 | harmless | Origin of readStep is missing a PostUpdateNode. |
| WrongUsageOfUnsafe.go:211:47:211:54 | harmless | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
reverseRead
| RemoteSources.go:98:9:98:24 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
reverseRead
| pkg1/embedding.go:56:29:56:39 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| pkg1/embedding.go:57:33:57:46 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| pkg1/embedding.go:58:14:58:22 | implicit read of field embedder | Origin of readStep is missing a PostUpdateNode. |
| pkg1/embedding.go:58:31:58:42 | implicit read of field embedder | Origin of readStep is missing a PostUpdateNode. |
| pkg1/tst.go:43:6:43:6 | t | Origin of readStep is missing a PostUpdateNode. |
| pkg1/tst.go:46:6:46:6 | t | Origin of readStep is missing a PostUpdateNode. |
| pkg1/tst.go:50:2:50:2 | t | Origin of readStep is missing a PostUpdateNode. |
| pkg1/tst.go:53:6:53:7 | t2 | Origin of readStep is missing a PostUpdateNode. |
| pkg1/tst.go:55:6:55:7 | t2 | Origin of readStep is missing a PostUpdateNode. |
| promoted.go:18:6:18:6 | t | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
reverseRead
| main.go:49:2:49:4 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| main.go:50:2:50:4 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| main.go:58:2:58:5 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| main.go:63:49:63:49 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| server.go:8:6:8:6 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| server.go:9:6:9:6 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| server.go:10:6:10:6 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| server.go:13:6:13:6 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
reverseRead
| Builtin.go:7:2:7:10 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| Builtin.go:13:2:13:10 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| Builtin.go:22:2:22:10 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| Builtin.go:32:2:32:10 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
| Builtin.go:39:2:39:10 | implicit dereference | Origin of readStep is missing a PostUpdateNode. |
Loading
Loading