Skip to content

Commit cab9c64

Browse files
owen-mcowen-mc
committed
Add docs for data flow in Go
Mostly based on the java and C# equivalents.
1 parent a0d2213 commit cab9c64

File tree

4 files changed

+373
-3
lines changed

4 files changed

+373
-3
lines changed

‎docs/codeql/codeql-language-guides/analyzing-data-flow-in-go.rst

Lines changed: 369 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,369 @@
1+
.. _analyzing-data-flow-in-go:
2+
3+
Analyzing data flow in Go
4+
=========================
5+
6+
You can use CodeQL to track the flow of data through a Go program to its use.
7+
8+
About this article
9+
------------------
10+
11+
This article describes how data flow analysis is implemented in the CodeQL libraries for Go and includes examples to help you write your own data flow queries.
12+
The following sections describe how to use the libraries for local data flow, global data flow, and taint tracking.
13+
14+
For a more general introduction to modeling data flow, see ":ref:`About data flow analysis <about-data-flow-analysis>`."
15+
16+
.. include:: ../reusables/new-data-flow-api.rst
17+
18+
Local data flow
19+
---------------
20+
21+
Local data flow is data flow within a single method or callable. Local data flow is usually easier, faster, and more precise than global data flow, and is sufficient for many queries.
22+
23+
Using local data flow
24+
~~~~~~~~~~~~~~~~~~~~~
25+
26+
The ``DataFlow`` module defines the class ``Node`` denoting any element that data can flow through.
27+
The ``Node`` class has a number of useful subclasses, such as ``ExprNode`` for expressions, ``ParameterNode`` for parameters, and ``InstructionNode`` for control-flow nodes.
28+
You can map between data flow nodes and expressions/control-flow nodes/parameters using the member predicates ``asExpr``, ``asParameter`` and ``asInstructionNode``:
29+
30+
.. code-block:: ql
31+
32+
class Node {
33+
/** Gets the expression corresponding to this node, if any. */
34+
Expr asExpr() { ... }
35+
36+
/** Gets the parameter corresponding to this node, if any. */
37+
Parameter asParameter() { ... }
38+
39+
/** Gets the IR instruction corresponding to this node, if any. */
40+
IR::Instruction asInstruction() { ... }
41+
42+
...
43+
}
44+
45+
or using the predicates ``exprNode``, ``parameterNode`` and ``instructionNode``:
46+
47+
.. code-block:: ql
48+
49+
/**
50+
* Gets the `Node` corresponding to `e`.
51+
*/
52+
ExprNode exprNode(Expr e) { ... }
53+
54+
/**
55+
* Gets the `Node` corresponding to the value of `p` at function entry.
56+
*/
57+
ParameterNode parameterNode(Parameter p) { ... }
58+
59+
/**
60+
* Gets the `Node` corresponding to `insn`.
61+
*/
62+
InstructionNode instructionNode(IR::Instruction insn) { ... }
63+
64+
The predicate ``localFlowStep(Node nodeFrom, Node nodeTo)`` holds if there is an immediate data flow edge from the node ``nodeFrom`` to the node ``nodeTo``. You can apply the predicate recursively by using the ``+`` and ``*`` operators, or by using the predefined recursive predicate ``localFlow``, which is equivalent to ``localFlowStep*``.
65+
66+
For example, you can find flow from a parameter ``source`` to an expression ``sink`` in zero or more local steps:
67+
68+
.. code-block:: ql
69+
70+
DataFlow::localFlow(DataFlow::parameterNode(source), DataFlow::exprNode(sink))
71+
72+
Using local taint tracking
73+
~~~~~~~~~~~~~~~~~~~~~~~~~~
74+
75+
Local taint tracking extends local data flow by including non-value-preserving flow steps. For example:
76+
77+
.. code-block:: go
78+
79+
temp := x;
80+
y := temp + ", " + temp;
81+
82+
If ``x`` is a tainted string then ``y`` is also tainted.
83+
84+
85+
The local taint tracking library is in the module ``TaintTracking``. Like local data flow, a predicate ``localTaintStep(DataFlow::Node nodeFrom, DataFlow::Node nodeTo)`` holds if there is an immediate taint propagation edge from the node ``nodeFrom`` to the node ``nodeTo``. You can apply the predicate recursively by using the ``+`` and ``*`` operators, or by using the predefined recursive predicate ``localTaint``, which is equivalent to ``localTaintStep*``.
86+
87+
For example, you can find taint propagation from a parameter ``source`` to an expression ``sink`` in zero or more local steps:
88+
89+
.. code-block:: ql
90+
91+
TaintTracking::localTaint(DataFlow::parameterNode(source), DataFlow::exprNode(sink))
92+
93+
Examples
94+
~~~~~~~~
95+
96+
This query finds the filename passed to ``os.Open(..)``.
97+
98+
.. code-block:: ql
99+
100+
import go
101+
102+
from Function osOpen, CallExpr call
103+
where
104+
osOpen.hasQualifiedName("os", "Open") and
105+
call.getTarget() = osOpen
106+
select call.getArgument(0)
107+
108+
Unfortunately, this only gives the expression in the argument, not the values which could be passed to it. So we use local data flow to find all expressions that flow into the argument:
109+
110+
.. code-block:: ql
111+
112+
import go
113+
114+
from Function osOpen, CallExpr call, Expr src
115+
where
116+
osOpen.hasQualifiedName("os", "Open") and
117+
call.getTarget() = osOpen and
118+
DataFlow::localFlow(DataFlow::exprNode(src), DataFlow::exprNode(call.getArgument(0)))
119+
select src
120+
121+
Then we can make the source more specific, for example an access to a parameter. This query finds where a public parameter is passed to ``os.Open(..)``:
122+
123+
.. code-block:: ql
124+
125+
import go
126+
127+
from Function osOpen, CallExpr call, Parameter p
128+
where
129+
osOpen.hasQualifiedName("os", "Open") and
130+
call.getTarget() = osOpen and
131+
DataFlow::localFlow(DataFlow::parameterNode(p), DataFlow::exprNode(call.getArgument(0)))
132+
select p
133+
134+
This query finds calls to formatting functions where the format string is not hard-coded.
135+
136+
.. code-block:: ql
137+
138+
import go
139+
140+
from StringOps::Formatting::Range format, CallExpr call, Expr formatString
141+
where
142+
call.getTarget() = format and
143+
formatString = call.getArgument(format.getFormatStringIndex()) and
144+
not exists(DataFlow::Node source, DataFlow::Node sink |
145+
DataFlow::localFlow(source, sink) and
146+
source.asExpr() instanceof StringLit and
147+
sink.asExpr() = formatString
148+
)
149+
select call, "Argument to String format method isn't hard-coded."
150+
151+
Exercises
152+
~~~~~~~~~
153+
154+
Exercise 1: Write a query that finds all hard-coded strings used to create a ``url.URL``, using local data flow. (`Answer <#exercise-1>`__)
155+
156+
Global data flow
157+
----------------
158+
159+
Global data flow tracks data flow throughout the entire program, and is therefore more powerful than local data flow. However, global data flow is less precise than local data flow, and the analysis typically requires significantly more time and memory to perform.
160+
161+
.. pull-quote:: Note
162+
163+
.. include:: ../reusables/path-problem.rst
164+
165+
Using global data flow
166+
~~~~~~~~~~~~~~~~~~~~~~
167+
168+
The global data flow library is used by implementing the signature ``DataFlow::ConfigSig`` and applying the module ``DataFlow::Global<ConfigSig>``:
169+
170+
.. code-block:: ql
171+
172+
import go
173+
174+
module MyFlowConfiguration implements DataFlow::ConfigSig {
175+
predicate isSource(DataFlow::Node source) {
176+
...
177+
}
178+
179+
predicate isSink(DataFlow::Node sink) {
180+
...
181+
}
182+
}
183+
184+
module MyFlow = DataFlow::Global<MyFlowConfiguration>;
185+
186+
These predicates are defined in the configuration:
187+
188+
- ``isSource`` - defines where data may flow from.
189+
- ``isSink`` - defines where data may flow to.
190+
- ``isBarrier`` - optional, restricts the data flow.
191+
- ``isAdditionalFlowStep`` - optional, adds additional flow steps.
192+
193+
The data flow analysis is performed using the predicate ``flow(DataFlow::Node source, DataFlow::Node sink)``:
194+
195+
.. code-block:: ql
196+
197+
from DataFlow::Node source, DataFlow::Node sink
198+
where MyFlow::flow(source, sink)
199+
select source, "Data flow to $@.", sink, sink.toString()
200+
201+
Using global taint tracking
202+
~~~~~~~~~~~~~~~~~~~~~~~~~~~
203+
204+
Global taint tracking is to global data flow what local taint tracking is to local data flow. That is, global taint tracking extends global data flow with additional non-value-preserving steps. The global taint tracking library is used by applying the module ``TaintTracking::Global<ConfigSig>`` to your configuration instead of ``DataFlow::Global<ConfigSig>``:
205+
206+
.. code-block:: ql
207+
208+
import go
209+
210+
module MyFlowConfiguration implements DataFlow::ConfigSig {
211+
predicate isSource(DataFlow::Node source) {
212+
...
213+
}
214+
215+
predicate isSink(DataFlow::Node sink) {
216+
...
217+
}
218+
}
219+
220+
module MyFlow = TaintTracking::Global<MyFlowConfiguration>;
221+
222+
The resulting module has an identical signature to the one obtained from ``DataFlow::Global<ConfigSig>``.
223+
224+
Flow sources
225+
~~~~~~~~~~~~
226+
227+
The data flow library contains some predefined flow sources. The class ``RemoteFlowSource`` (defined in ``semmle.code.java.dataflow.FlowSources``) represents data flow sources that may be controlled by a remote user, which is useful for finding security problems.
228+
229+
Examples
230+
~~~~~~~~
231+
232+
This query shows a taint-tracking configuration that uses remote user input as data sources.
233+
234+
.. code-block:: ql
235+
236+
import go
237+
238+
module MyFlowConfiguration implements DataFlow::ConfigSig {
239+
predicate isSource(DataFlow::Node source) {
240+
source instanceof RemoteFlowSource
241+
}
242+
243+
...
244+
}
245+
246+
module MyTaintFlow = TaintTracking::Global<MyFlowConfiguration>;
247+
248+
Exercises
249+
~~~~~~~~~
250+
251+
Exercise 2: Write a query that finds all hard-coded strings used to create a ``url.URL``, using global data flow. (`Answer <#exercise-2>`__)
252+
253+
Exercise 3: Write a class that represents flow sources from ``os.Getenv(..)``. (`Answer <#exercise-3>`__)
254+
255+
Exercise 4: Using the answers from 2 and 3, write a query which finds all global data flows from ``os.Getenv`` to ``url.URL``. (`Answer <#exercise-4>`__)
256+
257+
Answers
258+
-------
259+
260+
Exercise 1
261+
~~~~~~~~~~
262+
263+
.. code-block:: ql
264+
265+
import go
266+
267+
from Function urlParse, Expr arg, StringLit rawURL, CallExpr call
268+
where
269+
(
270+
urlParse.hasQualifiedName("url", "Parse") or
271+
urlParse.hasQualifiedName("url", "ParseRequestURI")
272+
) and
273+
call.getTarget() = urlParse and
274+
arg = call.getArgument(0) and
275+
DataFlow::localFlow(DataFlow::exprNode(rawURL), DataFlow::exprNode(arg))
276+
select call.getArgument(0)
277+
278+
Exercise 2
279+
~~~~~~~~~~
280+
281+
.. code-block:: ql
282+
283+
import go
284+
285+
module LiteralToURLConfig implements DataFlow::ConfigSig {
286+
predicate isSource(DataFlow::Node source) {
287+
source.asExpr() instanceof StringLit
288+
}
289+
290+
predicate isSink(DataFlow::Node sink) {
291+
exists(Function urlParse, CallExpr call |
292+
(
293+
urlParse.hasQualifiedName("url", "Parse") or
294+
urlParse.hasQualifiedName("url", "ParseRequestURI")
295+
) and
296+
call.getTarget() = urlParse and
297+
sink.asExpr() = call.getArgument(0)
298+
)
299+
}
300+
}
301+
302+
module LiteralToURLFlow = DataFlow::Global<LiteralToURLConfig>;
303+
304+
from DataFlow::Node src, DataFlow::Node sink
305+
where LiteralToURLFlow::flow(src, sink)
306+
select src, "This string constructs a URL $@.", sink, "here"
307+
308+
Exercise 3
309+
~~~~~~~~~~
310+
311+
.. code-block:: ql
312+
313+
import go
314+
315+
class GetenvSource extends CallExpr {
316+
GetenvSource() {
317+
exists(Function m | m = this.getTarget() |
318+
m.hasQualifiedName("os", "Getenv")
319+
)
320+
}
321+
}
322+
323+
Exercise 4
324+
~~~~~~~~~~
325+
326+
.. code-block:: ql
327+
328+
import go
329+
330+
class GetenvSource extends CallExpr {
331+
GetenvSource() {
332+
exists(Function m | m = this.getTarget() |
333+
m.hasQualifiedName("os", "Getenv")
334+
)
335+
}
336+
}
337+
338+
module GetenvToURLConfig implements DataFlow::ConfigSig {
339+
predicate isSource(DataFlow::Node source) {
340+
source instanceof GetenvSource
341+
}
342+
343+
predicate isSink(DataFlow::Node sink) {
344+
exists(Function urlParse, CallExpr call |
345+
(
346+
urlParse.hasQualifiedName("url", "Parse") or
347+
urlParse.hasQualifiedName("url", "ParseRequestURI")
348+
) and
349+
call.getTarget() = urlParse and
350+
sink.asExpr() = call.getArgument(0)
351+
)
352+
}
353+
}
354+
}
355+
356+
module GetenvToURLFlow = DataFlow::Global<GetenvToURLConfig>;
357+
358+
from DataFlow::Node src, DataFlow::Node sink
359+
where GetenvToURLFlow::flow(src, sink)
360+
select src, "This environment variable constructs a URL $@.", sink, "here"
361+
362+
Further reading
363+
---------------
364+
365+
- `Exploring data flow with path queries <https://docs.github.com/en/code-security/codeql-for-vs-code/getting-started-with-codeql-for-vs-code/exploring-data-flow-with-path-queries>`__ in the GitHub documentation.
366+
367+
368+
.. include:: ../reusables/go-further-reading.rst
369+
.. include:: ../reusables/codeql-ref-tools-further-reading.rst

‎docs/codeql/codeql-language-guides/codeql-for-go.rst

Lines changed: 2 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat
1111
basic-query-for-go-code
1212
codeql-library-for-go
1313
abstract-syntax-tree-classes-for-working-with-go-programs
14-
modeling-data-flow-in-go-libraries
14+
analyzing-data-flow-in-go
1515
customizing-library-models-for-go
1616

1717
- :doc:`Basic query for Go code <basic-query-for-go-code>`: Learn to write and run a simple CodeQL query.
@@ -22,7 +22,6 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat
2222

2323
- :doc:`Abstract syntax tree classes for working with Go programs <abstract-syntax-tree-classes-for-working-with-go-programs>`: CodeQL has a large selection of classes for representing the abstract syntax tree of Go programs.
2424

25-
- :doc:`Modeling data flow in Go libraries <modeling-data-flow-in-go-libraries>`: When analyzing a Go program, CodeQL does not examine the source code for external packages.
26-
To track the flow of untrusted data through a library, you can create a model of the library.
25+
- :doc:`Analyzing data flow in Go <analyzing-data-flow-in-go>`: You can use CodeQL to track the flow of data through a Go program to its use.
2726

2827
- :doc:`Customizing library models for Go <customizing-library-models-for-go>`: You can model frameworks and libraries that your codebase depends on using data extensions and publish them as CodeQL model packs.

‎docs/codeql/writing-codeql-queries/about-data-flow-analysis.rst

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ See the following tutorials for more information about analyzing data flow in sp
1818

1919
- ":ref:`Analyzing data flow in C/C++ <analyzing-data-flow-in-cpp>`"
2020
- ":ref:`Analyzing data flow in C# <analyzing-data-flow-in-csharp>`"
21+
- ":ref:`Analyzing data flow in Go <analyzing-data-flow-in-go>`"
2122
- ":ref:`Analyzing data flow in Java/Kotlin <analyzing-data-flow-in-java>`"
2223
- ":ref:`Analyzing data flow in JavaScript/TypeScript <analyzing-data-flow-in-javascript-and-typescript>`"
2324
- ":ref:`Analyzing data flow in Python <analyzing-data-flow-in-python>`"

0 commit comments

Comments
 (0)