Skip to content

Commit e7d267e

Browse files
asgerfasgerf
committed
JS: Add migration guide and change note
1 parent abea019 commit e7d267e

File tree

4 files changed

+313
-1
lines changed

4 files changed

+313
-1
lines changed

‎docs/codeql/codeql-language-guides/codeql-for-javascript.rst

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -18,6 +18,7 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat
1818
abstract-syntax-tree-classes-for-working-with-javascript-and-typescript-programs
1919
data-flow-cheat-sheet-for-javascript
2020
customizing-library-models-for-javascript
21+
migrating-javascript-dataflow-queries
2122

2223
- :doc:`Basic query for JavaScript and TypeScript code <basic-query-for-javascript-code>`: Learn to write and run a simple CodeQL query.
2324

@@ -37,4 +38,6 @@ Experiment and learn how to write effective and efficient queries for CodeQL dat
3738

3839
- :doc:`Data flow cheat sheet for JavaScript <data-flow-cheat-sheet-for-javascript>`: This article describes parts of the JavaScript libraries commonly used for variant analysis and in data flow queries.
3940

40-
- :doc:`Customizing library models for JavaScript <customizing-library-models-for-javascript>`: You can model frameworks and libraries that your codebase depends on using data extensions and publish them as CodeQL model packs.
41+
- :doc:`Customizing library models for JavaScript <customizing-library-models-for-javascript>`: You can model frameworks and libraries that your codebase depends on using data extensions and publish them as CodeQL model packs.
42+
43+
- :doc:`Migrating JavaScript dataflow queries <migrating-javascript-dataflow-queries>`: Guide on migrating data flow queries to the new data flow library.

‎docs/codeql/codeql-language-guides/migrating-javascript-dataflow-queries.rst

Lines changed: 298 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,298 @@
1+
.. _migrating-javascript-dataflow-queries:
2+
3+
Migrating JavaScript Dataflow Queries
4+
=====================================
5+
6+
The JavaScript analysis used to have its own data flow library, which differed from the shared data flow
7+
library used by other languages. This library has now been deprecated in favor of the shared library.
8+
9+
This article explains how to migrate JavaScript data flow queries to use the shared data flow library,
10+
and some important differences to be aware of. Note that the article on :ref:`analyzing data flow in JavaScript and TypeScript <analyzing-data-flow-in-javascript-and-typescript>`
11+
provides a general guide to new data flow library, whereas this article aims to help with migrating existing queries from the old data flow library.
12+
13+
Note that the ``DataFlow::Configuration`` class is still backed by the original data flow library, but has been marked as deprecated.
14+
This means data flow queries using this class will continue work, albeit with deprecation warnings, until the 1-year deprecation period expires in early 2026.
15+
It is recommended that all custom queries are migrated before this time, to ensure they continue to work in the future.
16+
17+
Data flow queries should be migrated to use ``DataFlow::ConfigSig``-style modules instead of the ``DataFlow::Configuration`` class.
18+
This is identical to the interface found in other languages.
19+
When making this switch, the query will become backed by the shared data flow library instead. That is, data flow queries will only work
20+
with the shared data flow library when they have been migrated to ``ConfigSig``-style, as shown in the following table:
21+
22+
.. list-table:: Data flow libraries
23+
:widths: 20 80
24+
:header-rows: 1
25+
26+
* - API
27+
- Implementation
28+
* - ``DataFlow::Configuration``
29+
- Old library (deprecated, to be removed in early 2026)
30+
* - ``DataFlow::ConfigSig``
31+
- Shared library
32+
33+
A straight-forward translation to ``DataFlow::ConfigSig``-style is usually possible, although there are some complications
34+
that may cause the query to behave differently.
35+
We'll first cover some straight-forward migration examples, and then go over some of the complications that may arise.
36+
37+
Simple migration example
38+
------------------------
39+
40+
A simple example of a query using the old data flow library is shown below:
41+
42+
.. code-block:: ql
43+
44+
/** @kind path-problem */
45+
import javascript
46+
import DataFlow::PathGraph
47+
48+
class MyConfig extends DataFlow::Configuration {
49+
MyConfig() { this = "MyConfig" }
50+
51+
override predicate isSource(DataFlow::Node node) { ... }
52+
53+
override predicate isSink(DataFlow::Node node) { ... }
54+
}
55+
56+
from MyConfig cfg, DataFlow::PathNode source, DataFlow::PathNode sink
57+
where cfg.hasFlowPath(source, sink)
58+
select sink, source, sink, "Flow found"
59+
60+
With the new style this would look like this:
61+
62+
.. code-block:: ql
63+
64+
/** @kind path-problem */
65+
import javascript
66+
67+
module MyConfig implements DataFlow::ConfigSig {
68+
predicate isSource(DataFlow::Node node) { ... }
69+
70+
predicate isSink(DataFlow::Node node) { ... }
71+
}
72+
73+
module MyFlow = DataFlow::Global<MyConfig>;
74+
75+
import MyFlow::PathGraph
76+
77+
from MyFlow::PathNode source, MyFlow::PathNode sink
78+
where MyFlow::flowPath(source, sink)
79+
select sink, source, sink, "Flow found"
80+
81+
The changes can be summarised as:
82+
83+
- The ``DataFlow::Configuration`` class was replaced with a module implementing ``DataFlow::ConfigSig``.
84+
- The characteristic predicate was removed (modules have no characteristic predicates)
85+
- Predicates such as ``isSource`` no longer have the ``override`` keyword (as they are defined in a module now).
86+
- The configuration module is being passed to ``DataFlow::Global``, resulting in a new module, called ``MyFlow`` in this example.
87+
- The query imports ``MyFlow::PathGraph`` instead of ``DataFlow::PathGraph``.
88+
- The ``MyConfig cfg`` variable was removed from the ``from`` clause.
89+
- The ``hasFlowPath`` call was replaced with ``MyFlow::flowPath``.
90+
- The type ``DataFlow::PathNode`` was replaced with ``MyFlow::PathNode``.
91+
92+
With these changes, we have produced an equivalent query that is backed by the new data flow library.
93+
94+
Taint tracking
95+
--------------
96+
97+
For configuration classes extending ``TaintTracking::Configuration``, the migration is similar but with few differences:
98+
99+
- The ``TaintTracking::Global`` module should be used instead of ``DataFlow::Global``.
100+
- The ``isSanitizer`` predicate should be renamed to ``isBarrier``.
101+
- The ``isAdditionalTaintStep`` predicate should be renamed to ``isAdditionalFlowStep``.
102+
103+
Note that there is no such thing as ``TaintTracking::ConfigSig``. The ``DataFlow::ConfigSig`` interface is used for both data flow and taint tracking.
104+
105+
For example:
106+
107+
.. code-block:: ql
108+
109+
class MyConfig extends TaintTracking::Configuration {
110+
predicate isSanitizer(DataFlow::Node node) { ... }
111+
predicate isAdditionalTaintStep(DataFlow::Node node1, DataFlow::Node node2) { ... }
112+
...
113+
}
114+
115+
The above configuration can be migrated to the shared data flow library as follows:
116+
117+
.. code-block:: ql
118+
119+
module MyConfig implements DataFlow::ConfigSig {
120+
predicate isBarrier(DataFlow::Node node) { ... }
121+
predicate isAdditionalFlowStep(DataFlow::Node node1, DataFlow::Node node2) { ... }
122+
...
123+
}
124+
125+
module MyFlow = TaintTracking::Global<MyConfig>;
126+
127+
128+
Flow labels and flow states
129+
---------------------------
130+
131+
The ``DataFlow::FlowLabel`` class has been deprecated. Queries that relied on flow labels should use the new `flow state` concept instead.
132+
This is done by implementing ``DataFlow::StateConfigSig`` instead of ``DataFlow::ConfigSig``, and passing the module to ``DataFlow::GlobalWithState``
133+
or ``TaintTracking::GlobalWithState``. See :ref:`using flow state <using-flow-labels-for-precise-data-flow-analysis>` for more details about flow state.
134+
135+
Some changes to be aware of:
136+
137+
- The 4-argument version of ``isAdditionalFlowStep`` now takes parameter in a different order.
138+
It now takes ``node1, state1, node2, state2`` instead of ``node1, node2, state1, state2``.
139+
- Taint steps apply to all flow states, not just the ``taint`` flow label. See more details further down in this article.
140+
141+
Barrier guards
142+
--------------
143+
144+
The predicates ``isBarrierGuard`` and ``isSanitizerGuard`` have been removed.
145+
146+
Instead, the ``isBarrier`` predicate must used to define all barriers. To do this, barrier guards can be reduced to a set of barrier nodes using the ``DataFlow::MakeBarrierGuard`` module.
147+
148+
For example, consider this data flow configuration using a barrier guard:
149+
150+
.. code-block:: ql
151+
152+
class MyConfig extends DataFlow::Configuration {
153+
override predicate isBarrierGuard(DataFlow::BarrierGuardNode node) {
154+
node instanceof MyBarrierGuard
155+
}
156+
..
157+
}
158+
159+
class MyBarrierGuard extends DataFlow::BarrierGuardNode {
160+
MyBarrierGuard() { ... }
161+
162+
override predicate blocks(Expr e, boolean outcome) { ... }
163+
}
164+
165+
This can be migrated to the shared data flow library as follows:
166+
167+
.. code-block:: ql
168+
169+
module MyConfig implements DataFlow::ConfigSig {
170+
predicate isBarrier(DataFlow::Node node) {
171+
node = DataFlow::MakeBarrierGuard<MyBarrierGuard>::getABarrierNode()
172+
}
173+
..
174+
}
175+
176+
class MyBarrierGuard extends DataFlow::Node {
177+
MyBarrierGuard() { ... }
178+
179+
predicate blocksExpr(Expr e, boolean outcome) { ... }
180+
}
181+
182+
The changes can be summarised as:
183+
- The contents of ``isBarrierGuard`` have been moved to ``isBarrier``.
184+
- The ``node instanceof MyBarrierGuard`` check was replaced with ``node = DataFlow::MakeBarrierGuard<MyBarrierGuard>::getABarrierNode()``.
185+
- The ``MyBarrierGuard`` class no longer has ``DataFlow::BarrierGuardNode`` as a base class. We simply use ``DataFlow::Node`` instead.
186+
- The ``blocks`` predicate has been renamed to ``blocksExpr`` and no longer has the ``override`` keyword.
187+
188+
See :ref:`using flow state <using-flow-labels-for-precise-data-flow-analysis>` for examples of how to use barrier guards with flow state.
189+
190+
Query-specific load and store steps
191+
-----------------------------------
192+
193+
The predicates ``isAdditionalLoadStep``, ``isAdditionalStoreStep``, and ``isAdditionalLoadStoreStep`` have been removed. There is no way to emulate the original behaviour.
194+
195+
Library models can still contribute such steps, but they will be applicable to all queries. Also see the section on jump steps further down.
196+
197+
Changes in behaviour
198+
--------------------
199+
200+
When the query has been migrated to the new interface, it may seem to behave differently due to some technical differences in the internals of
201+
the two data flow libraries. The most significant changes are described below.
202+
203+
Taint steps now propagate all flow states
204+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
205+
206+
There's an important change from the old data flow library when using flow state and taint-tracking together.
207+
208+
When using when using ``TaintTracking::GlobalWithState``, all flow states can propagate along taint steps.
209+
In the old data flow library, only the ``taint`` flow label could propagate along taint steps.
210+
A straight-forward translation of such a query may therefore result in new flow paths being found, which might be unexpected.
211+
212+
To emulate the old behaviour, use ``DataFlow::GlobalWithState`` instead of ``TaintTracking::GlobalWithState``,
213+
and manually add taint steps using ``isAdditionalFlowStep``. The predicate ``TaintTracking::defaultTaintStep`` can be used to access to the set of taint steps.
214+
215+
For example:
216+
217+
.. code-block:: ql
218+
219+
module MyConfig implements DataFlow::StateConfigSig {
220+
class FlowState extends string {
221+
FlowState() { this = ["taint", "foo"] }
222+
}
223+
224+
predicate isAdditionalFlowStep(DataFlow::Node node1, FlowState state1, DataFlow::Node node2, FlowState state2) {
225+
// Allow taint steps to propagate the "taint" flow state
226+
TaintTracking::defaultTaintStep(node1, node2) and
227+
state1 = "taint" and
228+
state2 = state
229+
}
230+
231+
...
232+
}
233+
234+
module MyFlow = DataFlow::GlobalWithState<MyConfig>;
235+
236+
237+
Jump steps across function boundaries
238+
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
239+
240+
When a flow step crosses a function boundary, that is, it starts and ends in two different functions, it will now be classified as a "jump" step.
241+
242+
Jump steps can be problematic in some cases. Roughly speaking, the data flow library will "forget" which call site it came from when following a jump step.
243+
This can lead to spurious flow paths that go into a function through one call site, and back out of a different call site.
244+
245+
If the step was generated by a library model, that is, the step is applicable to all queries, this is best mitigated by converting the step to a flow summary.
246+
For example, the following library model adds a taint step from ``x`` to ``y`` in ``foo.bar(x, y => {})``:
247+
248+
.. code-block:: ql
249+
250+
class MyStep extends TaintTracking::SharedTaintStep {
251+
override predicate step(DataFlow::Node node1, DataFlow::Node node2) {
252+
exists(DataFlow::CallNode call |
253+
call = DataFlow::moduleMember("foo", "bar").getACall() and
254+
node1 = call.getArgument(0) and
255+
node2 = call.getCallback(1).getParameter(0)
256+
)
257+
}
258+
}
259+
260+
Because this step crosses a function boundary, it becomes a jump step. This can be avoided by converting it to a flow summary as follows:
261+
262+
.. code-block:: ql
263+
264+
class MySummary extends DataFlow::SummarizedCallable {
265+
MySummary() { this = "MySummary" }
266+
267+
override DataFlow::CallNode getACall() { result = DataFlow::moduleMember("foo", "bar").getACall() }
268+
269+
override predicate propagatesFlow(string input, string output, boolean preservesValue) {
270+
input = "Argument[this]" and
271+
output = "Argument[1].Parameter[0]" and
272+
preservesValue = false // taint step
273+
}
274+
}
275+
276+
See :ref:`customizing library models for JavaScript <customizing-library-models-for-javascript>` for details about the format of the ``input`` and ``output`` strings.
277+
The aforementioned article also provides guidance on how to store the flow summary in a data extension.
278+
279+
For query-specific steps that cross function boundaries, that is, steps added with ``isAdditionalFlowStep``, there is currently no way to emulate the original behaviour.
280+
A possible workaround is to convert the query-specific step to a flow summary. In this case it should be stored in a data extension to avoid performance issues, although this also means
281+
that all other queries will be able to use the flow summary.
282+
283+
Barriers block all flows
284+
~~~~~~~~~~~~~~~~~~~~~~~~
285+
286+
In the shared data flow library, a barrier blocks all flows, even if the tracked value is inside a content.
287+
288+
In the old data flow library, only barriers specific to the ``data`` flow label blocked flows when the tracked value was inside a content.
289+
290+
This rarely has significant impact, but some users may observe some result changes because of this.
291+
292+
There is currently no way to emulate the original behavour.
293+
294+
Further reading
295+
---------------
296+
297+
- :ref:`Analyzing data flow in JavaScript and TypeScript <analyzing-data-flow-in-javascript-and-typescript>` provides a general guide to the new data flow library.
298+
- :ref:`Using flow state for precise data flow analysis <using-flow-labels-for-precise-data-flow-analysis>` provides a general guide on using flow state.

‎javascript/ql/lib/change-notes/2025-01-07-dataflow-deprecation.md

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,6 @@
1+
---
2+
category: deprecated
3+
---
4+
* Custom data flow queries will need to be migrated in order to use the shared data flow library. Until migrated, such queries will compile with deprecation warnings and run with a
5+
deprecated copy of the old data flow library. The deprecation layer will be removed in early 2026, after which any unmigrated queries will stop working.
6+
See more information in the [migration guide](https://codeql.github.com/docs/codeql-language-guides/migrating-javascript-dataflow-queries).

‎javascript/ql/lib/change-notes/2025-01-07-dataflow.md

Lines changed: 5 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,5 @@
1+
---
2+
category: majorAnalysis
3+
---
4+
* All data flow queries are now using the same underlying data flow library as the other languages analyses, replacing the old one written specifically for JavaScript/TypeScript.
5+
This is a significant change and users may consequently observe differences in the alerts generated by the analysis.

0 commit comments

Comments
 (0)