-
Notifications
You must be signed in to change notification settings - Fork 23
Adding and Running Mochi Tests
As Firefox is a highly complex software, modifying it is somewhat brittle. We rely on a set of tests to ensure changes do not break basic functionality. One set of tests are the so called Mochitests, which you can imagine as some kind of system test.
A Mochitest runs the full browser, directs it to some website, renders HTML, and executes embedded JavaScript. Our current set of Mochitests is located under taint/test/mochitest. The core idea of a mochitest is to test whether the browser currently introduces (i.e., sources work), propagates, and checks (i.e., sinks work) taints.
A simple example is the following test suite:
<!DOCTYPE HTML>
<html>
<head>
<meta charset="utf-8">
<title>Check Manual Source</title>
<script src="/tests/SimpleTest/SimpleTest.js"></script>
<link rel="stylesheet" href="/tests/SimpleTest/test.css"/>
<script>
add_task(async function test_untainted() {
check_untainted("hello");
});
add_task(async function test_tainted() {
check_tainted(String.tainted("hello"));
});
</script>
</head>
<body>
</body>
</html>Which highlights rather nicely how such a mochitest is written. Each add_task invocation takes an async test function that tests some piece of JavaScript. Here, the test_untainted test checks whether regular string literals are untainted and whether we correctly set taints when introducing them via String.tainted().
To perform various taint status checks, we added some helper functions to the SimpleTests framework.
These are:
-
function check_tainted(str): Checks whether the provided stringstris tainted. -
function check_untainted(str): The opposite, ensures the stringstris untainted -
function todo_tainted(str): The stringstrshould be tainted, but it is not due to some known limitation of Foxhound. This records a limitation but does not fail the tests. -
function check_taint_source(str, src): Assumes that the stringstris tainted (i.e., it should be preceded by acheck_taintedcall) and checks whether it originates from sourcesrc. -
function check_range_position(str, range_index, begin, end, content = null): Checks that a tainted stringstrcontains a taint range at the provided index and offsets, with an optional content equality check.
Testing for a sink working correctly and taints getting propagated into a sink is a bit more involved. This requires waiting for a __taintreport event and performing checks on those. This means we have to basically manually start/stop the test, but to make this simpler, we provide a helper function as well.
-
function setupSinkChecks(sink_names, contents, operation_index=2, flow_index=0): This first changes the tests into a mode where one has to manually terminate them upon fulfilling some condition. Then, it attaches an event listener for__taintreportevents, and upon such an event being fired, we check whether the string that ended up in the sink matches a provided string. Then, it checks whether the taint flow at positionflow_indexhas an operation at positionoperation_indexthat matches the expected sink name. Bothsink_namesandcontentsare arrays, i.e., you can test several conditions, and the test picks up the correct one based on the order of invocation. We terminate the test oncesink_names.lengthevents have been recorded.
This could look as follows:
<!DOCTYPE HTML>
<html>
<head>
<meta charset="utf-8">
<title>Test LocalStorage Sinks</title>
<script src="/tests/SimpleTest/SimpleTest.js"></script>
<link rel="stylesheet" href="/tests/SimpleTest/test.css"/>
<script>
let string_content = "hello";
let sink_names = [
"sessionStorage.setItem",
"sessionStorage.setItem(key)"
];
let strings = [
string_content, string_content
]
setupSinkChecks(sink_names, strings);
function startTest() {
let taint_string = String.tainted(string_content);
sessionStorage.setItem("tainted", taint_string);
sessionStorage.setItem(taint_string, "tainted");
}
</script>
</head>
<body onload="startTest();">
</html>Here, the functionality in startTests will result in two sink invocations, namely sessionStorage.setItem and sessionStorage.setItem(key). The test passes the string hello to both of them, i.e., the strings array contains the same string twice.
What is essential, due to the sink tests changing how the test suite completes, is that one should not mix regular tests with sink tests. The suggestion would be to split those into two files and have one file, test_fetch.html, for propagation tests and test_fetch_sinks.html for sink tests. This is currently not done, but new tests would be easier to read if they adhered to some naming scheme.
Once you have written the test file, you probably want to add it to the test suite and run it. This requires:
- Adding your test to the mochitest.ini
- Having run
./mach build
If both of these are true, you can execute your test as follows:
MOZ_UPLOAD_DIR=/tmp MOZ_HEADLESS=1 xvfb-run -- ./mach mochitest --headless taint/test/mochitest/test_fancy_new_test.htmlIf you are on a Desktop device, you can drop MOZ_HEADLESS and the invocation via xvfb-run, but on a server, you probably do not have an X server to run Foxhound regularly.
To run the whole mochitest suite, execute:
MOZ_UPLOAD_DIR=/tmp MOZ_HEADLESS=1 xvfb-run -- ./mach mochitest --headless taint/test/mochitest/Some tests are somewhat flaky, and some are known to fail at the moment. So, some timeouts about ServiceWorker and push-related tests are expected.
At times, you need to send a request to some target, for example, once you start testing the responses of fetch or XHR. To do so, you need to add a helper script that simulates a server.
A simple example is fetch_server.sjs:
function handleRequest(request, response) {
// Allow cross-origin, so you can XHR to it!
response.setHeader("Access-Control-Allow-Origin", "*", false);
// Avoid confusing cache behaviors
response.setHeader("Cache-Control", "no-cache", false);
response.setHeader("Content-Type", "text/plain", false);
switch(request.queryString) {
case "json":
let obj = { a: 3, b: 4, txt: "<b>hi</b>" };
response.write(`${JSON.stringify(obj)}`);
break;
default:
response.write("hello!");
}
}This script simulates a server, taking a request and returning a response. To add a new .sjs file, you have to add it to the support-files section of mochitest.ini. Then, you can fire off a fetch request like in the following:
let response = await fetch(`http://mochi.test:8888/tests/taint/test/mochitest/fetch_server.sjs?text`)The Firefox Source Documentation has some more pointers about writing such tests.