Skip to content

Commit cf4badc

Browse files
LiedtkeV8-internal LUCI CQ
authored andcommitted
LiveTests: Properly print unrecognized errors and wasm tags
A live test failing due to a too high failure rate reports sth. like: Observed failures: 10x RuntimeError: memory access out of bounds 9x Error: Timed out 4x RuntimeError: float unrepresentable in integer range If the runner did not include any line containing "Error:", these failures would just not appear in the "Observed failures" (but they'd still be included in the failure rate). With this change the observed failures list will contain an extra entry for failures without such "Error:"-line (if there is any). An extra magic entry is added for uncaught wasm exceptions from the exception-handling proposal. As these are opaque objects in JS, they don't create a nice error message nor do they have a proper stack trace. Change-Id: I838857c0ec38c91a4085c61b1e9aae2f98b50daf Reviewed-on: https://chrome-internal-review.googlesource.com/c/v8/fuzzilli/+/7967828 Auto-Submit: Matthias Liedtke <[email protected]> Reviewed-by: Carl Smith <[email protected]> Commit-Queue: Carl Smith <[email protected]>
1 parent fa2318a commit cf4badc

File tree

1 file changed

+9
-5
lines changed

1 file changed

+9
-5
lines changed

Tests/FuzzilliTests/LiveTests.swift

Lines changed: 9 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -17,7 +17,7 @@ import XCTest
1717

1818
class LiveTests: XCTestCase {
1919
enum ExecutionResult {
20-
case failed(failureMessage: String?)
20+
case failed(failureMessage: String)
2121
case succeeded
2222
}
2323

@@ -142,9 +142,7 @@ class LiveTests: XCTestCase {
142142
programs[i].program.storeToDisk(atPath: path.appendingPathComponent("failure_\(i).fzil").path)
143143
}
144144
failures += 1
145-
if let message = message {
146-
failureMessages[message] = (failureMessages[message] ?? 0) + 1
147-
}
145+
failureMessages[message] = (failureMessages[message] ?? 0) + 1
148146
case .succeeded:
149147
break
150148
}
@@ -180,6 +178,12 @@ class LiveTests: XCTestCase {
180178
if line.contains("Error:") {
181179
// Remove anything after a potential 2nd ":", which is usually testcase dependent content, e.g. "SyntaxError: Invalid regular expression: /ep{}[]Z7/: Incomplete quantifier"
182180
signature = line.split(separator: ":")[0...1].joined(separator: ":")
181+
} else if line.contains("[object WebAssembly.Exception]") {
182+
// An uncaught thrown wasm tag results in an output like this in d8:
183+
// undefined:0: [object WebAssembly.Exception]
184+
// Treat anything that contains the WebAssembly.Exception object as one
185+
// special signature.
186+
signature = "<Uncaught WebAssembly.Exception>"
183187
}
184188
}
185189

@@ -193,7 +197,7 @@ class LiveTests: XCTestCase {
193197
print(fuzzilProgram)
194198
}
195199

196-
return .failed(failureMessage: signature)
200+
return .failed(failureMessage: signature ?? "<Unrecognized error>")
197201
}
198202
} catch {
199203
XCTFail("Could not execute script: \(error)")

0 commit comments

Comments
 (0)