Skip to content

Commit 79b4c1b

Browse files
jservjserv
committed
CI: Add explicit TSAN race detection validation
Add comprehensive ThreadSanitizer output validation to prevent silent test failures. TSAN tests now explicitly check for data races and fail immediately with diagnostic output instead of masking errors. Implementation: - Capture TSAN stderr/stdout to log files for analysis - Pattern match race indicators: "ThreadSanitizer: data race", "ThreadSanitizer: race on", "WARNING: ThreadSanitizer:" - Exit with status 1 immediately upon race detection - Display race context (10 lines) for debugging - Progress indicators for 3-tier validation (Interpreter, JIT-T1, JIT-T2) Platform-Specific Handling: - Linux (x64/ARM64): Use setarch -R for ASLR mitigation (already gated in Makefile with ifeq ($(UNAME_S),Linux)) - macOS: NO setarch (not available), rely on MAP_FIXED allocations at 0x150000000000; gracefully handle SIP restrictions by distinguishing MAP_FAILED errors from actual race conditions Race Detection Patterns: ThreadSanitizer: data race # Standard race report ThreadSanitizer: race on # Race on specific object WARNING: ThreadSanitizer: # General TSAN warnings Error Handling (macOS): MAP_FAILED # mmap failure unexpected memory mapping # TSAN shadow conflict FATAL: ThreadSanitizer # Initialization failure → Skip test with warning (SIP restriction) → Still fail hard on actual races Benefits: - Immediate failure on race detection (fail-fast principle) - Clear diagnostic output in CI logs with race location/context - Platform-aware: Linux uses setarch -R, macOS handles SIP gracefully - No silent failures: Previously masked errors now cause test failure - Debugging support: Log files preserved for post-mortem analysis Validates race condition fixes from: - 2a2a5c4: TSAN with FULL4G and T2C support - f1b685e: ARM64 TSAN support and JIT cache coherency - 669efc1: Build system regressions (setarch gating, TSAN compatibility)
1 parent 30ac025 commit 79b4c1b

File tree

1 file changed

+134
-0
lines changed

1 file changed

+134
-0
lines changed

‎.github/workflows/main.yml‎

Lines changed: 134 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -266,6 +266,46 @@ jobs:
266266
make distclean && make ENABLE_UBSAN=1 check $PARALLEL
267267
make ENABLE_JIT=1 clean && make ENABLE_JIT=1 ENABLE_UBSAN=1 check $PARALLEL
268268
269+
- name: ThreadSanitizer race detection test
270+
if: success() || failure()
271+
env:
272+
CC: ${{ steps.install_cc.outputs.cc }}
273+
run: |
274+
set -o pipefail
275+
276+
# TSAN requires ASLR disabled to prevent allocations in shadow memory
277+
# Interpreter with FULL4G: Basic race detection across emulation core
278+
echo "=== TSAN Test 1/3: Interpreter + FULL4G ==="
279+
make distclean && setarch -R make ENABLE_TSAN=1 ENABLE_FULL4G=1 check $PARALLEL 2>&1 | tee tsan-interpreter.log
280+
if grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-interpreter.log; then
281+
echo "ERROR: Data race detected in interpreter mode!"
282+
grep -A 10 "ThreadSanitizer:" tsan-interpreter.log
283+
exit 1
284+
fi
285+
echo "✓ No races detected in interpreter mode"
286+
287+
# JIT tier-1: Race detection in template-based JIT compilation
288+
echo "=== TSAN Test 2/3: JIT Tier-1 ==="
289+
make ENABLE_JIT=1 clean && setarch -R make ENABLE_TSAN=1 ENABLE_FULL4G=1 ENABLE_JIT=1 check $PARALLEL 2>&1 | tee tsan-jit.log
290+
if grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-jit.log; then
291+
echo "ERROR: Data race detected in JIT tier-1 mode!"
292+
grep -A 10 "ThreadSanitizer:" tsan-jit.log
293+
exit 1
294+
fi
295+
echo "✓ No races detected in JIT tier-1 mode"
296+
297+
# JIT tier-2 (T2C): Race detection across LLVM compilation thread
298+
echo "=== TSAN Test 3/3: JIT Tier-2 (T2C) ==="
299+
make ENABLE_JIT=1 clean && setarch -R make ENABLE_TSAN=1 ENABLE_FULL4G=1 ENABLE_JIT=1 ENABLE_T2C=1 check $PARALLEL 2>&1 | tee tsan-t2c.log
300+
if grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-t2c.log; then
301+
echo "ERROR: Data race detected in JIT tier-2 (T2C) mode!"
302+
grep -A 10 "ThreadSanitizer:" tsan-t2c.log
303+
exit 1
304+
fi
305+
echo "✓ No races detected in JIT tier-2 (T2C) mode"
306+
307+
echo "=== All TSAN tests passed ==="
308+
269309
- name: boot Linux kernel test
270310
if: success()
271311
env:
@@ -384,6 +424,33 @@ jobs:
384424
make ENABLE_JIT=1 clean && make ENABLE_EXT_A=0 ENABLE_JIT=1 check $PARALLEL
385425
make ENABLE_JIT=1 clean && make ENABLE_EXT_F=0 ENABLE_JIT=1 check $PARALLEL
386426
make ENABLE_JIT=1 clean && make ENABLE_EXT_C=0 ENABLE_JIT=1 check $PARALLEL
427+
# TSAN on ARM64: Fixed memory layout (0x150000000000 for main, 0x151000000000 for JIT)
428+
set -o pipefail
429+
echo "=== TSAN Test 1/3: Interpreter + FULL4G (ARM64) ==="
430+
make distclean && setarch -R make ENABLE_TSAN=1 ENABLE_FULL4G=1 check $PARALLEL 2>&1 | tee tsan-interpreter.log
431+
if grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-interpreter.log; then
432+
echo "ERROR: Data race detected in interpreter mode!"
433+
grep -A 10 "ThreadSanitizer:" tsan-interpreter.log
434+
exit 1
435+
fi
436+
echo "✓ No races detected in interpreter mode"
437+
echo "=== TSAN Test 2/3: JIT Tier-1 (ARM64) ==="
438+
make ENABLE_JIT=1 clean && setarch -R make ENABLE_TSAN=1 ENABLE_FULL4G=1 ENABLE_JIT=1 check $PARALLEL 2>&1 | tee tsan-jit.log
439+
if grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-jit.log; then
440+
echo "ERROR: Data race detected in JIT tier-1 mode!"
441+
grep -A 10 "ThreadSanitizer:" tsan-jit.log
442+
exit 1
443+
fi
444+
echo "✓ No races detected in JIT tier-1 mode"
445+
echo "=== TSAN Test 3/3: JIT Tier-2 (T2C) (ARM64) ==="
446+
make ENABLE_JIT=1 clean && setarch -R make ENABLE_TSAN=1 ENABLE_FULL4G=1 ENABLE_JIT=1 ENABLE_T2C=1 check $PARALLEL 2>&1 | tee tsan-t2c.log
447+
if grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-t2c.log; then
448+
echo "ERROR: Data race detected in JIT tier-2 (T2C) mode!"
449+
grep -A 10 "ThreadSanitizer:" tsan-t2c.log
450+
exit 1
451+
fi
452+
echo "✓ No races detected in JIT tier-2 (T2C) mode"
453+
echo "=== All TSAN tests passed (ARM64) ==="
387454
388455
macOS-arm64:
389456
needs: [detect-code-related-file-changes]
@@ -573,6 +640,73 @@ jobs:
573640
make distclean && make ENABLE_UBSAN=1 check $PARALLEL
574641
make ENABLE_JIT=1 clean && make ENABLE_JIT=1 ENABLE_UBSAN=1 check $PARALLEL
575642
643+
- name: ThreadSanitizer race detection test
644+
if: (success() || failure()) && steps.install_cc.outputs.cc == 'clang' # Only clang supports TSAN on macOS
645+
env:
646+
CC: ${{ steps.install_cc.outputs.cc }}
647+
run: |
648+
set -o pipefail
649+
650+
# macOS TSAN: Fixed memory at 0x150000000000 (main) and 0x151000000000 (JIT)
651+
# Note: ASLR disabled via mmap(MAP_FIXED), but SIP may restrict full ASLR control on GitHub runners
652+
653+
# Test 1: Interpreter + FULL4G
654+
echo "=== TSAN Test 1/3: Interpreter + FULL4G (macOS ARM64) ==="
655+
make distclean && make ENABLE_TSAN=1 ENABLE_FULL4G=1 check $PARALLEL 2>&1 | tee tsan-interpreter.log || {
656+
# Check if failure is due to MAP_FIXED restriction vs actual race
657+
if grep -q "MAP_FAILED\|unexpected memory mapping\|FATAL: ThreadSanitizer" tsan-interpreter.log; then
658+
echo "⚠️ TSAN memory allocation failed (SIP/ASLR restriction) - test skipped"
659+
else
660+
echo "ERROR: Test execution failed"
661+
cat tsan-interpreter.log
662+
exit 1
663+
fi
664+
}
665+
if [ -f tsan-interpreter.log ] && grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-interpreter.log; then
666+
echo "ERROR: Data race detected in interpreter mode!"
667+
grep -A 10 "ThreadSanitizer:" tsan-interpreter.log
668+
exit 1
669+
fi
670+
echo "✓ No races detected in interpreter mode"
671+
672+
# Test 2: JIT tier-1
673+
echo "=== TSAN Test 2/3: JIT Tier-1 (macOS ARM64) ==="
674+
make ENABLE_JIT=1 clean && make ENABLE_TSAN=1 ENABLE_FULL4G=1 ENABLE_JIT=1 check $PARALLEL 2>&1 | tee tsan-jit.log || {
675+
if grep -q "MAP_FAILED\|unexpected memory mapping\|FATAL: ThreadSanitizer" tsan-jit.log; then
676+
echo "⚠️ TSAN memory allocation failed (SIP/ASLR restriction) - test skipped"
677+
else
678+
echo "ERROR: Test execution failed"
679+
cat tsan-jit.log
680+
exit 1
681+
fi
682+
}
683+
if [ -f tsan-jit.log ] && grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-jit.log; then
684+
echo "ERROR: Data race detected in JIT tier-1 mode!"
685+
grep -A 10 "ThreadSanitizer:" tsan-jit.log
686+
exit 1
687+
fi
688+
echo "✓ No races detected in JIT tier-1 mode"
689+
690+
# Test 3: JIT tier-2 (T2C)
691+
echo "=== TSAN Test 3/3: JIT Tier-2 (T2C) (macOS ARM64) ==="
692+
make ENABLE_JIT=1 clean && make ENABLE_TSAN=1 ENABLE_FULL4G=1 ENABLE_JIT=1 ENABLE_T2C=1 check $PARALLEL 2>&1 | tee tsan-t2c.log || {
693+
if grep -q "MAP_FAILED\|unexpected memory mapping\|FATAL: ThreadSanitizer" tsan-t2c.log; then
694+
echo "⚠️ TSAN memory allocation failed (SIP/ASLR restriction) - test skipped"
695+
else
696+
echo "ERROR: Test execution failed"
697+
cat tsan-t2c.log
698+
exit 1
699+
fi
700+
}
701+
if [ -f tsan-t2c.log ] && grep -q "ThreadSanitizer: data race\|ThreadSanitizer: race on\|WARNING: ThreadSanitizer:" tsan-t2c.log; then
702+
echo "ERROR: Data race detected in JIT tier-2 (T2C) mode!"
703+
grep -A 10 "ThreadSanitizer:" tsan-t2c.log
704+
exit 1
705+
fi
706+
echo "✓ No races detected in JIT tier-2 (T2C) mode"
707+
708+
echo "=== All TSAN tests completed (macOS ARM64) ==="
709+
576710
- name: boot Linux kernel test
577711
if: success()
578712
env:

0 commit comments

Comments
 (0)