runtime: acquire/release C TSAN lock when calling cgo symbolizer/tracebacker

When calling into C via cmd/cgo, the generated code calls
_cgo_tsan_acquire / _cgo_tsan_release around the C call to report a
dummy lock to the C/C++ TSAN runtime. This is necessary because the
C/C++ TSAN runtime does not understand synchronization within Go and
would otherwise report false positive race reports. See the comment in
cmd/cgo/out.go for more details.

Various C functions in runtime/cgo also contain manual calls to
_cgo_tsan_acquire/release where necessary to suppress race reports.

However, the cgo symbolizer and cgo traceback functions called from
callCgoSymbolizer and cgoContextPCs, respectively, do not have any
instrumentation [1]. They call directly into user C functions with no
TSAN instrumentation.

This means they have an opportunity to report false race conditions. The
most direct way is via their argument. Both are passed a pointer to a
struct stored on the Go stack, and both write to fields of the struct.
If two calls are passed the same pointer from different threads, the C
TSAN runtime will think this is a race.

This is simple to achieve for the cgo symbolizer function, which the
new regression test does. callCgoSymbolizer is called on the standard
goroutine stack, so the argument is a pointer into the goroutine stack.
If the goroutine moves Ms between two calls, it will look like a race.

On the other hand, cgoContextPCs is called on the system stack. Each M
has a unique system stack, so for it to pass the same argument pointer
on different threads would require the first M to exit, free its stack,
and the same region of address space to be used as the stack for a new
M. Theoretically possible, but quite unlikely.

Both of these are addressed by providing a C wrapper in runtime/cgo that
calls _cgo_tsan_acquire/_cgo_tsan_release around calls to the symbolizer
and traceback functions.

There is a lot of room for future cleanup here. Most runtime/cgo
functions have manual instrumentation in their C implementation. That
could be removed in favor of instrumentation in the runtime. We could
even theoretically remove the instrumentation from cmd/cgo and move it
to cgocall. None of these are necessary, but may make things more
consistent and easier to follow.

[1] Note that the cgo traceback function called from the signal handler
via x_cgo_callers _does_ have manual instrumentation.

Fixes #73949.

Cq-Include-Trybots: luci.golang.try:gotip-freebsd-amd64,gotip-linux-amd64-longtest,gotip-windows-amd64-longtest
Change-Id: I6a6a636c9daa38f7fd00694af76b75cb93ba1886
Reviewed-on: https://go-review.googlesource.com/c/go/+/677955
Reviewed-by: Michael Knyszek <[email protected]>
Auto-Submit: Michael Pratt <[email protected]>
Reviewed-by: Ian Lance Taylor <[email protected]>
LUCI-TryBot-Result: Go LUCI <[email protected]>

Author: prattmic

src/cmd/cgo/internal/testsanitizers/testdata/tsan_tracebackctxt/main.go

@@ -0,0 +1,78 @@
+// Copyright 2025 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+package main
+
+/*
+// Defined in tracebackctxt_c.c.
+extern void C1(void);
+extern void C2(void);
+extern void tcContext(void*);
+extern void tcTraceback(void*);
+extern void tcSymbolizer(void*);
+*/
+import "C"
+
+import (
+	"fmt"
+	"runtime"
+	"sync"
+	"unsafe"
+)
+
+// Regression test for https://go.dev/issue/73949. TSAN should not report races
+// on writes to the argument passed to the symbolizer function.
+//
+// Triggering this race requires calls to the symbolizer function with the same
+// argument pointer on multiple threads. The runtime passes a stack variable to
+// this function, so that means we need to get a single goroutine to execute on
+// two threads, calling the symbolizer function on each.
+//
+// runtime.CallersFrames / Next will call the symbolizer function (if there are
+// C frames). So the approach here is, with GOMAXPROCS=2, have 2 goroutines
+// that use CallersFrames over and over, both frequently calling Gosched in an
+// attempt to get picked up by the other P.
+
+var tracebackOK bool
+
+func main() {
+	runtime.GOMAXPROCS(2)
+	runtime.SetCgoTraceback(0, unsafe.Pointer(C.tcTraceback), unsafe.Pointer(C.tcContext), unsafe.Pointer(C.tcSymbolizer))
+	C.C1()
+	if tracebackOK {
+		fmt.Println("OK")
+	}
+}
+
+//export G1
+func G1() {
+	C.C2()
+}
+
+//export G2
+func G2() {
+	pc := make([]uintptr, 32)
+	n := runtime.Callers(0, pc)
+
+	var wg sync.WaitGroup
+	for range 2 {
+		wg.Go(func() {
+			for range 1000 {
+				cf := runtime.CallersFrames(pc[:n])
+				var frames []runtime.Frame
+				for {
+					frame, more := cf.Next()
+					frames = append(frames, frame)
+					if !more {
+						break
+					}
+				}
+				runtime.Gosched()
+			}
+		})
+	}
+	wg.Wait()
+
+	tracebackOK = true
+}

src/cmd/cgo/internal/testsanitizers/testdata/tsan_tracebackctxt/tracebackctxt_c.c

@@ -0,0 +1,70 @@
+// Copyright 2025 The Go Authors. All rights reserved.
+// Use of this source code is governed by a BSD-style
+// license that can be found in the LICENSE file.
+
+// The C definitions for tracebackctxt.go. That file uses //export so
+// it can't put function definitions in the "C" import comment.
+
+#include <stdint.h>
+#include <stdio.h>
+
+// Functions exported from Go.
+extern void G1(void);
+extern void G2(void);
+
+void C1() {
+	G1();
+}
+
+void C2() {
+	G2();
+}
+
+struct cgoContextArg {
+	uintptr_t context;
+};
+
+struct cgoTracebackArg {
+	uintptr_t  context;
+	uintptr_t  sigContext;
+	uintptr_t* buf;
+	uintptr_t  max;
+};
+
+struct cgoSymbolizerArg {
+	uintptr_t   pc;
+	const char* file;
+	uintptr_t   lineno;
+	const char* func;
+	uintptr_t   entry;
+	uintptr_t   more;
+	uintptr_t   data;
+};
+
+void tcContext(void* parg) {
+	struct cgoContextArg* arg = (struct cgoContextArg*)(parg);
+	if (arg->context == 0) {
+		arg->context = 1;
+	}
+}
+
+void tcTraceback(void* parg) {
+	int base, i;
+	struct cgoTracebackArg* arg = (struct cgoTracebackArg*)(parg);
+	if (arg->max < 1) {
+		return;
+	}
+	arg->buf[0] = 6; // Chosen by fair dice roll.
+}
+
+void tcSymbolizer(void *parg) {
+	struct cgoSymbolizerArg* arg = (struct cgoSymbolizerArg*)(parg);
+	if (arg->pc == 0) {
+		return;
+	}
+	// Report two lines per PC returned by traceback, to test more handling.
+	arg->more = arg->file == NULL;
+	arg->file = "tracebackctxt.go";
+	arg->func = "cFunction";
+	arg->lineno = arg->pc + (arg->more << 16);
+}

src/cmd/cgo/internal/testsanitizers/tsan_test.go

@@ -56,6 +56,7 @@ func TestTSAN(t *testing.T) {
 		{src: "tsan13.go", needsRuntime: true},
 		{src: "tsan14.go", needsRuntime: true},
 		{src: "tsan15.go", needsRuntime: true},
+		{src: "tsan_tracebackctxt", needsRuntime: true}, // Subdirectory
 	}
 	for _, tc := range cases {
 		tc := tc
@@ -67,7 +68,7 @@ func TestTSAN(t *testing.T) {
 			defer dir.RemoveAll(t)
 
 			outPath := dir.Join(name)
-			mustRun(t, config.goCmd("build", "-o", outPath, srcPath(tc.src)))
+			mustRun(t, config.goCmd("build", "-o", outPath, "./"+srcPath(tc.src)))
 
 			cmdArgs := []string{outPath}
 			if goos == "linux" {

src/runtime/cgo.go

@@ -15,7 +15,9 @@ import "unsafe"
 //go:linkname _cgo_sys_thread_create _cgo_sys_thread_create
 //go:linkname _cgo_notify_runtime_init_done _cgo_notify_runtime_init_done
 //go:linkname _cgo_callers _cgo_callers
-//go:linkname _cgo_set_context_function _cgo_set_context_function
+//go:linkname _cgo_set_traceback_functions _cgo_set_traceback_functions
+//go:linkname _cgo_call_traceback_function _cgo_call_traceback_function
+//go:linkname _cgo_call_symbolizer_function _cgo_call_symbolizer_function
 //go:linkname _cgo_yield _cgo_yield
 //go:linkname _cgo_pthread_key_created _cgo_pthread_key_created
 //go:linkname _cgo_bindm _cgo_bindm
@@ -27,7 +29,9 @@ var (
 	_cgo_sys_thread_create        unsafe.Pointer
 	_cgo_notify_runtime_init_done unsafe.Pointer
 	_cgo_callers                  unsafe.Pointer
-	_cgo_set_context_function     unsafe.Pointer
+	_cgo_set_traceback_functions  unsafe.Pointer
+	_cgo_call_traceback_function  unsafe.Pointer
+	_cgo_call_symbolizer_function unsafe.Pointer
 	_cgo_yield                    unsafe.Pointer
 	_cgo_pthread_key_created      unsafe.Pointer
 	_cgo_bindm                    unsafe.Pointer

src/runtime/cgo/callbacks.go

@@ -121,13 +121,30 @@ var _cgo_bindm = &x_cgo_bindm
 var x_cgo_notify_runtime_init_done byte
 var _cgo_notify_runtime_init_done = &x_cgo_notify_runtime_init_done
 
-// Sets the traceback context function. See runtime.SetCgoTraceback.
-
-//go:cgo_import_static x_cgo_set_context_function
-//go:linkname x_cgo_set_context_function x_cgo_set_context_function
-//go:linkname _cgo_set_context_function _cgo_set_context_function
-var x_cgo_set_context_function byte
-var _cgo_set_context_function = &x_cgo_set_context_function
+// Sets the traceback, context, and symbolizer functions. See
+// runtime.SetCgoTraceback.
+
+//go:cgo_import_static x_cgo_set_traceback_functions
+//go:linkname x_cgo_set_traceback_functions x_cgo_set_traceback_functions
+//go:linkname _cgo_set_traceback_functions _cgo_set_traceback_functions
+var x_cgo_set_traceback_functions byte
+var _cgo_set_traceback_functions = &x_cgo_set_traceback_functions
+
+// Call the traceback function registered with x_cgo_set_traceback_functions.
+
+//go:cgo_import_static x_cgo_call_traceback_function
+//go:linkname x_cgo_call_traceback_function x_cgo_call_traceback_function
+//go:linkname _cgo_call_traceback_function _cgo_call_traceback_function
+var x_cgo_call_traceback_function byte
+var _cgo_call_traceback_function = &x_cgo_call_traceback_function
+
+// Call the symbolizer function registered with x_cgo_set_symbolizer_functions.
+
+//go:cgo_import_static x_cgo_call_symbolizer_function
+//go:linkname x_cgo_call_symbolizer_function x_cgo_call_symbolizer_function
+//go:linkname _cgo_call_symbolizer_function _cgo_call_symbolizer_function
+var x_cgo_call_symbolizer_function byte
+var _cgo_call_symbolizer_function = &x_cgo_call_symbolizer_function
 
 // Calls a libc function to execute background work injected via libc
 // interceptors, such as processing pending signals under the thread

src/runtime/cgo/gcc_context.c

@@ -8,11 +8,11 @@
 
 // Releases the cgo traceback context.
 void _cgo_release_context(uintptr_t ctxt) {
-	void (*pfn)(struct context_arg*);
+	void (*pfn)(struct cgoContextArg*);
 
 	pfn = _cgo_get_context_function();
 	if (ctxt != 0 && pfn != nil) {
-		struct context_arg arg;
+		struct cgoContextArg arg;
 
 		arg.Context = ctxt;
 		(*pfn)(&arg);

src/runtime/cgo/gcc_libinit.c

@@ -32,8 +32,14 @@ static void pthread_key_destructor(void* g);
 uintptr_t x_cgo_pthread_key_created;
 void (*x_crosscall2_ptr)(void (*fn)(void *), void *, int, size_t);
 
+// The traceback function, used when tracing C calls.
+static void (*cgo_traceback_function)(struct cgoTracebackArg*);
+
 // The context function, used when tracing back C calls into Go.
-static void (*cgo_context_function)(struct context_arg*);
+static void (*cgo_context_function)(struct cgoContextArg*);
+
+// The symbolizer function, used when symbolizing C frames.
+static void (*cgo_symbolizer_function)(struct cgoSymbolizerArg*);
 
 void
 x_cgo_sys_thread_create(void* (*func)(void*), void* arg) {
@@ -52,7 +58,7 @@ x_cgo_sys_thread_create(void* (*func)(void*), void* arg) {
 
 uintptr_t
 _cgo_wait_runtime_init_done(void) {
-	void (*pfn)(struct context_arg*);
+	void (*pfn)(struct cgoContextArg*);
 	int done;
 
 	pfn = __atomic_load_n(&cgo_context_function, __ATOMIC_CONSUME);
@@ -70,7 +76,6 @@ _cgo_wait_runtime_init_done(void) {
 			x_cgo_pthread_key_created = 1;
 		}
 
-
 		// TODO(iant): For the case of a new C thread calling into Go, such
 		// as when using -buildmode=c-archive, we know that Go runtime
 		// initialization is complete but we do not know that all Go init
@@ -87,7 +92,7 @@ _cgo_wait_runtime_init_done(void) {
 	}
 
 	if (pfn != nil) {
-		struct context_arg arg;
+		struct cgoContextArg arg;
 
 		arg.Context = 0;
 		(*pfn)(&arg);
@@ -138,17 +143,71 @@ x_cgo_notify_runtime_init_done(void* dummy __attribute__ ((unused))) {
 	pthread_mutex_unlock(&runtime_init_mu);
 }
 
-// Sets the context function to call to record the traceback context
-// when calling a Go function from C code. Called from runtime.SetCgoTraceback.
-void x_cgo_set_context_function(void (*context)(struct context_arg*)) {
-	__atomic_store_n(&cgo_context_function, context, __ATOMIC_RELEASE);
+// Sets the traceback, context, and symbolizer functions. Called from
+// runtime.SetCgoTraceback.
+void x_cgo_set_traceback_functions(struct cgoSetTracebackFunctionsArg* arg) {
+	__atomic_store_n(&cgo_traceback_function, arg->Traceback, __ATOMIC_RELEASE);
+	__atomic_store_n(&cgo_context_function, arg->Context, __ATOMIC_RELEASE);
+	__atomic_store_n(&cgo_symbolizer_function, arg->Symbolizer, __ATOMIC_RELEASE);
+}
+
+// Gets the traceback function to call to trace C calls.
+void (*(_cgo_get_traceback_function(void)))(struct cgoTracebackArg*) {
+	return __atomic_load_n(&cgo_traceback_function, __ATOMIC_CONSUME);
+}
+
+// Call the traceback function registered with x_cgo_set_traceback_functions.
+//
+// The traceback function is an arbitrary user C function which may be built
+// with TSAN, and thus must be wrapped with TSAN acquire/release calls. For
+// normal cgo calls, cmd/cgo automatically inserts TSAN acquire/release calls.
+// Since the traceback, context, and symbolizer functions are registered at
+// startup and called via the runtime, they do not get automatic TSAN
+// acquire/release calls.
+//
+// The only purpose of this wrapper is to perform TSAN acquire/release.
+// Alternatively, if the runtime arranged to safely call TSAN acquire/release,
+// it could perform the call directly.
+void x_cgo_call_traceback_function(struct cgoTracebackArg* arg) {
+	void (*pfn)(struct cgoTracebackArg*);
+
+	pfn = _cgo_get_traceback_function();
+	if (pfn == nil) {
+		return;
+	}
+
+	_cgo_tsan_acquire();
+	(*pfn)(arg);
+	_cgo_tsan_release();
 }
 
-// Gets the context function.
-void (*(_cgo_get_context_function(void)))(struct context_arg*) {
+// Gets the context function to call to record the traceback context
+// when calling a Go function from C code.
+void (*(_cgo_get_context_function(void)))(struct cgoContextArg*) {
 	return __atomic_load_n(&cgo_context_function, __ATOMIC_CONSUME);
 }
 
+// Gets the symbolizer function to call to symbolize C frames.
+void (*(_cgo_get_symbolizer_function(void)))(struct cgoSymbolizerArg*) {
+	return __atomic_load_n(&cgo_symbolizer_function, __ATOMIC_CONSUME);
+}
+
+// Call the symbolizer function registered with x_cgo_set_traceback_functions.
+//
+// See comment on x_cgo_call_traceback_function.
+void x_cgo_call_symbolizer_function(struct cgoSymbolizerArg* arg) {
+	void (*pfn)(struct cgoSymbolizerArg*);
+
+	pfn = _cgo_get_symbolizer_function();
+	if (pfn == nil) {
+		return;
+	}
+
+	_cgo_tsan_acquire();
+	(*pfn)(arg);
+	_cgo_tsan_release();
+}
+
 // _cgo_try_pthread_create retries pthread_create if it fails with
 // EAGAIN.
 int