Skip to content

Conversation

@zJiaJun
Copy link
Contributor

@zJiaJun zJiaJun commented Apr 21, 2025

Fixes #13188

Problem:

The runtime instrumentation logic in tools/go-agent/instrument/runtime/instrument.go incorrectly checked for a hardcoded number of parameters (3) for the internal runtime.newproc1 function. This check failed for Go versions 1.23 and later, where the relevant function signature expects 2 parameters for the instrumentation purpose. Consequently, the agent skipped the injection of the necessary code for automatic context propagation when new goroutines were created using newproc1, breaking automatic tracing in Go 1.23+ environments.

Solution:

This patch addresses the issue by implementing dynamic parameter count checking based on the Go version specified during compilation via the -lang flag:

  1. Introduced Robust Version Checking: Added a new method CheckGoVersionGreaterOrEqual(major, minor int) bool to the api.CompileOptions struct in tools/go-agent/instrument/api/flags.go. This method reliably parses the -lang flag value (e.g., "go1.22", "go1.23", "go2.0") to accurately determine the Go version and compare it against required major and minor versions.
  2. Dynamic Parameter Check in Runtime Instrument: Modified the FilterAndEdit method within runtime.Instrument (tools/go-agent/instrument/runtime/instrument.go). It now utilizes the new opts.CheckGoVersionGreaterOrEqual(1, 23) method to determine the expectedParamCount for newproc1 (3 if Go version < 1.23, 2 if Go version >= 1.23). The instrumentation logic for newproc1 now only proceeds if the detected parameter count matches the expected count for the target Go version.

Impact:

With this fix, the skywalking-go agent can now correctly identify and instrument the runtime.newproc1 function on Go 1.23 and later versions, in addition to older versions. This ensures that the automatic context propagation mechanism functions as expected for goroutines created via the standard go keyword across all supported Go versions specified by the -lang flag during the build process.

Files Changed:

  • tools/go-agent/instrument/api/flags.go
  • tools/go-agent/instrument/runtime/instrument.go

@wu-sheng
Copy link
Member

@zJiaJun Please refer to e2e to add new go version and use some existing plugin tests.

@wu-sheng wu-sheng requested review from Copilot and mrproliu April 21, 2025 14:35
@wu-sheng wu-sheng added this to the 0.6.0 milestone Apr 21, 2025
@wu-sheng wu-sheng added the enhancement New feature or request label Apr 21, 2025
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This pull request fixes the instrumentation logic for the runtime.newproc1 function by dynamically checking the Go version to determine the correct number of expected parameters.

  • Updated the parameter count logic in the runtime instrumentation based on the Go version.
  • Introduced a new helper method, CheckGoVersionGreaterOrEqual, to verify the Go version from the -lang flag.
  • Updated the CHANGES.md file to document the fix.

Reviewed Changes

Copilot reviewed 3 out of 3 changed files in this pull request and generated 3 comments.

File Description
tools/go-agent/instrument/runtime/instrument.go Update of parameter count check for newproc1 instrumentation
tools/go-agent/instrument/api/flags.go Addition of CheckGoVersionGreaterOrEqual and version parsing logic
CHANGES.md Documentation update for the new instrumentation parameter counts

if len(n.Type.Params.List) != 3 {
expectedParamCount := 3
if r.opts.CheckGoVersionGreaterOrEqual(1, 23) {
expectedParamCount = 5
Copy link

Copilot AI Apr 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

According to the PR description, for Go 1.23 and later, runtime.newproc1 should expect 2 parameters rather than 5. Please update the logic to set the expected parameter count to 2 for Go 1.23+.

Suggested change
expectedParamCount = 5
expectedParamCount = 2

Copilot uses AI. Check for mistakes.
}
currentMinor := int(currentMinor64)

if currentMajor > requiredMinor {
Copy link

Copilot AI Apr 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The comparison should be with requiredMajor, not requiredMinor, to correctly determine if the current major version exceeds the required major version.

Suggested change
if currentMajor > requiredMinor {
if currentMajor > requiredMajor {

Copilot uses AI. Check for mistakes.
if currentMajor > requiredMinor {
return true
}
if currentMajor == requiredMajor && currentMinor >= requiredMajor {
Copy link

Copilot AI Apr 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The condition incorrectly compares currentMinor with requiredMajor; it should compare currentMinor with requiredMinor for an accurate version check.

Suggested change
if currentMajor == requiredMajor && currentMinor >= requiredMajor {
if currentMajor == requiredMajor && currentMinor >= requiredMinor {

Copilot uses AI. Check for mistakes.
@zJiaJun
Copy link
Contributor Author

zJiaJun commented Apr 21, 2025

@zJiaJun Please refer to e2e to add new go version and use some existing plugin tests.

Okay, I understand. I will update the .github/workflows/plugin-tests.yaml workflow to include Go 1.23/1.24 in the test matrix to ensure the fix is validated on the target versions.

@mrproliu
Copy link
Contributor

Hi @zJiaJun. Looks like the go agent don’t have the logical to verify the tracer on cross goroutine scenario. Could you help to add one?

@zJiaJun
Copy link
Contributor Author

zJiaJun commented Apr 21, 2025

Hi @zJiaJun. Looks like the go agent don’t have the logical to verify the tracer on cross goroutine scenario. Could you help to add one?

Hi @mrproliu, thanks for the feedback! That's a valid point. I agree that adding a specific test case to verify tracer propagation across goroutines is important for this fix.

I will work on adding a new test scenario for this. Let's discuss the specific implementation details if needed.

@wu-sheng
Copy link
Member

Please add ASF APLv2 header to fix CI.

@wu-sheng
Copy link
Member

irisv12 case seems to have some issues. Could you try to fix that? @zJiaJun

@zJiaJun
Copy link
Contributor Author

zJiaJun commented Apr 22, 2025

irisv12 case seems to have some issues. Could you try to fix that? @zJiaJun

Hi @wu-sheng
Following up on the irisv12 test failures:
I found that the iris v12.1.0 framework version specifically causes issues across different Go versions:
With Go 1.19, it fails due to dependencies requiring Go 1.21+ (missing slices package).
Even with Go 1.21, it fails during the Docker build phase at the go mod tidy step, suggesting deeper dependency resolution problems specific to v12.1.0.
The other tested framework versions (v12.2.0 and v12.2.5) appear more stable with Go 1.19 and Go 1.21 respectively in my local tests.
To resolve the CI failures caused by these specific dependency issues, I've removed v12.1.0 from the test matrix in plugin.yml for now.

@wu-sheng
Copy link
Member

@mrproliu Could you recheck this test failing? Any suggestion?

@mrproliu mrproliu merged commit b7f3404 into apache:main Apr 23, 2025
39 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] Runtime instrumentation check for newproc1 parameters incorrect for Go 1.23+

3 participants