Add OpenTelemetry tracing for individual Go tests #321
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Parse go test -json output during the test phase to create spans for each test. Test spans are children of the package span, showing test parallelism and duration in build traces. Co-authored-by: Ona <[email protected]>
CompositeReporter now delegates GetGoTestTracer to any wrapped reporter that implements TestTracingReporter, enabling test tracing when using multiple reporters. Co-authored-by: Ona <[email protected]>
Test tracing is opt-in since it can generate many spans for large test suites. Use --enable-test-tracing to create OpenTelemetry spans for individual Go tests. Co-authored-by: Ona <[email protected]>
WVerlaek
force-pushed
the
wv/go-test-tracing
branch
from
e8e3a3e to
ea21843
Compare
In non-verbose mode, only show package-level output and failed test output. Buffer test output and flush on failure to preserve error details while reducing noise for passing tests. Co-authored-by: Ona <[email protected]>
Simplify the design by detecting go test commands in the run() function rather than using a separate TestExecutor. This removes: - TestExecutor field from packageBuild - Special case in executeBuildPhase - GetGoTestTracer from Reporter interface - CompositeReporter delegation for test tracing The tracing is now an implementation detail of command execution. Co-authored-by: Ona <[email protected]>
kylos101
approved these changes
Jan 7, 2026
Contributor
kylos101
left a comment
kylos101
left a comment
There was a problem hiding this comment.
LGTM, I added some questions and nits, but nothing blocking. I didn't get a chance to test, but see you included success and failures, which is great. Also, i see in the results a test ran for 200ms (so much longer than others).
| key := spanKey(event.Package, event.Test) | ||
| if output, ok := testOutput[key]; ok { | ||
| for _, line := range output { | ||
| _, _ = outputWriter.Write([]byte(line)) |
Contributor
There was a problem hiding this comment.
Nit, not blocking:
We don't handle when outputWriter is nil for the fail scenario, but do above.
| type testSpanData struct { | ||
| span trace.Span | ||
| startTime time.Time | ||
| pkg string |
Contributor
There was a problem hiding this comment.
Question, not blocking:
How is the pkg field being used? I see it is being set, but not read anywhere.
|
|
||
| // goTestEvent represents a single event from `go test -json` output. | ||
| // See https://pkg.go.dev/cmd/test2json for the format specification. | ||
| type goTestEvent struct { |
Contributor
There was a problem hiding this comment.
Question, not blocking:
I see FailedBuild was omitted, intentional?
| _, _ = outputWriter.Write([]byte(event.Output)) | ||
| } else if event.Test == "" { | ||
| // Non-verbose: always show package-level output | ||
| _, _ = outputWriter.Write([]byte(event.Output)) |
Contributor
There was a problem hiding this comment.
Question, not blocking:
Did you intend to have this be event.Package, instead of Output?
Comment on lines
+133
to
+149
| func (t *GoTestTracer) handleEvent(event *goTestEvent) { | ||
| switch event.Action { | ||
| case "run": | ||
| t.handleRun(event) | ||
| case "pause": | ||
| t.handlePause(event) | ||
| case "cont": | ||
| t.handleCont(event) | ||
| case "pass", "fail", "skip": | ||
| t.handleEnd(event) | ||
| case "output": | ||
| // Output is already written to outputWriter | ||
| case "start": | ||
| // Package test started - we could create a package-level span here | ||
| t.handlePackageStart(event) | ||
| } | ||
| } |
Contributor
There was a problem hiding this comment.
Observation, not blocking:
We are missing bench, which I guess is generally only done locally
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Add per-test OpenTelemetry spans during Go package builds. When tracing is enabled, leeway parses
go test -jsonoutput and creates child spans for each test under the package span. This provides visibility into individual test duration and parallelism in build traces.Can be enabled by passing the
--enable-test-tracingflag, disabled by defaulthttps://ui.honeycomb.io/gitpod/environments/ci/datasets/leeway/result/FhLg2E7Cfp4/trace/8cAj6RnnZpf?fields%5B%5D=s_name&fields%5B%5D=s_serviceName&fields%5B%5D=c_leeway.package.name&fields%5B%5D=c_leeway.package.last_phase&span=14df73420d330fac
https://ui.honeycomb.io/gitpod/environments/ci/datasets/leeway/result/chTYCtKiNU8/trace/3gqNBXCwcwZ?fields%5B%5D=s_name&fields%5B%5D=s_serviceName&fields%5B%5D=c_leeway.package.name&fields%5B%5D=c_leeway.package.last_phase&span=2bc30322f9351793
Related Issue(s)
Fixes CORE-6451
How to test