-
Notifications
You must be signed in to change notification settings - Fork 273
Description
Summary
openai-go/v3 defines a struct that embeds reflect.Type and stores it in sync.Map (which takes any). This promotes ~20 interface methods onto the struct, and the compiler tags the promoted wrappers with REFLECTMETHOD + UsedInIface. On its own this is harmless — but when combined with any dependency whose init() calls reflect.Type.Method(int) (common in protobuf-generated code), the Go linker's dead-code elimination is defeated globally, inflating binaries by 30-40 MB.
Location
github.com/openai/openai-go/v3
internal/apijson/decoder.go
// line 22
var decoders sync.Map
// line 82-86
type decoderEntry struct {
reflect.Type // embedded interface
dateFormat string
root bool
}
// line 110-116 — stored in sync.Map as `any`
entry := decoderEntry{
Type: t,
dateFormat: d.dateFormat,
root: d.root,
}
fi, loaded := decoders.LoadOrStore(entry, decoderFunc(...))Mechanism
The Go linker's deadcode pass (cmd/link/internal/ld/deadcode.go) tracks two things:
-
ifaceMethodset — populated fromR_USEIFACEMETHODrelocations emitted by the compiler when a method is called on an interface value. -
REFLECTMETHODsymbols — methods on concrete types that are tagged by the compiler when the method could be discovered via reflection.
When a struct embeds an interface (like reflect.Type) and that struct is stored as any (via sync.Map.LoadOrStore), the compiler generates promoted method wrappers on the struct. These wrappers are tagged:
REFLECTMETHOD + UsedInIface
This creates ingredient #2 — dormant, waiting for a match.
Ingredient #1 comes from any protobuf .pb.go file. Every generated message type registers itself via init() → protoimpl.TypeBuilder.Build() → which calls reflect.Type.Method(int). The compiler emits:
R_USEIFACEMETHOD type:reflect.Type+112
(offset 112 = Method(int) reflect.Method in the reflect.Type interface)
When both ingredients are present:
linker deadcode pass:
ifaceMethod set ∋ {Method, symID} ← from protobuf init()
decoderEntry.Method is REFLECTMETHOD ← from embedded reflect.Type
→ match found
→ reflectSeen = true
→ keep ALL exported methods of ALL reachable types
→ +30-40 MB of dead code retained
Neither ingredient triggers this alone. It is an emergent interaction between two unrelated dependencies.
Minimal reproduction
// main.go
package main
import (
"reflect"
"sync"
)
// Mimics openai-go's decoderEntry — embedded reflect.Type
type decoderEntry struct {
reflect.Type
}
var cache sync.Map
func store(t reflect.Type) {
cache.LoadOrStore(decoderEntry{Type: t}, true)
}
func main() {
store(reflect.TypeOf(0))
}# Without any protobuf dependency:
$ go build -ldflags='-dumpdep' -o /dev/null . 2>&1 | grep -c ReflectMethod
0
# Add a blank import of any package containing .pb.go with init():
# import _ "github.com/blevesearch/bleve/v2/index/upsidedown"
# (or any protobuf-using package)
$ go build -ldflags='-dumpdep' -o /dev/null . 2>&1 | grep -c ReflectMethod
12
Actual measurements (picoclaw project, linux/amd64, stripped):
| Build | Binary size | ReflectMethod count |
|---|---|---|
| without protobuf dep | 17 MB | 0 |
| with protobuf dep | 49 MB | 12 |
Fix
Don't embed reflect.Type. Use a named field instead:
// before (broken)
type decoderEntry struct {
reflect.Type
dateFormat string
root bool
}
// after (fixed)
type decoderEntry struct {
typ reflect.Type // named field, no method promotion
dateFormat string
root bool
}Then update references (Type → typ):
entry := decoderEntry{
typ: t, // was: Type: t
dateFormat: d.dateFormat,
root: d.root,
}A named field does not promote methods onto the struct. Without promoted Method(int) wrappers, no REFLECTMETHOD tag is generated, and the linker's DCE is never defeated.
This is a zero-behavioral-change fix: the field is unexported, in an internal package, and only used as a sync.Map key for equality comparison. reflect.Type is comparable, so map key semantics are preserved.
Detection
For any Go project:
go build -ldflags='-dumpdep' -o /dev/null ./... 2>&1 | grep -c ReflectMethodIf >0, use verbose mode to find the matching pair:
go build -ldflags='-v=2 -dumpdep' -o /dev/null ./... 2>&1 | grep 'reached iface method:\|markable method:'Affected versions
All versions of openai-go that have the embedded reflect.Type in decoderEntry (checked: v3.5.0, v3.22.0). The bug only manifests when combined with a protobuf dependency in the same binary.