Skip to content

bug: embedded reflect.Type breaks Go linker dead-code elimination #609

@blib

Description

@blib

Summary

openai-go/v3 defines a struct that embeds reflect.Type and stores it in sync.Map (which takes any). This promotes ~20 interface methods onto the struct, and the compiler tags the promoted wrappers with REFLECTMETHOD + UsedInIface. On its own this is harmless — but when combined with any dependency whose init() calls reflect.Type.Method(int) (common in protobuf-generated code), the Go linker's dead-code elimination is defeated globally, inflating binaries by 30-40 MB.

Location

github.com/openai/openai-go/v3
  internal/apijson/decoder.go
// line 22
var decoders sync.Map

// line 82-86
type decoderEntry struct {
	reflect.Type        // embedded interface
	dateFormat string
	root       bool
}

// line 110-116 — stored in sync.Map as `any`
entry := decoderEntry{
	Type:       t,
	dateFormat: d.dateFormat,
	root:       d.root,
}
fi, loaded := decoders.LoadOrStore(entry, decoderFunc(...))

Mechanism

The Go linker's deadcode pass (cmd/link/internal/ld/deadcode.go) tracks two things:

  1. ifaceMethod set — populated from R_USEIFACEMETHOD relocations emitted by the compiler when a method is called on an interface value.

  2. REFLECTMETHOD symbols — methods on concrete types that are tagged by the compiler when the method could be discovered via reflection.

When a struct embeds an interface (like reflect.Type) and that struct is stored as any (via sync.Map.LoadOrStore), the compiler generates promoted method wrappers on the struct. These wrappers are tagged:

REFLECTMETHOD + UsedInIface

This creates ingredient #2 — dormant, waiting for a match.

Ingredient #1 comes from any protobuf .pb.go file. Every generated message type registers itself via init()protoimpl.TypeBuilder.Build() → which calls reflect.Type.Method(int). The compiler emits:

R_USEIFACEMETHOD type:reflect.Type+112

(offset 112 = Method(int) reflect.Method in the reflect.Type interface)

When both ingredients are present:

linker deadcode pass:
  ifaceMethod set ∋ {Method, symID}          ← from protobuf init()
  decoderEntry.Method is REFLECTMETHOD       ← from embedded reflect.Type
  → match found
  → reflectSeen = true
  → keep ALL exported methods of ALL reachable types
  → +30-40 MB of dead code retained

Neither ingredient triggers this alone. It is an emergent interaction between two unrelated dependencies.

Minimal reproduction

// main.go
package main

import (
	"reflect"
	"sync"
)

// Mimics openai-go's decoderEntry — embedded reflect.Type
type decoderEntry struct {
	reflect.Type
}

var cache sync.Map

func store(t reflect.Type) {
	cache.LoadOrStore(decoderEntry{Type: t}, true)
}

func main() {
	store(reflect.TypeOf(0))
}
# Without any protobuf dependency:
$ go build -ldflags='-dumpdep' -o /dev/null . 2>&1 | grep -c ReflectMethod
0

# Add a blank import of any package containing .pb.go with init():
#   import _ "github.com/blevesearch/bleve/v2/index/upsidedown"
# (or any protobuf-using package)
$ go build -ldflags='-dumpdep' -o /dev/null . 2>&1 | grep -c ReflectMethod
12

Actual measurements (picoclaw project, linux/amd64, stripped):

Build Binary size ReflectMethod count
without protobuf dep 17 MB 0
with protobuf dep 49 MB 12

Fix

Don't embed reflect.Type. Use a named field instead:

// before (broken)
type decoderEntry struct {
	reflect.Type
	dateFormat string
	root       bool
}

// after (fixed)
type decoderEntry struct {
	typ        reflect.Type  // named field, no method promotion
	dateFormat string
	root       bool
}

Then update references (Typetyp):

entry := decoderEntry{
	typ:        t,         // was: Type: t
	dateFormat: d.dateFormat,
	root:       d.root,
}

A named field does not promote methods onto the struct. Without promoted Method(int) wrappers, no REFLECTMETHOD tag is generated, and the linker's DCE is never defeated.

This is a zero-behavioral-change fix: the field is unexported, in an internal package, and only used as a sync.Map key for equality comparison. reflect.Type is comparable, so map key semantics are preserved.

Detection

For any Go project:

go build -ldflags='-dumpdep' -o /dev/null ./... 2>&1 | grep -c ReflectMethod

If >0, use verbose mode to find the matching pair:

go build -ldflags='-v=2 -dumpdep' -o /dev/null ./... 2>&1 | grep 'reached iface method:\|markable method:'

Affected versions

All versions of openai-go that have the embedded reflect.Type in decoderEntry (checked: v3.5.0, v3.22.0). The bug only manifests when combined with a protobuf dependency in the same binary.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions