jsonschema: memory leak in schema generator

[jbpratt]

Describe the bug
It seems there is a leak when trying to generate schemas from certain structs!

To Reproduce

package main

import (
        "fmt"
        "runtime"
        "time"

        "github.com/modelcontextprotocol/go-sdk/jsonschema"
        v1 "github.com/tektoncd/pipeline/pkg/apis/pipeline/v1"
)

func main() {
        ticker := time.NewTicker(5 * time.Second)
        go func() {
                for range ticker.C {
                        var m runtime.MemStats
                        runtime.ReadMemStats(&m)
                        fmt.Printf("Alloc: %d KB, Sys: %d KB, NumGC: %d\n",
                                m.Alloc/1024, m.Sys/1024, m.NumGC)
                }
        }()
        _, err := jsonschema.For[v1.Pipeline]()
        if err != nil {
                panic(err)
        }
}

Expected behavior
Generating the schema correctly!

Logs

❯ go run main.go
Alloc: 3126601 KB, Sys: 3275482 KB, NumGC: 13
Alloc: 6046591 KB, Sys: 6332210 KB, NumGC: 14
Alloc: 8992110 KB, Sys: 9524295 KB, NumGC: 15
Alloc: 12696764 KB, Sys: 13284884 KB, NumGC: 15
Alloc: 14358771 KB, Sys: 15591427 KB, NumGC: 16
signal: killed

Additional context
OS: Linux
Go version: 1.24.3
Deps:

github.com/modelcontextprotocol/go-sdk v0.0.0-20250630184440-2facfc6ffe0b
github.com/tektoncd/pipeline v1.1.0

[albertsundjaja]

hello!

I think the issue is because of a cycle in your type. The jsonschema.For have this comment (https://github.com/modelcontextprotocol/go-sdk/blob/main/jsonschema/infer.go#L39)

// The types must not have cycles.

It looks to me that v1.Pipeline have a cycle

type Pipeline struct {
        ...
	Spec PipelineSpec `json:"spec"`
}

type PipelineSpec struct {
	...
	Finally []PipelineTask `json:"finally,omitempty"`
}

type PipelineTask struct {
        ...
	PipelineSpec *PipelineSpec `json:"pipelineSpec,omitempty"`
}

I could be wrong ;)

[jba]

There is a cycle in the type, but For should terminate fairly quickly in a stack overflow panic.

@albertsundjaja, do you want to look at this further, given your work at #72? I'd start by making a test for it.

[albertsundjaja]

@jba sure, let me pick this one up

[albertsundjaja]

@jba @samthanawalla I found out the cause, so there are several factors that cause this

1. the Tekton Pipeline struct is massive
2. the heap memory allocation outgrow the stack growth
3. this is due to the recursive forType recursing every single type again. in my test, the recursion goes to a whooping count: 90072800 before I terminate it
4. hence the program crashes before it can throw stack overflow panic

I tried implementing a simple cache in jsonschema/infer.go

var cache = make(map[reflect.Type]*Schema)
func forType(t reflect.Type) (*Schema, error) {
        ...
	if s, ok := cache[t]; ok {
		return s, nil
	}
        ...
        cache[t] = s
        return s
}

I then run the sample program that @jbpratt provided. This time it produces the stack overflow error

go run ./test-sdk/main.go
runtime: goroutine stack exceeds 1000000000-byte limit
runtime: sp=0x1407e200340 stack=[0x1407e200000, 0x1409e200000]
fatal error: stack overflow

I think we should implement memoization in jsonschema.For

[jba]

@albertsundjaja Please proceed. Use sync.Map; it's perfect for this use case. You don't need a mutex because For always computes the same value for a given type. As an example, see https://go.googlesource.com/go/+/refs/heads/master/src/encoding/binary/binary.go#698.

[albertsundjaja]

@jba hmmm I think I get why the memoization was not implemented in the first place.

we are storing a cache map[reflect.Type]*Schema. If the caller mutate the Schema after calling For then it will also mutate the cached schema (i.e. different caller might get the mutated result!)

I tried:

1. deep copying during store/load (only fields touched by For)
2. immutable form of storage using gob so the map is map[reflect.Type][]byte (this need encode/decode during store/load)

But when trying the deeply nested Pipeline type, it still went OOM before it panic. So the memoization doesn't address this issue. We need a cycle detector to prevent the memory leak

I tried a quick performance test to see if there's any value in having a cache with the below structs

type SimpleStruct struct {
	Name    string                 `json:"name"`
	Age     int                    `json:"age"`
	Active  bool                   `json:"active"`
	Score   float64                `json:"score"`
	Tags    []string               `json:"tags"`
	Options map[string]interface{} `json:"options"`
}

type NestedStruct struct {
	ID       string                 `json:"id"`
	Metadata map[string]interface{} `json:"metadata"`
	Config   struct {
		Enabled bool `json:"enabled"`
		Timeout int  `json:"timeout"`
		Retries struct {
			Max     int `json:"max"`
			Backoff int `json:"backoff"`
		} `json:"retries"`
	} `json:"config"`
	Items []struct {
		Name        string `json:"name"`
		Description string `json:"description"`
		Properties  map[string]struct {
			Type  string `json:"type"`
			Value string `json:"value"`
		} `json:"properties"`
	} `json:"items"`
	Settings struct {
		Theme    string `json:"theme"`
		Language string `json:"language"`
		Features struct {
			Notifications bool `json:"notifications"`
			Analytics     bool `json:"analytics"`
			Backup        bool `json:"backup"`
		} `json:"features"`
	} `json:"settings"`
}

and the result

// Without memoization
1. Testing SimpleStruct:
Running 10000 iterations...
Performance: 22.360083ms for 10000 iterations
Average per iteration: 2.236µs

2. Testing NestedStruct:
Running 10000 iterations...
Performance: 79.656125ms for 10000 iterations
Average per iteration: 7.965µs

// Memoization with deep copy
1. Testing SimpleStruct:
Running 10000 iterations...
Performance: 14.800583ms for 10000 iterations
Average per iteration: 1.48µs

2. Testing NestedStruct:
Running 10000 iterations...
Performance: 50.574417ms for 10000 iterations
Average per iteration: 5.057µs

// With memoization using gob
1. Testing SimpleStruct:
Running 10000 iterations...
Performance: 515.872416ms for 10000 iterations
Average per iteration: 51.587µs

2. Testing NestedStruct:
Running 10000 iterations...
Performance: 646.161875ms for 10000 iterations
Average per iteration: 64.616µs

So there's value in implementing the memoization with deep copy, but I think we also need a cycle detector for huge struct like the Tekton Pipeline (tbh this is the biggest struct I have ever seen to date)