Skip to content

Commit f5f69a3

Browse files
dsnetgopherbot
authored andcommitted
encoding/json/jsontext: avoid pinning application data in pools
Previously, we put a jsontext.Encoder (or Decoder) back into a pool with minimal reset logic. This was semantically safe since we always did a full reset after obtaining an Encoder/Decoder back out of the pool. However, this meant that so long as an Encoder/Decoder was alive in the pool, any application data referenced by the coder would be kept alive longer than necessary. Explicitly, clear such fields so that application data can be more aggressively garbage collected. Performance: name old time/op new time/op delta Unmarshal/Bool-32 52.0ns ± 3% 50.3ns ± 3% -3.30% (p=0.001 n=10+10) Marshal/Bool-32 55.4ns ± 3% 54.4ns ± 2% -1.75% (p=0.006 n=10+9) This only impacts the performance of discrete Marshal/Unmarshal calls. For the simplest possible call (i.e., to marsha/unmarshal a bool), there is a 1-2ns slow down. This is an appropriate slowdown for an improvement in memory utilization. Change-Id: I5e7d7827473773e53a9dcb3d7fe9052a75481e9f Reviewed-on: https://go-review.googlesource.com/c/go/+/713640 Reviewed-by: Damien Neil <[email protected]> Reviewed-by: Johan Brandhorst-Satzkorn <[email protected]> TryBot-Bypass: Damien Neil <[email protected]> Reviewed-by: Dmitri Shuralyov <[email protected]> Reviewed-by: Daniel Martí <[email protected]> Auto-Submit: Damien Neil <[email protected]>
1 parent a6a59f0 commit f5f69a3

File tree

2 files changed

+13
-0
lines changed

2 files changed

+13
-0
lines changed

src/encoding/json/jsontext/pools.go

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -54,6 +54,10 @@ func getBufferedEncoder(opts ...Options) *Encoder {
5454
return e
5555
}
5656
func putBufferedEncoder(e *Encoder) {
57+
if cap(e.s.availBuffer) > 64<<10 {
58+
e.s.availBuffer = nil // avoid pinning arbitrarily large amounts of memory
59+
}
60+
5761
// Recycle large buffers only if sufficiently utilized.
5862
// If a buffer is under-utilized enough times sequentially,
5963
// then it is discarded, ensuring that a single large buffer
@@ -95,9 +99,14 @@ func getStreamingEncoder(w io.Writer, opts ...Options) *Encoder {
9599
}
96100
}
97101
func putStreamingEncoder(e *Encoder) {
102+
if cap(e.s.availBuffer) > 64<<10 {
103+
e.s.availBuffer = nil // avoid pinning arbitrarily large amounts of memory
104+
}
98105
if _, ok := e.s.wr.(*bytes.Buffer); ok {
106+
e.s.wr, e.s.Buf = nil, nil // avoid pinning the provided bytes.Buffer
99107
bytesBufferEncoderPool.Put(e)
100108
} else {
109+
e.s.wr = nil // avoid pinning the provided io.Writer
101110
if cap(e.s.Buf) > 64<<10 {
102111
e.s.Buf = nil // avoid pinning arbitrarily large amounts of memory
103112
}
@@ -126,6 +135,7 @@ func getBufferedDecoder(b []byte, opts ...Options) *Decoder {
126135
return d
127136
}
128137
func putBufferedDecoder(d *Decoder) {
138+
d.s.buf = nil // avoid pinning the provided buffer
129139
bufferedDecoderPool.Put(d)
130140
}
131141

@@ -142,8 +152,10 @@ func getStreamingDecoder(r io.Reader, opts ...Options) *Decoder {
142152
}
143153
func putStreamingDecoder(d *Decoder) {
144154
if _, ok := d.s.rd.(*bytes.Buffer); ok {
155+
d.s.rd, d.s.buf = nil, nil // avoid pinning the provided bytes.Buffer
145156
bytesBufferDecoderPool.Put(d)
146157
} else {
158+
d.s.rd = nil // avoid pinning the provided io.Reader
147159
if cap(d.s.buf) > 64<<10 {
148160
d.s.buf = nil // avoid pinning arbitrarily large amounts of memory
149161
}

src/encoding/json/v2/arshal.go

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -571,5 +571,6 @@ func putStrings(s *stringSlice) {
571571
if cap(*s) > 1<<10 {
572572
*s = nil // avoid pinning arbitrarily large amounts of memory
573573
}
574+
clear(*s) // avoid pinning a reference to each string
574575
stringsPools.Put(s)
575576
}

0 commit comments

Comments
 (0)