Measure impact of exposing StructMeta

[ofek]

This PR attempts to measure the performance impact of #890 as discussed in #894 (comment). I just introduced a performance profiling test in #923 which we will use to compare.

• The stats for each benchmark are computed from 10,000 rounds, with 10 runs in each round. There are also 50 warm-up rounds.
• The profiling test job was retried multiple times for each referenced commit.
  • The summary title of each job's expandable section refers to the median timings.
• All timings are in microseconds.
• Legend:
  • Outliers: 1 Standard Deviation from Mean; 1.5 IQR (InterQuartile Range) from 1st Quartile and 3rd Quartile.
  • OPS: Operations Per Second, computed as 1 / Mean

---

The current stats from most recent commit (0191e29) on main are as follows:

encode: 207.0729 / decode: 472.8765

https://github.com/jcrist/msgspec/actions/runs/19278707083/job/55124831438

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	201.1243 (1.0)	367.4259 (1.0)	207.9662 (1.0)	8.8061 (1.0)	207.0729 (1.0)	5.2049 (1.44)	217;186	4.8085 (1.0)	10000	10
decode	466.6362 (2.32)	842.8725 (2.29)	474.6967 (2.28)	14.2482 (1.62)	472.8765 (2.28)	3.6026 (1.0)	163;428	2.1066 (0.44)	10000	10

encode: 208.4342 / decode: 473.5738

https://github.com/jcrist/msgspec/actions/runs/19278707083/job/55151162634

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	203.7415 (1.0)	464.0016 (1.0)	209.6607 (1.0)	8.7269 (1.0)	208.4342 (1.0)	2.5723 (1.0)	179;548	4.7696 (1.0)	10000	10
decode	467.9630 (2.30)	962.4667 (2.07)	475.4964 (2.27)	16.5374 (1.89)	473.5738 (2.27)	2.9682 (1.15)	128;633	2.1031 (0.44)	10000	10

encode: 203.9945 / decode: 475.7701

https://github.com/jcrist/msgspec/actions/runs/19278707083/job/55213730802

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	200.3558 (1.0)	356.4271 (1.0)	204.9474 (1.0)	8.4551 (1.0)	203.9945 (1.0)	1.7164 (1.0)	144;609	4.8793 (1.0)	10000	10
decode	469.4412 (2.34)	906.2031 (2.54)	476.8766 (2.33)	9.8599 (1.17)	475.7701 (2.33)	3.0983 (1.81)	174;555	2.0970 (0.43)	10000	10

---

The first commit of this PR (39679f2 (#927)) reverts the relevant code changes to the state before StructMeta was exposed (7acceff). I manually removed the import in src/msgspec/__init__.py and then ran:

git show 7acceff82601a18f0bd5284da27eaf8279f753c8:msgspec/_core.c > src/msgspec/_core.c
git show 7acceff82601a18f0bd5284da27eaf8279f753c8:msgspec/__init__.pyi > src/msgspec/__init__.pyi

encode: 201.8113 / decode: 470.4537

https://github.com/jcrist/msgspec/actions/runs/19279043046/job/55125796315

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	198.8778 (1.0)	369.6624 (1.0)	202.8351 (1.0)	6.8731 (1.0)	201.8113 (1.0)	1.3997 (1.0)	230;734	4.9301 (1.0)	10000	10
decode	464.6508 (2.34)	933.8461 (2.53)	471.9030 (2.33)	11.5599 (1.68)	470.4537 (2.33)	2.9133 (2.08)	170;653	2.1191 (0.43)	10000	10

encode: 201.3001 / decode: 470.7170

https://github.com/jcrist/msgspec/actions/runs/19279043046/job/55150123158

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	197.8401 (1.0)	528.8221 (1.0)	202.3317 (1.0)	9.2803 (1.0)	201.3001 (1.0)	1.9451 (1.0)	119;524	4.9424 (1.0)	10000	10
decode	464.6451 (2.35)	902.2993 (1.71)	471.8639 (2.33)	11.1040 (1.20)	470.7170 (2.34)	3.2227 (1.66)	179;551	2.1193 (0.43)	10000	10

encode: 202.9358 / decode: 471.8717

https://github.com/jcrist/msgspec/actions/runs/19279043046/job/55150356878

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	199.5605 (1.0)	354.7804 (1.0)	203.8711 (1.0)	7.1735 (1.0)	202.9358 (1.0)	1.8154 (1.0)	170;569	4.9051 (1.0)	10000	10
decode	465.3009 (2.33)	754.9604 (2.13)	472.9567 (2.32)	7.2304 (1.01)	471.8717 (2.33)	3.3296 (1.83)	346;500	2.1144 (0.43)	10000	10

---

The second commit of this PR (12ca746 (#927)) adds back the StructMeta changes but leaves the older conditional statements.

encode: 202.0596 / decode: 465.4165

https://github.com/jcrist/msgspec/actions/runs/19279328219/job/55126685658

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	198.6341 (1.0)	349.1181 (1.0)	202.7256 (1.0)	3.9781 (1.0)	202.0596 (1.0)	1.7832 (1.0)	432;621	4.9328 (1.0)	10000	10
decode	459.5138 (2.31)	848.7020 (2.43)	466.6248 (2.30)	8.8354 (2.22)	465.4165 (2.30)	3.2405 (1.82)	234;530	2.1430 (0.43)	10000	10

encode: 202.1670 / decode: 467.7030

https://github.com/jcrist/msgspec/actions/runs/19279328219/job/55220713879

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	199.4497 (1.0)	356.6277 (1.0)	203.2230 (1.0)	8.1591 (1.0)	202.1670 (1.0)	1.0973 (1.0)	129;843	4.9207 (1.0)	10000	10
decode	461.3140 (2.31)	818.2023 (2.29)	469.0633 (2.31)	10.5852 (1.30)	467.7030 (2.31)	3.0546 (2.78)	194;614	2.1319 (0.43)	10000	10

encode: 200.9772 / decode: 469.2514

https://github.com/jcrist/msgspec/actions/runs/19279328219/job/55221019147

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	197.5292 (1.0)	363.9527 (1.0)	201.6201 (1.0)	4.0040 (1.0)	200.9772 (1.0)	1.6488 (1.0)	457;712	4.9598 (1.0)	10000	10
decode	461.9943 (2.34)	955.5254 (2.63)	470.8668 (2.34)	15.9028 (3.97)	469.2514 (2.33)	3.1925 (1.94)	127;479	2.1237 (0.43)	10000	10

---

The third commit of this PR (b0034bf (#927)) only adds back the Python import of StructMeta in src/msgspec/__init__.py, in case that for some reason makes a difference.

encode: 203.1143 / decode: 471.9352

https://github.com/jcrist/msgspec/actions/runs/19279428342/job/55127014303

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	200.2113 (1.0)	339.6793 (1.0)	203.9622 (1.0)	6.7081 (1.0)	203.1143 (1.0)	1.3481 (1.0)	143;614	4.9029 (1.0)	10000	10
decode	466.8242 (2.33)	889.5582 (2.62)	473.0216 (2.32)	8.4826 (1.26)	471.9352 (2.32)	2.5813 (1.91)	212;695	2.1141 (0.43)	10000	10

encode: 205.6086 / decode: 469.1426

https://github.com/jcrist/msgspec/actions/runs/19279428342/job/55219914604

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	202.1548 (1.0)	362.9893 (1.0)	206.5176 (1.0)	6.9384 (1.13)	205.6086 (1.0)	1.9924 (1.0)	178;522	4.8422 (1.0)	10000	10
decode	462.6665 (2.29)	730.6224 (2.01)	469.9820 (2.28)	6.1526 (1.0)	469.1426 (2.28)	3.2699 (1.64)	469;492	2.1277 (0.44)	10000	10

encode: 203.7604 / decode: 472.6652

https://github.com/jcrist/msgspec/actions/runs/19279428342/job/55220536395

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	200.6007 (1.0)	358.5063 (1.0)	204.8942 (1.0)	8.1759 (1.0)	203.7604 (1.0)	2.0921 (1.0)	163;553	4.8806 (1.0)	10000	10
decode	466.5682 (2.33)	820.0344 (2.29)	473.9527 (2.31)	10.8879 (1.33)	472.6652 (2.32)	3.2736 (1.56)	188;605	2.1099 (0.43)	10000	10

---

The fourth commit of this PR (90cfce6 (#927)) restored the final 2 code files to the way they are on main. Then, in src/msgspec/_core.c I tried refactoring the conditional logic a bit to be more maintainable/DRY.

encode: 204.3880 / decode: 487.9419

https://github.com/jcrist/msgspec/actions/runs/19280475608/job/55130310717

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	201.2467 (1.0)	359.8429 (1.0)	205.6013 (1.0)	9.8727 (1.0)	204.3880 (1.0)	1.6625 (1.0)	139;639	4.8638 (1.0)	10000	10
decode	482.1533 (2.40)	921.6361 (2.56)	489.7668 (2.38)	12.7218 (1.29)	487.9419 (2.39)	3.6917 (2.22)	199;595	2.0418 (0.42)	10000	10

encode: 204.3767 / decode: 479.7496

https://github.com/jcrist/msgspec/actions/runs/19280475608/job/55130695429

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	201.9173 (1.0)	355.1093 (1.0)	205.3439 (1.0)	7.3993 (1.0)	204.3767 (1.0)	1.7447 (1.0)	182;639	4.8699 (1.0)	10000	10
decode	474.3272 (2.35)	838.1676 (2.36)	480.9436 (2.34)	8.7424 (1.18)	479.7496 (2.35)	3.1857 (1.83)	253;629	2.0792 (0.43)	10000	10

encode: 188.4990 / decode: 441.3936

https://github.com/jcrist/msgspec/actions/runs/19280475608/job/55219760315

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	186.4530 (1.0)	312.4932 (1.0)	188.7505 (1.0)	2.6518 (1.0)	188.4990 (1.0)	0.8344 (1.0)	339;549	5.2980 (1.0)	10000	10
decode	437.3303 (2.35)	854.9992 (2.74)	442.5566 (2.34)	9.4223 (3.55)	441.3936 (2.34)	2.2717 (2.72)	116;738	2.2596 (0.43)	10000	10

encode: 200.0785 / decode: 478.5824

https://github.com/jcrist/msgspec/actions/runs/19280475608/job/55221584281

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	196.6335 (1.0)	386.5246 (1.0)	201.3585 (1.0)	8.9834 (1.07)	200.0785 (1.0)	2.3201 (1.0)	131;537	4.9663 (1.0)	10000	10
decode	472.6448 (2.40)	878.2723 (2.27)	479.7974 (2.38)	8.4194 (1.0)	478.5824 (2.39)	3.0813 (1.33)	300;649	2.0842 (0.42)	10000	10

encode: 202.2184 / decode: 481.9860

https://github.com/jcrist/msgspec/actions/runs/19280475608/job/55222104467

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	198.6067 (1.0)	350.5100 (1.0)	202.7252 (1.0)	3.4900 (1.0)	202.2184 (1.0)	1.8860 (1.0)	472;483	4.9328 (1.0)	10000	10
decode	476.2773 (2.40)	832.8137 (2.38)	483.1080 (2.38)	8.0567 (2.31)	481.9860 (2.38)	2.9876 (1.58)	243;535	2.0699 (0.42)	10000	10

---

The fifth commit of this PR (2e79e6e (#927)) improves the previous refactor to be faster.

encode: 204.4297 / decode: 473.7804

https://github.com/jcrist/msgspec/actions/runs/19284323563/job/55141735437

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	201.8770 (1.0)	349.1732 (1.0)	205.3730 (1.0)	6.6212 (1.0)	204.4297 (1.0)	2.0717 (1.0)	206;522	4.8692 (1.0)	10000	10
decode	468.0291 (2.32)	825.7441 (2.36)	475.0507 (2.31)	10.6493 (1.61)	473.7804 (2.32)	3.2670 (1.58)	182;542	2.1050 (0.43)	10000	10

encode: 206.1753 / decode: 475.5152

https://github.com/jcrist/msgspec/actions/runs/19284323563/job/55142261913

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	204.0911 (1.0)	351.3634 (1.0)	207.2974 (1.0)	6.9403 (1.0)	206.1753 (1.0)	1.7064 (1.0)	244;771	4.8240 (1.0)	10000	10
decode	469.2734 (2.30)	891.3440 (2.54)	477.4993 (2.30)	16.5942 (2.39)	475.5152 (2.31)	3.1734 (1.86)	174;613	2.0942 (0.43)	10000	10

encode: 206.0142 / decode: 472.8009

https://github.com/jcrist/msgspec/actions/runs/19284323563/job/55142778982

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	202.0598 (1.0)	350.2796 (1.0)	206.6133 (1.0)	5.2972 (1.0)	206.0142 (1.0)	2.1084 (1.0)	308;534	4.8400 (1.0)	10000	10
decode	465.6591 (2.30)	860.5129 (2.46)	473.8062 (2.29)	9.4256 (1.78)	472.8009 (2.29)	3.6421 (1.73)	182;414	2.1106 (0.44)	10000	10

encode: 192.2017 / decode: 442.4200

https://github.com/jcrist/msgspec/actions/runs/19284323563/job/55221462586

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	188.8096 (1.0)	353.0260 (1.0)	192.6355 (1.0)	4.1865 (1.0)	192.2017 (1.0)	1.9293 (1.0)	281;362	5.1912 (1.0)	10000	10
decode	438.9683 (2.32)	1,469.8088 (4.16)	444.1291 (2.31)	14.7933 (3.53)	442.4200 (2.30)	2.7108 (1.41)	93;1163	2.2516 (0.43)	10000	10

encode: 204.8113 / decode: 474.8899

https://github.com/jcrist/msgspec/actions/runs/19284323563/job/55221890208

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	202.7853 (1.0)	578.1251 (1.0)	205.8727 (1.0)	9.1838 (1.0)	204.8113 (1.0)	1.7907 (1.0)	117;511	4.8574 (1.0)	10000	10
decode	467.1764 (2.30)	1,092.5360 (1.89)	475.9442 (2.31)	12.8802 (1.40)	474.8899 (2.32)	3.0806 (1.72)	122;446	2.1011 (0.43)	10000	10

---

Commits de51303 (#927) & 21da11e (#927) introduce a final implementation of the helpers that optimizes in the case of subclasses.

encode: 189.0633 / decode: 444.3397

https://github.com/jcrist/msgspec/actions/runs/19320644266/job/55261304734

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	187.2177 (1.0)	335.8345 (1.0)	189.6079 (1.0)	5.1489 (1.0)	189.0633 (1.0)	0.8365 (1.0)	160;680	5.2740 (1.0)	10000	10
decode	441.0673 (2.36)	866.4667 (2.58)	445.8347 (2.35)	11.5915 (2.25)	444.3397 (2.35)	2.1955 (2.62)	107;1405	2.2430 (0.43)	10000	10

encode: 202.6852 / decode: 472.9807

https://github.com/jcrist/msgspec/actions/runs/19320644266/job/55261459898

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	198.4325 (1.0)	353.4426 (1.0)	203.5165 (1.0)	6.3106 (1.0)	202.6852 (1.0)	2.0679 (1.0)	286;610	4.9136 (1.0)	10000	10
decode	466.2792 (2.35)	798.7269 (2.26)	474.2315 (2.33)	9.3709 (1.48)	472.9807 (2.33)	3.5315 (1.71)	246;527	2.1087 (0.43)	10000	10

encode: 203.8291 / decode: 470.4672

https://github.com/jcrist/msgspec/actions/runs/19320644266/job/55261700792

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	200.1261 (1.0)	354.6502 (1.0)	204.7488 (1.0)	6.5646 (1.0)	203.8291 (1.0)	2.0593 (1.0)	186;576	4.8840 (1.0)	10000	10
decode	463.3951 (2.32)	1,195.7785 (3.37)	471.8159 (2.30)	16.2059 (2.47)	470.4672 (2.31)	3.6633 (1.78)	117;404	2.1195 (0.43)	10000	10

encode: 202.9539 / decode: 472.7815

https://github.com/jcrist/msgspec/actions/runs/19320644266/job/55261836412

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	198.3519 (1.0)	438.4243 (1.0)	204.4805 (1.0)	11.8608 (1.0)	202.9539 (1.0)	2.4232 (1.0)	137;517	4.8904 (1.0)	10000	10
decode	465.4753 (2.35)	1,163.0335 (2.65)	474.5200 (2.32)	17.4162 (1.47)	472.7815 (2.33)	3.5426 (1.46)	144;531	2.1074 (0.43)	10000	10

encode: 204.2903 / decode: 476.9618

https://github.com/jcrist/msgspec/actions/runs/19320644266/job/55262300900

Name (time in us)	Min	Max	Mean	StdDev	Median	IQR	Outliers	OPS (Kops/s)	Rounds	Iterations
encode	201.3492 (1.0)	327.2208 (1.0)	205.2371 (1.0)	4.5759 (1.0)	204.2903 (1.0)	2.0764 (1.0)	659;837	4.8724 (1.0)	10000	10
decode	468.2807 (2.33)	873.8386 (2.67)	479.3891 (2.34)	10.2809 (2.25)	476.9618 (2.33)	7.3895 (3.56)	928;407	2.0860 (0.43)	10000	10