Skip to content

Conversation

@cscheid
Copy link
Collaborator

@cscheid cscheid commented Jan 31, 2025

normalize-3 was among our slowest filters on main. This PR rewrites the post-processing of FloatRefTargets so that the work done in that filter happens inside normalize-2. Our A/B tester shows statistically significant improvement:

{'outcome': 'significant', 'winner': 'B', 'p_value': np.float64(5.343099192664396e-05), 'total_samples': 20, 'alpha_spent': 0.05, 'statistics': {'counts': {'A': 10, 'B': 10}, 'means': {'A': 0.7966938972473144, 'B': 0.7741650581359864}, 'variances': {'A': 7.135757199649984e-05, 'B': 9.23875652483197e-05}}}
Statistical Summary:
===================
Variant     Count         Mean      Std Dev
-------- -------- ------------ ------------
A              10     0.796694     0.008447
B              10     0.774165     0.009612

@cscheid cscheid merged commit 527c335 into main Jan 31, 2025
47 checks passed
@cscheid cscheid deleted the perf/2025-01-31-combine-ast branch January 31, 2025 19:43
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant