Skip to content

Releases: microprediction/precise

v1.1.0

Choose a tag to compare

@microprediction microprediction released this 13 Sep 04:12
79903eb

Added

  • NonlinearShrinkageCovariance: online analytical nonlinear shrinkage of the covariance
    spectrum (Ledoit & Wolf 2020). The zoo's linear shrinkers pull every eigenvalue towards a common
    target; this moves each one separately while keeping the sample eigenvectors, and stays finite and
    invertible when p > n. Accumulation is a plain Welford update (O(p^2)/step, no window); the
    spectral work is lazy, in _state_to_cov, and memoized per state.
  • WindowedNonlinearShrinkageCovariance: the same map over a rolling window of the last W
    observations. A window is the forgetting variant that keeps the asymptotics exact — equal
    weights inside the window are precisely the sample they describe, with n = W — where an
    exponential decay would need the weighted theory rather than a moment-matched effective sample
    size. Bounded state, O(p^2) add-and-drop per step. research/forgetting.py scores the two
    against each other; research/turnover.py scores what squared error cannot see — the window's
    hard boundary echoes every shock one window later, which costs churn that grows with tail
    weight.
  • EwaNonlinearShrinkageCovariance: the same map over an exponentially weighted sample, at
    n_eff = (2-r)/r. Statistically the approximate one — matching one moment of the weight
    distribution is not an equivalence, and Oriol (arXiv:2410.14420) derives the weighted formulas
    properly — but it has no window boundary, so it does not pay for a shock twice.
    research/turnover.py measures the difference.
  • A JOSS paper under papers/joss/, and the note Spectral Calibration Without a Split under
    papers/online_spectral_calibration/.

Fixed

  • LedoitWolfCovariance collapsed to a scaled identity under heavy tails. pi_bar averages a
    quantity growing like the fourth power of the observation, so one fat-tailed draw could pin the
    shrinkage intensity at 1 and leave the estimate with no off-diagonals: at t(3) innovations it
    retained 0.088 of the covariance structure where OAS retained 0.543. Each observation's
    contribution is now winsorized at ten times the running mean, which leaves Gaussian behaviour
    unchanged to three decimals and takes t(3) retention to 0.409.
  • The frozen recommender was inert. sklearn's tree_.value holds class proportions, and the
    exporter cast them with int(), flooring every value under 1.0 to zero — 46 of 47 nodes carried
    no weight, so suggest() had been ranking on the heuristic ruleset alone. The model now covers
    19 of 20 estimators (it was 9), and two training runs produce a byte-identical artifact.
  • Training was irreproducible whenever randomcov was installed: three generative ensembles in
    research/oos.py ignored the rng they were passed and drew from global state.
  • The training grid stopped at n/p = 3, so every data-rich low-dimensional problem was
    extrapolation. It now spans p from 5 to 60 and n/p from 0.5 to 25.
  • A malformed generated model can no longer break import precise.

Changed

  • suggest()'s safe default moves from LedoitWolfCovariance to NonlinearShrinkageCovariance,
    which has the best mean rank of any single fixed choice over eleven ensembles and seven (p, n)
    regimes. It is not uniformly best: at p close to n it ranks 9.41 and the trained model is
    worth far more there, which is why the model still leads and this only breaks ties.

precise 1.0.0 — online covariance/correlation

Choose a tag to compare

@microprediction microprediction released this 05 Jun 03:22
3cb57c3

First stable release of the rewritten precise: a focused library for online (incremental) covariance and correlation estimation — the online complement to sklearn.covariance.

pip install precise

  • sklearn-style online estimators (partial_fit; covariance_/correlation_/precision_/location_); 14 via all_estimators()
  • keyed river-style adapters for dynamic universes (keyed, FixedUniverse, DynamicUniverse)
  • assessment layer (all_assessors()), incl. the Schur pseudo-likelihood
  • recommender (suggest) — frozen, numpy-only
  • numpy-only core (numpy>=1.21, tested on 1.26 & 2.x); optional [pandas]/[research]/[dev] extras; ships py.typed

Breaking vs 0.16.x: the functional "skater" API is removed (from precise.skaters… now raises a pointer to MIGRATING.md), and portfolio/allocation code moved to schur. See MIGRATING.md and CHANGELOG.md.

precise 1.0.0rc1 — online covariance makeover (pre-release)

Choose a tag to compare

@microprediction microprediction released this 05 Jun 02:21

First pre-release of the online covariance/correlation makeover.

Pre-release: pip install precise continues to serve 0.16.7 (pip skips pre-releases). To try this build: pip install precise==1.0.0rc1.

  • sklearn-style online estimators (partial_fit, covariance_/correlation_/precision_/location_); 14 estimators via all_estimators()
  • keyed river-style adapters for dynamic universes
  • assessment layer (all_assessors()), incl. the Schur likelihood
  • recommender (suggest) — frozen, numpy-only
  • numpy-only core; optional [pandas]/[research]/[dev] extras

Breaking vs 0.16.x: the functional "skater" API and portfolio/manager code are removed (see MIGRATING.md). Not yet for downstreams that import precise.skaters....

fix to EMA

Choose a tag to compare

@microprediction microprediction released this 16 Dec 19:57

fix to EMA

quietude!

Choose a tag to compare

@microprediction microprediction released this 05 Dec 14:54
v0.16.5

Merge branch 'main' of https://github.com/microprediction/precise

Clean up and minor fix

Choose a tag to compare

@microprediction microprediction released this 03 Dec 20:12

Clean up and minor fix

covariance skater for dict inputs like river

Choose a tag to compare

@microprediction microprediction released this 03 Dec 19:59

covariance skater for dict inputs like river

Breaking release Python 3.10 and 3.11 are supported only

Choose a tag to compare

@microprediction microprediction released this 03 Dec 03:37
49076d1

Breaking release Python 3.10 and 3.11 are supported only

Breaking release Python 3.10 and 3.11 are supported only

Choose a tag to compare

@microprediction microprediction released this 03 Dec 03:31
b4eb44b

Breaking release Python 3.10 and 3.11 are supported only

Windows compat maybe

Choose a tag to compare

@microprediction microprediction released this 05 Dec 13:20
v0.15.0

windows compat maybe :)