Merge branch 'main' of https://github.com/microprediction/precise

Author: microprediction

LITERATURE.md

@@ -436,3 +436,23 @@ out-of-sample risk and return characteristics. In addition, a practical approach
 under box and group constraints is introduced. A comprehensive set of portfolio simulations over 6 equity
 universes demonstrates the appeal of the algorithm for portfolios consisting of 20 − 200 assets. HRP delivers
 highly diversified allocations with low volatility, low portfolio turnover and competitive performance metrics
+
+## Schur Complement and Symmetric Positive Semidefinite Matrices [pdf](https://www.cis.upenn.edu/~jean/schur-comp.pdf)
+Jean Gallier
+
+In this note, we provide some details and proofs of some results from Appendix A.5 (especially
+Section A.5.5) of Convex Optimization by Boyd and Vandenberghe. 
+
+
+## Fast and accurate techniques for computing schur complements and performing numerical coarse graining [slides](https://amath.colorado.edu/faculty/martinss/Talks/2009_banff.pdf)
+Gunnar Martinsson
+
+## A Closer look at the Minimum-Variance Portfolio Optimization Model [pdf](https://downloads.hindawi.com/journals/mpe/2019/1452762.pdf)
+Zhifeng Dai
+
+Recently, by imposing the regularization term to objective function or additional norm constraint to portfolio weights, a number
+of alternative portfolio strategies have been proposed to improve the empirical performance of the minimum-variance portfolio.
+In this paper, we firstly examine the relation between the weight norm-constrained method and the objective function regularization method in minimum-variance problems by analyzing the Karush–Kuhn–Tucker conditions of their Lagrangian
+functions. We give the range of parameters for the two models and the corresponding relationship of parameters. Given the range
+and manner of parameter selection, it will help researchers and practitioners better understand and apply the relevant portfolio
+models. We apply these models to construct optimal portfolios and test the proposed propositions by employing real market data

README.md

@@ -1,19 +1,34 @@
 # precise ![tests](https://github.com/microprediction/precise/workflows/tests/badge.svg) ![License: MIT](https://img.shields.io/badge/License-MIT-yellow.svg)
 
-A collection of autonomous incremental estimators for covariance, precision, correlation and associated quantities.  
+A collection of incremental estimators for covariance, precision, correlation, portfolios and ensembles.   
 
 ## TLDR: "Just a pile of functions that forecast covariance in online fashion"
 The [running_empirical_covariance](https://github.com/microprediction/precise/blob/main/examples_colab_notebooks/running_empirical_population_covariance.ipynb) colab notebook illustrates the style. To see all the other online methods of covariance estimation supplied here, run the [cov skaters manifest](https://github.com/microprediction/precise/blob/main/examples_colab_notebooks/list_all_cov_methods.ipynb) notebook. Or to look at Elo ratings,
-run the [elo_ratings_and_urls](https://github.com/microprediction/precise/blob/main/examples_colab_notebooks/elo_ratings_and_code_urls.ipynb). Oh and if you are looking for M6 example entries, they are [here](https://github.com/microprediction/precise/tree/main/examples_m6/full). 
+run the [elo_ratings_and_urls](https://github.com/microprediction/precise/blob/main/examples_colab_notebooks/elo_ratings_and_code_urls.ipynb). 
+
+<img src="https://github.com/microprediction/precise/blob/main/images/incremental.png" width="700">
+
 
 ## Install 
 
     pip install precise 
+    
+or for latest:
 
-### Covariance skaters  
-Similar in style to skaters used in the [timemachines](https://github.com/microprediction/timemachines) package, this package may be thought of as a collection of covariance prediction functions taking one vector at a time, and also the prior state, and spitting out a prediction mean vector *x*, a prediction covariance *x_cov*, and a posterior state whose interpretation is the responsibility of the skater, not the caller. 
+    pip install git+https://github.com/microprediction/precise.git
 
-This mildly unusual convention requires the caller to maintain state from one call to the next:  
+## M6 Financial forecasting contest utilities
+You *could* use this library to enter the M6 Financial Forecasting competition:
+
+   1. Pick a cov estimator (i.e. a "cov skater"), if you wish
+   2. Pick a portfolio generator, if you wish
+   3. Pick extra shrinkage params, if you wish
+   4. Pick love and hate ticker lists, if you wish
+
+See [precise/examples_m6](https://github.com/microprediction/precise/tree/main/examples_m6) and register at the [m6 competition](https://m6competition.com/). See disclaimer below.  
+
+## Covariance skaters  
+Similar in style to skaters used in the [timemachines](https://github.com/microprediction/timemachines) package, this package may be thought of as a collection of covariance prediction functions taking one vector at a time, and also the prior state, and spitting out a prediction mean vector *x*, a prediction covariance *x_cov*, and a posterior state whose interpretation is the responsibility of the skater, not the caller. 
 
     from precise.skatertools.syntheticdata.miscellaneous import create_correlated_dataset
     from precise.skaters.covariance.runemmp import run_emp_pcov_d0 # <-- Running empirical population covariance
@@ -26,28 +41,25 @@ This mildly unusual convention requires the caller to maintain state from one ca
             x, x_cov, s = run_emp_pcov_d0(s=s, y=y)
         pprint(x_cov)
      
-See [/examples_basic_usage](https://github.com/microprediction/precise/tree/main/examples_basic_usage). 
+See [/examples_basic_usage](https://github.com/microprediction/precise/tree/main/examples_basic_usage). And yes, this mildly unusual convention requires the caller to maintain state from one call to the next:  See the timemachines [faq](https://github.com/microprediction/timemachines/blob/main/FAQ.md) for justification of this style. 
 
-See the timemachines [faq](https://github.com/microprediction/timemachines/blob/main/FAQ.md) for justification of this style. 
-     
-### Skater Elo ratings 
+### Elo ratings 
 
 As noted, see the [elo_ratings_and_urls](https://github.com/microprediction/precise/blob/main/examples_colab_notebooks/elo_ratings_and_code_urls.ipynb).  
 
 ### Browsing for skaters
      
 You can hunt for skaters other than *run_emp_pcov_d0* in [precise/skaters/covariance](https://github.com/microprediction/precise/tree/main/precise/skaters/covariance). There are some location utilities in [precise/whereami](https://github.com/microprediction/precise/blob/main/precise/whereami.py). 
 
-Examples of interpretation:
+### Interpreting skater names 
+Examples:
 
 | Skater name            | Location   | Meaning            |
 |------------------------|------------|--------------------|
 | buf_huber_pcov_d1_a1_b2_n50 | [skaters/covariance/bufhuber](https://github.com/microprediction/precise/blob/main/precise/skaters/covariance/bufhuber.py) | Applies an approach that exploits Huber pseudo-means to a buffer of data of length 50 in need of differencing once, with generalized Huber loss parameters a=1, b=2. | 
 | buf_sk_ld_pcov_d0_n100 | [skaters/covariance/bufsk](https://github.com/microprediction/precise/blob/main/precise/skaters/covariance/bufsk.py) | Applies sk-learn's implementation of Ledoit-Wolf to stationary buffered data of length 100 | 
 | ewa_pm_emp_scov_r01 | [skaters/covariance/ewapartial](https://github.com/microprediction/precise/blob/main/precise/skaters/covariance/ewapartial.py) | Performs an incremental, recency-weighted sample covariance estimate that exploits partial moments. Uses a memory parameter r=0.01 | 
 
-### Reading skater names 
-
 Broad calculation style categories
 
 | Shorthand | Interpretation                                                                  | Incremental ? |
@@ -87,36 +99,27 @@ Differencing hints:
 | d1        | For use on data that is iid after taking one difference | 
      
 
-
-## Portfolio & mixture of experts
-
-See the portfolio directories in [skaters](https://github.com/microprediction/precise/tree/main/precise/skaters). Work in progress. 
-
-## M6 Financial forecasting contest utilities
-
-You can use this library to enter M6 as follows:
-
-   1. Pick a cov estimator (i.e. a "cov skater"), if you wish
-   2. Pick a portfolio generator, if you wish
-   3. Pick extra shrinkage params, if you wish
-   4. Pick love and hate ticker lists, if you wish
-
-See [precise/examples_m6](https://github.com/microprediction/precise/tree/main/examples_m6). 
-
-## Stand-alone utilities
+### Stand-alone covariance utilities
 
    1. The [covariance/statefunctions](https://github.com/microprediction/precise/blob/main/precise/skaters/covarianceutil/statefunctions.py) are illustrated by the example [running_oas_covariance](https://github.com/microprediction/precise/blob/main/examples_basic_usage/running_oas_covariance.py). 
    2. State [covariatnce/statemutations](https://github.com/microprediction/precise/blob/main/precise/skaters/covarianceutil/statemutations.py) do things like ensuring both covariance and precision matrices exist in the state. Or for instance:  s = both_cov(s) ensures both sample and population covariances are present. 
    3. Some [/covariance/datascatterfunctions](https://github.com/microprediction/precise/blob/main/precise/skaters/covarianceutil/datascatterfunctions.py)
-   4. The [/covariance/datacovfunctions](https://github.com/microprediction/precise/blob/main/precise/skaters/covarianceutil/datacovfunctions.py) take data and produce covariance functions. 
+   4. The [/covariance/datafunctions](https://github.com/microprediction/precise/blob/main/precise/skaters/covarianceutil/datafunctions.py) take data and produce covariance functions. 
    5. The  [/covariance/covfunctions](https://github.com/microprediction/precise/blob/main/precise/skaters/covarianceutil/covfunctions.pyy) manipulate 2d cov arrays. 
-  
 
-## Miscellaneous 
+## Portfolios, ensembles & mixture of experts
+Too fluid to document currently. See the portfolio directories in [skaters](https://github.com/microprediction/precise/tree/main/precise/skaters). 
+
+## Miscellaneous remarks
 
  - Here is some related, and potentially related, [literature](https://github.com/microprediction/precise/blob/main/LITERATURE.md). 
  - This is a piece of the microprediction project, should you ever care to [cite](https://github.com/microprediction/microprediction/blob/master/CITE.md) the same. The uses include mixtures of experts models for time-series analysis, buried in [timemachines](https://github.com/microprediction/timemachines/tree/main/timemachines/skatertools) somewhere. 
  - If you just want univariate calculations, and don't want numpy as a dependency, there is [momentum](https://github.com/microprediction/momentum). However if you want univariate forecasts of the variance of something, as distinct from mere online calculations of the same, I would suggest checking the [time-series elo ratings](https://microprediction.github.io/timeseries-elo-ratings/html_leaderboards/special-k_001.html) and the "special" category in particular. 
  - The name of this package refers to precision matrices, not numerical precision. This isn't a source of high precision covariance *calculations* per se. The intent is more in forecasting future realized covariance. Perhaps I'll include some more numerically stable methods from [this survey](https://dbs.ifi.uni-heidelberg.de/files/Team/eschubert/publications/SSDBM18-covariance-authorcopy.pdf) to make the name more fitting. Pull requests are welcome!
  - The intent is that methods are parameter free. However some not-quite autonomous methods admit a few parameters (the factories). A few might even use just one additional scalar parameter *r* with a space-filling curve convention - somewhat akin to the tuning of skaters explained [here](https://github.com/microprediction/timemachines/tree/main/timemachines/skatertools/tuning) in the timemachines package).
 
+## Disclaimer 
+Not investment advice. 
+
+
+