[fix] keep group_keys=False in Average Ensemble#1913
[fix] keep group_keys=False in Average Ensemble#1913SunsetWolf merged 1 commit intomicrosoft:mainfrom
Conversation
|
Hi, @lingbai-kong |
|
@SunsetWolf Sure, you can test with this script: |
|
I'm happy to see your reply, I tested it with the script you provided, and this PR does solve the problem of multi index after |
|
@SunsetWolf This PR doesn't directly fix #1893, PR (#1898) fixed it. I mentioned the issue just because they are same kind of problem: the newest Pandas makes group_keys default to True. |
ff34fd5 to
a107bf3
Compare
a107bf3 to
0733c5e
Compare
|
Hi, @lingbai-kong |
* download orderbook data (microsoft#1754) * download orderbook data * fix CI error * fix CI error * test fix CI error * test fix CI error * test fix CI error * test fix CI error * test fix CI error * test fix CI error * test fix CI error * test fix CI error * test fix CI error * test fix CI error * test fix CI error * test fix CI error * test fix CI error * test fix CI error * test fix CI error * test fix CI error * test fix CI error * test fix CI error * test fix CI error * optimize get_data code * optimize get_data code * optimize get_data code * optimize README --------- Co-authored-by: Linlang <v-linlanglv@microsoft.com> * Enhance README with LightGBM Installation Guidance for Mac M1 Users (microsoft#1766) * Update README.md * Update README.md * Update README.md * Update version * bump version (microsoft#1784) Co-authored-by: Linlang Lv (iSoftStone Information) <v-lvlinlang@microsoft.com> * Fix issue 1729 (microsoft#1776) * fix issue 1729 * fix issue 1729 * fix issue 1729 --------- Co-authored-by: Linlang Lv (iSoftStone Information) <v-lvlinlang@microsoft.com> * Fix the bug of reading string NA as NaN in the function exists_qlib_data. (microsoft#1736) * Fix the bug of reading NA string as NaN in exists_qlib_data. * Fix the .gitignore file. * Update the fix and add some comments. * format with black --------- Co-authored-by: Chuan Xu <chuan.xu@sas.com> Co-authored-by: Linlang Lv (iSoftStone Information) <v-lvlinlang@microsoft.com> * Update dump_pit.py (microsoft#1759) seperated -> separated Co-authored-by: Linlang Lv (iSoftStone Information) <v-lvlinlang@microsoft.com> * data_collector: cn_index: fix missing dependencies package in requirements.txt (microsoft#1770) add yahooquery and openpyxl in requirements.txt Signed-off-by: YuLong Yao <feilongphone@gmail.com> Co-authored-by: Linlang Lv (iSoftStone Information) <v-lvlinlang@microsoft.com> * Delete redundant copy() code to speed up (microsoft#1732) Delete redundant copy() code to speed up Co-authored-by: Linlang Lv (iSoftStone Information) <v-lvlinlang@microsoft.com> * fix docs (microsoft#1721) * fix docs * modify file extension * modify file extension --------- Co-authored-by: Linlang Lv (iSoftStone Information) <v-lvlinlang@microsoft.com> * fix docs (microsoft#1788) Co-authored-by: Linlang Lv (iSoftStone Information) <v-lvlinlang@microsoft.com> * Update README.md's dataset * fix get data error (microsoft#1793) * fix get data error * fix get v0 data error * optimize get_data code * fix pylint error * add comments * update version * bump verison * changed concat of strings to f-strings and redundant type conversion was removed (microsoft#1767) Co-authored-by: Linlang <Lv.Linlang@hotmail.com> * Update Dev in README.md (microsoft#1800) * fix the bug that the HS_SYMBOLS_URL is 404 (microsoft#1758) * fix the bug that the HS_SYMBOLS_URL is 404 * fix bug * format with black * fix pylint error * change error code * fix ci error * fix ci error * optimize code * optimize code * add comments --------- Co-authored-by: Linlang <Lv.Linlang@hotmail.com> * Update utils.py for typo (microsoft#1751) Fix typo Co-authored-by: Linlang <Lv.Linlang@hotmail.com> * fix panic during normalizing the invalid data (microsoft#1698) * fix panic during normalizing the invalid data * fix yaml load * change error to warning * change error code * optimize code --------- Co-authored-by: Linlang <Lv.Linlang@hotmail.com> * fix logo display error (microsoft#1804) * Add "mse" metric option to ALSTM.metric_fn (microsoft#1810) * Fix typo (microsoft#1809) Co-authored-by: LeeYuntong <nukuihayu@outlook.com> * Fix typo (microsoft#1811) Co-authored-by: LeeYuntong <nukuihayu@outlook.com> * Fix TSDataSampler Slicing Bug microsoft#1716 (microsoft#1803) * Fix TSDataSampler Slicing Bug microsoft#1716 * Fix TSDataSampler Slicing Bug microsoft#1716 * Fix TSDataSampler Slicing Bug microsoft#1716 * Fix TSDataSampler Slicing Bug with simplyer implmentation#1716 with Simplified Implementation * Refactor: Fix CI errors by addressing pylint formatting issues * Refactor: Remove extraneous whitespace for improved code formatting with Black * Fix Yahoo daily data format inconsistent (microsoft#1517) * Fix FutureWarning: Passing unit-less datetime64 dtype to .astype is deprecated and will raise in a future version. Pass 'datetime64[ns]' instead * align index format while end date contains current day data * fix black * fix black * optimize code * optimize code * optimize code * fix ci error * check ci error * fix ci error * check ci error * check ci error * check ci error * check ci error * check ci error * check ci error * fix ci error * fix ci error * fix ci error * fix ci error * fix ci error --------- Co-authored-by: Cadenza-Li <362237642@qq.com> Co-authored-by: Linlang <Lv.Linlang@hotmail.com> * change weight data download url (microsoft#1812) * add a note for code standard (microsoft#1814) * add a note for code standard * handle both cases --------- Co-authored-by: taozhiwang <taozhiwa@gmail.com> * Update index_data.py for datatype conversion and alignment (microsoft#1813) * Update index_data.py for data convertion and alignment * Update qlib/utils/index_data.py * Update qlib/utils/index_data.py * fix linting --------- Co-authored-by: taozhiwang <taozhiwa@gmail.com> Co-authored-by: you-n-g <you-n-g@users.noreply.github.com> * Add some misc features. (microsoft#1816) * Normal mod * Black linting * Linting * Nested data loader (microsoft#1822) * nested data loader * Amend * add data loder test * fix pylint error * fix pytest error * fix pytest error * delete comments * Update qlib/contrib/data/handler.py --------- Co-authored-by: Linlang <Lv.Linlang@hotmail.com> * More dataloader example (microsoft#1823) * More dataloader example * optimize code * optimeze code * optimeze code * optimeze code * optimeze code * optimeze code * fix pylint error * fix CI error * fix CI error * Comments * fix error type --------- Co-authored-by: Young <afe.young@gmail.com> * Ptnn4both datatypes and alignment tests (microsoft#1827) * Init model for both dataset * Remove some deprecated code * Add model template; * We must align with previous results * We choose another mode as the initial version * Almost success to run GRU * Successfully run training * Passed general_nn test * gru test * Alignment test passed * comment * fix readme & minor errors * general nn updates & benchmarks * Update examples/benchmarks/GeneralPtNN/workflow_config_gru2mlp.yaml --------- Co-authored-by: Young <afe.young@gmail.com> Co-authored-by: you-n-g <you-n-g@users.noreply.github.com> * 🔥LLM-driven Auto Quant Factory🔥 (microsoft#1840) * Update README.md * Update README.md * Update README.md * fix break img (microsoft#1842) * Update README.md (microsoft#1839) Update data example to 20240809 * Update README.md to show rdagent in qlib front page (microsoft#1848) * update readme * Update README.md add english and chinese link to rdagent * add the logo of rdagent to readme add the logo of rdagent to readme * adjust the height of the logo * improve some works in readme * add a line * add dockerfile (microsoft#1817) * add dockerfile * add execute script * add docs * optimize docs * optimize dockerfile * optimize docs * optimize dockerfile * update code & update README * doc build error * update docs * update code * Update requirements.txt (microsoft#1829) Update urllib3 dependency according to GHSA-34jh-p97f-mpxf * Saurabh12571257/main (microsoft#1866) * Update README.md * test macos ci * test macos ci * test macos ci * fix ci error * fix ci error --------- Co-authored-by: saurabh dave <87791567+saurabh12571257@users.noreply.github.com> * Update README.md * fix duplicate log (microsoft#1661) * fix duplicate log * fix unit test * fix log * fix_duplicate_log * fix_duplicate_log * add comments --------- Co-authored-by: Linlang <Lv.Linlang@hotmail.com> * Fix Async Call (microsoft#1869) * update python version (microsoft#1868) * update python version * fix: Correct selector handling and add time filtering in storage.py * fix: convert index and columns to list in repr methods * feat: Add Makefile for managing project prerequisites * feat: Add Cython extensions for rolling and expanding operations * resolve install error * fix lint error * fix lint error * fix lint error * fix lint error * fix lint error * update build package * update makefile * update ci yaml * fix docs build error * fix ubuntu install error * fix docs build error * fix install error * fix install error * fix install error * fix install error * fix pylint error * fix pylint error * fix pylint error * fix pylint error * fix pylint error E1123 * fix pylint error R0917 * fix pytest error * fix pytest error * fix pytest error * update code * update code * fix ci error * fix pylint error * fix black error * fix pytest error * fix CI error * fix CI error * add python version to CI * add python version to CI * add python version to CI * fix pylint error * fix pytest general nn error * fix CI error * optimize code * add coments * Extended macos version * remove build package --------- Co-authored-by: Young <afe.young@gmail.com> * Update publish (microsoft#1871) * update publish * reformat with black * Update version * Bump version (microsoft#1872) * bump version * bump version * Update README.md * fix_ci_error * fix_ci_error * fix_ci_error * fix_ci_error --------- Co-authored-by: you-n-g <you-n-g@users.noreply.github.com> * Fix the empty price_s case and self.instruments in SBBStrategyEMA. (microsoft#1677) * Fix the empty price_s case and self.instruments in SBBStrategyEMA. * Update qlib/contrib/strategy/rule_strategy.py * Update qlib/contrib/strategy/rule_strategy.py --------- Co-authored-by: you-n-g <you-n-g@users.noreply.github.com> Co-authored-by: Linlang <Lv.Linlang@hotmail.com> * Fix broken URL for RL (microsoft#1881) * fix_issue_1878 * fix_issue_1878 * Update links to chenditc/investment_data to always point to latest release (microsoft#1877) * Update README.md Link to latest release. https://docs.github.com/en/repositories/releasing-projects-on-github/linking-to-releases#linking-to-the-latest-release * Update README.md Link to latest release. https://docs.github.com/en/repositories/releasing-projects-on-github/linking-to-releases#linking-to-the-latest-release * Update README.md Link to latest release. https://docs.github.com/en/repositories/releasing-projects-on-github/linking-to-releases#linking-to-the-latest-release * Update README.md Link to latest release. https://docs.github.com/en/repositories/releasing-projects-on-github/linking-to-releases#linking-to-the-latest-release * Update README.md * Update README.md * Fix csi300 constituents url (microsoft#1883) * fix_csi300_constituents_url * Fix issue in readme * format with black * DRAFT add Data Health Checker (microsoft#1574) * microsoft#854 implement first data health checker draft * microsoft#854 added support for qlib's data format, implemented factor check, reformatted summary * adaptation current dataset * format with black * add data health check to docs * fix sphinx error * fix pylint error * update code * format with black * format with pylint --------- Co-authored-by: Linlang <Lv.Linlang@hotmail.com> * fix pkl file not loading in StaticDataLoader (microsoft#1896) * fix pkl file not loading in StaticDataLoader * resolve hard code * resolve hard code * docs: fix README.md link * fix col name error when fetch data (microsoft#1904) * fix col name error when fetch data * fix col name error when fetch data * fix install qlib error * optimize code * optimize code * optimize code * fix fillna bug (microsoft#1914) * fix fillna bug * fix flake8 error * fix pylint error * update ubuntu version for action * fix pytest error * fix pylint error * fix black error * fix pylint error * add Fillna test * fix black error * add instruments * remove code * Fix issue 1892 (microsoft#1916) * fix: resolve microsoft#1892 by retriving the data page by page * fix: resolve microsoft#1892 by retriving the data page by page * reformat with black --------- Co-authored-by: shengyuhong <shengyuhong@bytedance.com> Co-authored-by: fibers <yu8582@126.com> * fix bugs in the documentation (microsoft#1918) * fix bugs in the documentation * fix docs error * fix ci error (microsoft#1921) * fix ci error * fix ci error * add comments * add comments * [fix] keep group_keys=False in Average Ensemble (microsoft#1913) * fixed a problem with multi index caused by the default value of groupkey (microsoft#1917) * fixed a problem with multi index caused by the default value of groupkey * modify group_key default value * limit pandas verion * format with black * fix docs error * fix docs error * fixed bugs caused by pandas upgrade * remove needless code * reformat with black * limit version & add docs * [Fix]Update data preparation part in README.md (microsoft#1924) * [feat] fix a bug and adapt general_nn for use with rdagent_qlib (microsoft#1928) * update qlib general_nn for rdagent_qlib * fix install lightgbm error * fix install lightgbm error & format with black --------- Co-authored-by: Linlang <Lv.Linlang@hotmail.com> * The plotly figure is empty in the code block "Basic data" (microsoft#1902) * Update detailed_workflow.ipynb the result figure is empty * Update detailed_workflow.ipynb fix issue: the plotly figure is empty * The error message indicated that my code did not comply with the code style guidelines. Specifically, I had used double quotes "notebook" for the string, whereas the required format was single quotes 'notebook'. This has now been corrected. * comply with the code style guidelines. Specifically, I had used double quotes "notebook" for the string, whereas the required format was single quotes 'notebook'. This has now been corrected. * I didn't use nbqa black to reformat my code. Now is done! * recover_code --------- Co-authored-by: Linlang <Lv.Linlang@hotmail.com> * doc: update README.md (microsoft#1929) * Update README.md * Update README.md * Add util function to help automatically get horizon (microsoft#1509) * Add util function to help automatically get horizon * Reformat for CI * Leverage horizon change * Udpate config yaml * Update for formatting * Adapt to pickled handler * Fix CI error * remove blank * Fix lint * Update tests * Remove redundant check * modify the code as suggested * format code with pylint * fix pytest error --------- Co-authored-by: Linlang <Lv.Linlang@hotmail.com> * Implement geometric accumulation mode for risk_analysis function (microsoft#964) (microsoft#1938) Co-authored-by: eabjab <buege.ethan@gmail.com> * Update README.md (microsoft#1940) * Fixing Security Vulnerabilities (microsoft#1941) * Fixing Security Vulnerabilities * Fixing Pylint Error * Fixing Security Vulnerabilities windows * format with black * using returncode to locate problems * fix pylint error * fix: typo (microsoft#1943) * refactor: introduce BaseDataHandler and unify fetch interface (microsoft#1958) * refactor: introduce BaseDataHandler and unify fetch interface * refactor: include data_key in seg_kwargs and simplify segments loop * refactor: default data_key to BaseDataHandler.DK_I in _get_df_by_key * style: fix indentation and remove extra blank lines in data handlers * refactor: use BaseDataHandler.DK_I as default data_key * docs: fix BaseDataHandler docstring grammar and formatting * refactor: remove unused **kwargs from storage fetch methods * docs: refine BaseDataHandler and DataHandler docstrings * refactor: rename BaseDataHandler to DataHandlerABC, update type hints * feat: add flt_col to TSDatasetH and list-to-slice conversion in storage * lint * comment * disable pylint error (microsoft#1960) * disable pylint error * try fix build docs error * try fix build docs error * optimize code * feat: use pydantic-settings for MLflow config and update dependencies (microsoft#1962) * feat: use pydantic-settings for MLflow config and update dependencies * docs * feat: data improve, support parquet (microsoft#1966) * refactor: relocate CLI modules to qlib.cli and update references * refactor: introduce read_as_df and rename csv_path to data_path * lint * refactor: rename csv_path to data_path and use QSettings.provider_uri * fix pylint error * fix get_data command * add comments to CI yaml * update docs --------- Co-authored-by: Linlang <Lv.Linlang@hotmail.com> --------- Signed-off-by: YuLong Yao <feilongphone@gmail.com> Co-authored-by: Linlang <30293408+SunsetWolf@users.noreply.github.com> Co-authored-by: Linlang <v-linlanglv@microsoft.com> Co-authored-by: Xisen Wang <118058822+Xisen-Wang@users.noreply.github.com> Co-authored-by: Young <afe.young@gmail.com> Co-authored-by: Linlang Lv (iSoftStone Information) <v-lvlinlang@microsoft.com> Co-authored-by: Chuan Xu <xuchuan0304@gmail.com> Co-authored-by: Chuan Xu <chuan.xu@sas.com> Co-authored-by: Ikko Eltociear Ashimine <eltociear@gmail.com> Co-authored-by: fei long <feilongphone@gmail.com> Co-authored-by: playfund <windev@foxmail.com> Co-authored-by: you-n-g <you-n-g@users.noreply.github.com> Co-authored-by: Linlang <Lv.Linlang@hotmail.com> Co-authored-by: igeni <kublin@it8.ru> Co-authored-by: Hao Zhao <zhstark@icloud.com> Co-authored-by: block-gpt <5486391+block-gpt@users.noreply.github.com> Co-authored-by: Yang <3349368+m3ngyang@users.noreply.github.com> Co-authored-by: raikiriww <raikiriww@gmail.com> Co-authored-by: Lee Yuntong <89683513+akazeakari@users.noreply.github.com> Co-authored-by: LeeYuntong <nukuihayu@outlook.com> Co-authored-by: 陈屹华 <37462254+YeewahChan@users.noreply.github.com> Co-authored-by: Fivele-Li <128388363+Fivele-Li@users.noreply.github.com> Co-authored-by: Cadenza-Li <362237642@qq.com> Co-authored-by: cyncyw <47289405+taozhiwang@users.noreply.github.com> Co-authored-by: taozhiwang <taozhiwa@gmail.com> Co-authored-by: Another <835166018@qq.com> Co-authored-by: Xu Yang <peteryang@vip.qq.com> Co-authored-by: Di <chenditc@umich.edu> Co-authored-by: saurabh dave <87791567+saurabh12571257@users.noreply.github.com> Co-authored-by: Chia-hung Tai <glaxy.tai@gmail.com> Co-authored-by: codecnotsupported <codecnotsupported@users.noreply.github.com> Co-authored-by: Ben Heckmann <79015931+benheckmann@users.noreply.github.com> Co-authored-by: shengyuhong <shengyuhong@bytedance.com> Co-authored-by: fibers <yu8582@126.com> Co-authored-by: Dred <1577347854@qq.com> Co-authored-by: Yuante Li <104308117+WinstonLiyt@users.noreply.github.com> Co-authored-by: ziphei <41258328+ziphei@users.noreply.github.com> Co-authored-by: eabjab <buege.ethan@gmail.com> Co-authored-by: Emre <emre.kb776@gmail.com>
Description
Explicitly passing the group_keys param in AverageEnsemble
Motivation and Context
The related issues is #1893. And this PR is similar to #1898
When we use the OnlineManager with AverageEnsemble, the

The name datetime occurs multiple timesraises. I believe this caused by the new feature in Pandas 2.0.0, where release notice says "group_keys now defaults to True."How Has This Been Tested?
pytest qlib/tests/test_all_pipeline.pyunder upper directory ofqlib.Screenshots of Test Results (if appropriate):
Types of changes