Skip to content

Commit d89fa01

Browse files
SunsetWolfSunsetWolf
authored
fix: upgrade the method of installing LightGBM on MacOS (#1980)
* fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * fix: upgrade the method of installing LightGBM on MacOS * add: comments * test: build package && check package * test: build package && check package * test: build package && check package * optimize yml
1 parent 1b42650 commit d89fa01

File tree

4 files changed

+38
-36
lines changed

4 files changed

+38
-36
lines changed

‎.github/workflows/test_qlib_from_pip.yml‎

Lines changed: 5 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -13,7 +13,7 @@ jobs:
1313
runs-on: ${{ matrix.os }}
1414
strategy:
1515
matrix:
16-
os: [windows-latest, ubuntu-24.04, ubuntu-22.04, macos-13, macos-14, macos-15]
16+
os: [windows-latest, ubuntu-24.04, ubuntu-22.04, macos-14, macos-15]
1717
# In github action, using python 3.7, pip install will not match the latest version of the package.
1818
# Also, python 3.7 is no longer supported from macos-14, and will be phased out from macos-13 in the near future.
1919
# All things considered, we have removed python 3.7.
@@ -46,19 +46,12 @@ jobs:
4646
python -m pip install pyqlib
4747
python -m pip install "joblib<=1.4.2"
4848
49-
# install.sh file contents from: https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh
50-
# brew_install.sh file contents from: https://raw.githubusercontent.com/Microsoft/qlib/main/.github/brew_install.sh
5149
- name: Install Lightgbm for MacOS
52-
if: ${{ matrix.os == 'macos-13' || matrix.os == 'macos-14' || matrix.os == 'macos-15' }}
50+
if: ${{ matrix.os == 'macos-14' || matrix.os == 'macos-15' }}
5351
run: |
54-
/bin/bash -c "$(curl -fsSL https://github.com/SunsetWolf/qlib_dataset/releases/download/maocs_lightgbm/install.sh)"
55-
/bin/bash -c "$(curl -fsSL https://github.com/SunsetWolf/qlib_dataset/releases/download/maocs_lightgbm/brew_install.sh)"
56-
HOMEBREW_NO_AUTO_UPDATE=1 brew install lightgbm
57-
# FIX MacOS error: Segmentation fault
58-
# reference: https://github.com/microsoft/LightGBM/issues/4229
59-
wget https://raw.githubusercontent.com/Homebrew/homebrew-core/fb8323f2b170bd4ae97e1bac9bf3e2983af3fdb0/Formula/libomp.rb
60-
brew unlink libomp
61-
brew install libomp.rb
52+
brew update
53+
brew install libomp || brew reinstall libomp
54+
python -m pip install --no-binary=:all: lightgbm
6255
6356
# When the new version is released it should be changed to:
6457
# python -m qlib.cli.data qlib_data --target_dir ~/.qlib/qlib_data/cn_data --region cn

‎.github/workflows/test_qlib_from_source.yml‎

Lines changed: 24 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ jobs:
1414
runs-on: ${{ matrix.os }}
1515
strategy:
1616
matrix:
17-
os: [windows-latest, ubuntu-24.04, ubuntu-22.04, macos-13, macos-14, macos-15]
17+
os: [windows-latest, ubuntu-24.04, ubuntu-22.04, macos-14, macos-15]
1818
# In github action, using python 3.7, pip install will not match the latest version of the package.
1919
# Also, python 3.7 is no longer supported from macos-14, and will be phased out from macos-13 in the near future.
2020
# All things considered, we have removed python 3.7.
@@ -34,7 +34,7 @@ jobs:
3434
python -m pip install --upgrade pip
3535
3636
- name: Installing pytorch for macos
37-
if: ${{ matrix.os == 'macos-13' || matrix.os == 'macos-14' || matrix.os == 'macos-15' }}
37+
if: ${{ matrix.os == 'macos-14' || matrix.os == 'macos-15' }}
3838
run: |
3939
python -m pip install torch torchvision torchaudio
4040
@@ -83,19 +83,12 @@ jobs:
8383
python scripts/get_data.py qlib_data --name qlib_data_simple --target_dir ~/.qlib/qlib_data/cn_data --interval 1d --region cn
8484
python scripts/get_data.py download_data --file_name rl_data.zip --target_dir tests/.data/rl
8585
86-
# install.sh file contents from: https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh
87-
# brew_install.sh file contents from: https://raw.githubusercontent.com/Microsoft/qlib/main/.github/brew_install.sh
8886
- name: Install Lightgbm for MacOS
89-
if: ${{ matrix.os == 'macos-13' || matrix.os == 'macos-14' || matrix.os == 'macos-15' }}
87+
if: ${{ matrix.os == 'macos-14' || matrix.os == 'macos-15' }}
9088
run: |
91-
/bin/bash -c "$(curl -fsSL https://github.com/SunsetWolf/qlib_dataset/releases/download/maocs_lightgbm/install.sh)"
92-
/bin/bash -c "$(curl -fsSL https://github.com/SunsetWolf/qlib_dataset/releases/download/maocs_lightgbm/brew_install.sh)"
93-
HOMEBREW_NO_AUTO_UPDATE=1 brew install lightgbm
94-
# FIX MacOS error: Segmentation fault
95-
# reference: https://github.com/microsoft/LightGBM/issues/4229
96-
wget https://raw.githubusercontent.com/Homebrew/homebrew-core/fb8323f2b170bd4ae97e1bac9bf3e2983af3fdb0/Formula/libomp.rb
97-
brew unlink libomp
98-
brew install libomp.rb
89+
brew update
90+
brew install libomp || brew reinstall libomp
91+
python -m pip install --no-binary=:all: lightgbm
9992
10093
- name: Check Qlib ipynb with nbconvert
10194
run: |
@@ -106,7 +99,24 @@ jobs:
10699
python -m pip install numba
107100
python qlib/cli/run.py examples/benchmarks/LightGBM/workflow_config_lightgbm_Alpha158.yaml
108101
109-
- name: Unit tests with Pytest
102+
- name: Unit tests with Pytest (MacOS)
103+
if: ${{ matrix.os == 'macos-14' || matrix.os == 'macos-15' }}
104+
uses: nick-fields/retry@v2
105+
with:
106+
timeout_minutes: 60
107+
max_attempts: 3
108+
command: |
109+
# Limit the number of threads in various libraries to prevent Segmentation faults caused by OpenMP multithreading conflicts under macOS.
110+
export OMP_NUM_THREADS=1 # Limit the number of OpenMP threads
111+
export MKL_NUM_THREADS=1 # Limit the number of Intel MKL threads
112+
export NUMEXPR_NUM_THREADS=1 # Limit the number of NumExpr threads
113+
export OPENBLAS_NUM_THREADS=1 # Limit the number of OpenBLAS threads
114+
export VECLIB_MAXIMUM_THREADS=1 # Limit the number of macOS Accelerate/vecLib threads
115+
cd tests
116+
python -m pytest . -m "not slow" --durations=0
117+
118+
- name: Unit tests with Pytest (Ubuntu and Windows)
119+
if: ${{ matrix.os != 'macos-13' && matrix.os != 'macos-14' && matrix.os != 'macos-15' }}
110120
uses: nick-fields/retry@v2
111121
with:
112122
timeout_minutes: 60

‎.github/workflows/test_qlib_from_source_slow.yml‎

Lines changed: 5 additions & 10 deletions
Original file line numberDiff line numberDiff line change
@@ -14,7 +14,7 @@ jobs:
1414
runs-on: ${{ matrix.os }}
1515
strategy:
1616
matrix:
17-
os: [windows-latest, ubuntu-24.04, ubuntu-22.04, macos-13, macos-14, macos-15]
17+
os: [windows-latest, ubuntu-24.04, ubuntu-22.04, macos-14, macos-15]
1818
# In github action, using python 3.7, pip install will not match the latest version of the package.
1919
# Also, python 3.7 is no longer supported from macos-14, and will be phased out from macos-13 in the near future.
2020
# All things considered, we have removed python 3.7.
@@ -40,16 +40,11 @@ jobs:
4040
# install.sh file contents from: https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh
4141
# brew_install.sh file contents from: https://raw.githubusercontent.com/Microsoft/qlib/main/.github/brew_install.sh
4242
- name: Install Lightgbm for MacOS
43-
if: ${{ matrix.os == 'macos-13' || matrix.os == 'macos-14' || matrix.os == 'macos-15' }}
43+
if: ${{ matrix.os == 'macos-14' || matrix.os == 'macos-15' }}
4444
run: |
45-
/bin/bash -c "$(curl -fsSL https://github.com/SunsetWolf/qlib_dataset/releases/download/maocs_lightgbm/install.sh)"
46-
/bin/bash -c "$(curl -fsSL https://github.com/SunsetWolf/qlib_dataset/releases/download/maocs_lightgbm/brew_install.sh)"
47-
HOMEBREW_NO_AUTO_UPDATE=1 brew install lightgbm
48-
# FIX MacOS error: Segmentation fault
49-
# reference: https://github.com/microsoft/LightGBM/issues/4229
50-
wget https://raw.githubusercontent.com/Homebrew/homebrew-core/fb8323f2b170bd4ae97e1bac9bf3e2983af3fdb0/Formula/libomp.rb
51-
brew unlink libomp
52-
brew install libomp.rb
45+
brew update
46+
brew install libomp || brew reinstall libomp
47+
python -m pip install --no-binary=:all: lightgbm
5348
5449
- name: Unit tests with Pytest
5550
uses: nick-fields/retry@v2

‎pyproject.toml‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -97,10 +97,14 @@ analysis = [
9797
"statsmodels",
9898
]
9999

100+
# In the process of releasing a new version, when checking the manylinux package with twine, an error is reported:
101+
# InvalidDistribution: Invalid distribution metadata: unrecognized or malformed field 'license-file'
102+
# To solve this problem, we added license-files here. Refs: https://github.com/pypa/twine/issues/1216
100103
[tool.setuptools]
101104
packages = [
102105
"qlib",
103106
]
107+
license-files = []
104108

105109
[project.scripts]
106110
qrun = "qlib.cli.run:run"

0 commit comments

Comments
 (0)