Skip to content

Commit e5d5901

Browse files
Bordatchatoncarmoccaakihironittaawaelchli
authored
releasing 1.8.5 (#16051)
* update chlog * CI: Add remote fetch (#16001) Co-authored-by: thomas <[email protected]> (cherry picked from commit 37fe3f6) * Set the logger explicitly in tests (#15815) (cherry picked from commit 9ed43c6) * [App] Fix `AutoScaler` trying to replicate multiple works in a single machine (#15991) * dont try to replicate new works in the existing machine * update chglog * Update comment * Update src/lightning_app/components/auto_scaler.py * add test (cherry picked from commit c1d0156) * Fix typo in PR titles generated by github-actions bot (#16003) (cherry picked from commit 2dcebc2) * Update docker requirement from <=5.0.3,>=5.0.0 to >=5.0.0,<6.0.2 in /requirements (#16007) Update docker requirement in /requirements Updates the requirements on [docker](https://github.com/docker/docker-py) to permit the latest version. - [Release notes](https://github.com/docker/docker-py/releases) - [Commits](docker/docker-py@5.0.0...6.0.1) --- updated-dependencies: - dependency-name: docker dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit 4083b20) * Update deepdiff requirement from <=5.8.1,>=5.7.0 to >=5.7.0,<6.2.3 in /requirements (#16006) Update deepdiff requirement in /requirements Updates the requirements on [deepdiff](https://github.com/seperman/deepdiff) to permit the latest version. - [Release notes](https://github.com/seperman/deepdiff/releases) - [Changelog](https://github.com/seperman/deepdiff/blob/master/docs/changelog.rst) - [Commits](seperman/deepdiff@5.7.0...6.2.2) --- updated-dependencies: - dependency-name: deepdiff dependency-type: direct:production ... Signed-off-by: dependabot[bot] <[email protected]> Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> (cherry picked from commit 5e705fa) * app: update doctest_skip (#15997) simple Co-authored-by: hhsecond <[email protected]> (cherry picked from commit 4fea6bf) * CI: clean install & share pkg build (#15986) * abstract pkg build * share ci * syntax * Checkgroup * folders * whl 1st * doctest Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> Co-authored-by: Carlos Mocholí <[email protected]> (cherry picked from commit 18a4638) * Adding hint to the logger's error messages (#16034) Co-authored-by: Jirka Borovec <[email protected]> Co-authored-by: Carlos Mocholí <[email protected]> Fixes #15143 (cherry picked from commit 7ce3825) * fix publish * Introduce `{Work,Flow}.lightningignore` (#15818) (cherry picked from commit edd2b42) * [App] Support running on multiple clusters (#16016) (cherry picked from commit d3a7226) * [App] Improve lightning connect experience (#16035) (cherry picked from commit e522a12) * Cleanup cluster waiting (#16054) (cherry picked from commit 6458a5a) * feature(cli): login flow fixes and improvements (#16052) Co-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com> (cherry picked from commit ebe7848) * Add guards to cluster deletion from cli (#16053) Adds guards to cluster deletion. - If cluster has running apps -> throw an error - If cluster has stopped apps -> confirm w/ user that apps and logs will be deleted (cherry picked from commit 64d0ebb) * Load app before setting LIGHTNING_DISPATCHED (#16057) (cherry picked from commit 8d3339a) * [App] Hot fix: Resolve detection of python debugger (#16068) Co-authored-by: thomas <[email protected]> Co-authored-by: Carlos Mocholí <[email protected]> (cherry picked from commit eae56ee) * fix(cloud): detect and ignore venv (#16056) Co-authored-by: Ethan Harris <[email protected]> (cherry picked from commit 3b323c8) * version 1.8.5 * update chlog Co-authored-by: thomas chaton <[email protected]> Co-authored-by: Carlos Mocholí <[email protected]> Co-authored-by: Akihiro Nitta <[email protected]> Co-authored-by: Adrian Wälchli <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com> Co-authored-by: Huy Đỗ <[email protected]> Co-authored-by: Ethan Harris <[email protected]> Co-authored-by: Luca Furst <[email protected]> Co-authored-by: Yurij Mikhalevich <[email protected]> Co-authored-by: Luca Antiga <[email protected]>
1 parent 60b3cc9 commit e5d5901

File tree

77 files changed

+1238
-529
lines changed

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

77 files changed

+1238
-529
lines changed

‎.github/actions/pkg-check/action.yml

Lines changed: 0 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -5,7 +5,6 @@ inputs:
55
pkg-name:
66
description: package name inside lightning.*
77
required: true
8-
default: ""
98
nb-dirs:
109
description: nb of packages in the wrap/distribution
1110
required: false

‎.github/actions/pkg-install/action.yml

Lines changed: 14 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -2,6 +2,9 @@ name: Install and validate the package
22
description: Install and validate the package
33

44
inputs:
5+
pkg-folder:
6+
description: Define folder with packages
7+
required: true
58
pkg-name:
69
description: Package name to import
710
required: true
@@ -14,22 +17,26 @@ runs:
1417
using: "composite"
1518
steps:
1619
- name: Choose package import
20+
working-directory: ${{ inputs.pkg-folder }}
1721
run: |
18-
python -c "print('PKG_IMPORT=' + {'app': 'lightning_app', 'lite': 'lightning_lite', 'pytorch': 'pytorch_lightning', 'lightning': 'lightning'}['${{matrix.pkg-name}}'])" >> $GITHUB_ENV
22+
ls -l
23+
python -c "print('PKG_IMPORT=' + {'app': 'lightning_app', 'lite': 'lightning_lite', 'pytorch': 'pytorch_lightning'}.get('${{matrix.pkg-name}}', 'lightning'))" >> $GITHUB_ENV
24+
python -c "import glob ; ls = glob.glob('*.tar.gz') ; print('PKG_SOURCE=' + ls[0])" >> $GITHUB_ENV
25+
python -c "import glob ; ls = glob.glob('*.whl') ; print('PKG_WHEEL=' + ls[0])" >> $GITHUB_ENV
1926
shell: bash
2027

21-
- name: Install package - archive
22-
working-directory: ./dist
28+
- name: Install package - wheel
29+
working-directory: ${{ inputs.pkg-folder }}
2330
run: |
24-
pip install *.tar.gz ${{ inputs.pip-flags }}
31+
pip install ${PKG_WHEEL} ${{ inputs.pip-flags }}
2532
pip list | grep lightning
2633
python -c "import ${{ env.PKG_IMPORT }}; print(${{ env.PKG_IMPORT }}.__version__)"
2734
shell: bash
2835

29-
- name: Install package - wheel
30-
working-directory: ./dist
36+
- name: Install package - archive
37+
working-directory: ${{ inputs.pkg-folder }}
3138
run: |
32-
pip install *.whl ${{ inputs.pip-flags }}
39+
pip install ${PKG_SOURCE} ${{ inputs.pip-flags }}
3340
pip list | grep lightning
3441
python -c "import ${{ env.PKG_IMPORT }}; print(${{ env.PKG_IMPORT }}.__version__)"
3542
shell: bash

‎.github/actions/pkg-publish/action.yml

Lines changed: 5 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -2,8 +2,8 @@ name: Publish package
22
description: publishing whl and src to PyPI
33

44
inputs:
5-
pkg-pattern:
6-
description: what file pattern is searched in folder, so for example `*_app*`
5+
pkg-folder:
6+
description: define folder with packages
77
required: true
88
pypi-test-token:
99
description: login token for PyPI
@@ -18,10 +18,7 @@ runs:
1818
using: "composite"
1919
steps:
2020

21-
- name: filter packages
22-
run: |
23-
mv dist/${{ inputs.pkg-pattern }} pypi/
24-
ls -l pypi/
21+
- run: ls -lh ${{ inputs.pkg-folder }}
2522
shell: bash
2623

2724
# We do this, since failures on test.pypi aren't that bad
@@ -32,7 +29,7 @@ runs:
3229
user: __token__
3330
password: ${{ inputs.pypi-test-token }}
3431
repository_url: https://test.pypi.org/legacy/
35-
packages_dir: pypi/
32+
packages_dir: ${{ inputs.pkg-folder }}
3633
verbose: true
3734

3835
- name: Publish distribution 📦 to PyPI
@@ -41,9 +38,5 @@ runs:
4138
with:
4239
user: __token__
4340
password: ${{ inputs.pypi-token }}
44-
packages_dir: pypi/
41+
packages_dir: ${{ inputs.pkg-folder }}
4542
verbose: true
46-
47-
- name: filter packages
48-
run: rm pypi/*
49-
shell: bash

‎.github/checkgroup.yml

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -312,6 +312,9 @@ subprojects:
312312
- id: "install"
313313
paths:
314314
- ".actions/**"
315+
- ".github/actions/pkg-check/*"
316+
- ".github/actions/pkg-install/*"
317+
- ".github/workflows/_build-packages.yml"
315318
- ".github/workflows/ci-pkg-install.yml"
316319
- "setup.py"
317320
- "src/**"

‎.github/workflows/_build-packages.yml

Lines changed: 65 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,65 @@
1+
name: Building packages
2+
3+
on:
4+
workflow_call:
5+
inputs:
6+
artifact-name:
7+
description: 'Unique name for collecting artifacts'
8+
required: true
9+
type: string
10+
pkg-names:
11+
description: 'list package names to be build in json format'
12+
required: false
13+
type: string
14+
default: |
15+
["lightning", "app", "lite", "pytorch"]
16+
17+
defaults:
18+
run:
19+
shell: bash
20+
21+
jobs:
22+
23+
init:
24+
runs-on: ubuntu-20.04
25+
steps:
26+
- uses: actions/checkout@v3
27+
- run: |
28+
mkdir dist && touch dist/.placeholder
29+
- uses: actions/upload-artifact@v3
30+
with:
31+
name: ${{ inputs.artifact-name }}
32+
path: dist
33+
34+
35+
build-packages:
36+
needs: init
37+
runs-on: ubuntu-20.04
38+
strategy:
39+
max-parallel: 1 # run sequential to prevent download/upload collisions
40+
matrix:
41+
pkg-name: ${{ fromJSON(inputs.pkg-names) }}
42+
steps:
43+
- uses: actions/checkout@v3
44+
- uses: actions/download-artifact@v3
45+
with:
46+
name: ${{ inputs.artifact-name }}
47+
path: pypi
48+
- uses: actions/setup-python@v4
49+
with:
50+
python-version: 3.9
51+
52+
- run: python -c "print('NB_DIRS=' + str(2 if '${{ matrix.pkg-name }}' == 'pytorch' else 1))" >> $GITHUB_ENV
53+
- uses: ./.github/actions/pkg-check
54+
with:
55+
pkg-name: ${{ matrix.pkg-name }}
56+
nb-dirs: ${{ env.NB_DIRS }}
57+
58+
- run: |
59+
mkdir pypi/${{ matrix.pkg-name }}
60+
cp dist/* pypi/${{ matrix.pkg-name }}/
61+
62+
- uses: actions/upload-artifact@v3
63+
with:
64+
name: ${{ inputs.artifact-name }}
65+
path: pypi

‎.github/workflows/ci-app-examples.yml

Lines changed: 6 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -93,12 +93,16 @@ jobs:
9393

9494
- name: Adjust tests
9595
if: ${{ matrix.pkg-name == 'lightning' }}
96-
run: python .actions/assistant.py copy_replace_imports --source_dir="./tests" --source_import="lightning_app" --target_import="lightning.app"
96+
run: |
97+
python .actions/assistant.py copy_replace_imports --source_dir="./tests" \
98+
--source_import="lightning_app" --target_import="lightning.app"
9799
98100
- name: Adjust examples
99101
if: ${{ matrix.pkg-name != 'lightning' }}
100102
run: |
101-
python .actions/assistant.py copy_replace_imports --source_dir="./examples" --source_import="lightning.app,lightning" --target_import="lightning_app,lightning_app"
103+
python .actions/assistant.py copy_replace_imports --source_dir="./examples" \
104+
--source_import="lightning.app,lightning" \
105+
--target_import="lightning_app,lightning_app"
102106
103107
- name: Switch coverage scope
104108
run: python -c "print('COVERAGE_SCOPE=' + str('lightning' if '${{matrix.pkg-name}}' == 'lightning' else 'lightning_app'))" >> $GITHUB_ENV

‎.github/workflows/ci-app-tests.yml

Lines changed: 7 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -99,12 +99,17 @@ jobs:
9999

100100
- name: Adjust tests
101101
if: ${{ matrix.pkg-name == 'lightning' }}
102-
run: python .actions/assistant.py copy_replace_imports --source_dir="./tests" --source_import="lightning_app,lightning_lite,pytorch_lightning" --target_import="lightning.app,lightning.lite,lightning.pytorch"
102+
run: |
103+
python .actions/assistant.py copy_replace_imports --source_dir="./tests" \
104+
--source_import="lightning_app,lightning_lite,pytorch_lightning" \
105+
--target_import="lightning.app,lightning.lite,lightning.pytorch"
103106
104107
- name: Adjust examples
105108
if: ${{ matrix.pkg-name != 'lightning' }}
106109
run: |
107-
python .actions/assistant.py copy_replace_imports --source_dir="./examples" --source_import="lightning.app,lightning" --target_import="lightning_app,lightning_app"
110+
python .actions/assistant.py copy_replace_imports --source_dir="./examples" \
111+
--source_import="lightning.app,lightning" \
112+
--target_import="lightning_app,lightning_app"
108113
109114
- name: Switch coverage scope
110115
run: python -c "print('COVERAGE_SCOPE=' + str('lightning' if '${{matrix.pkg-name}}' == 'lightning' else 'lightning_app'))" >> $GITHUB_ENV

‎.github/workflows/ci-lite-tests.yml

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -114,7 +114,9 @@ jobs:
114114
115115
- name: Adjust tests
116116
if: ${{ matrix.pkg-name == 'lightning' }}
117-
run: python .actions/assistant.py copy_replace_imports --source_dir="./tests" --source_import="lightning_lite" --target_import="lightning.lite"
117+
run: |
118+
python .actions/assistant.py copy_replace_imports --source_dir="./tests" \
119+
--source_import="lightning_lite" --target_import="lightning.lite"
118120
119121
- name: Testing Warnings
120122
# the stacklevel can only be set on >=3.7

‎.github/workflows/ci-pkg-install.yml

Lines changed: 42 additions & 17 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,9 @@ on:
99
types: [opened, reopened, ready_for_review, synchronize] # added `ready_for_review` since draft is skipped
1010
paths:
1111
- ".actions/**"
12+
- ".github/actions/pkg-check/*"
13+
- ".github/actions/pkg-install/*"
14+
- ".github/workflows/_build-packages.yml"
1215
- ".github/workflows/ci-pkg-install.yml"
1316
- "setup.py"
1417
- "src/**"
@@ -28,49 +31,71 @@ defaults:
2831

2932
jobs:
3033

34+
build-packages:
35+
uses: ./.github/workflows/_build-packages.yml
36+
with:
37+
artifact-name: dist-packages-${{ github.sha }}
38+
3139
install-pkg:
40+
needs: build-packages
3241
runs-on: ${{ matrix.os }}
3342
strategy:
3443
fail-fast: false
3544
matrix:
3645
os: [ubuntu-22.04, macOS-12, windows-2022]
3746
pkg-name: ["app", "lite", "pytorch", "lightning"]
3847
python-version: ["3.7" , "3.10"]
48+
# TODO: add also install from source
3949
steps:
4050
- uses: actions/checkout@v3
41-
4251
- uses: actions/setup-python@v4
4352
with:
4453
python-version: ${{ matrix.python-version }}
45-
46-
- name: DocTests actions
47-
working-directory: .actions/
48-
run: |
49-
pip install -q pytest
50-
python -m pytest setup_tools.py
51-
52-
- run: python -c "print('NB_DIRS=' + str(2 if '${{ matrix.pkg-name }}' == 'pytorch' else 1))" >> $GITHUB_ENV
53-
54-
- uses: ./.github/actions/pkg-check
54+
- uses: actions/download-artifact@v3
5555
with:
56-
pkg-name: ${{ matrix.pkg-name }}
57-
nb-dirs: ${{ env.NB_DIRS }}
56+
name: dist-packages-${{ github.sha }}
57+
path: dist
5858

59+
- run: |
60+
python -c "print('PKG_DIR=' + {'notset': 'lightning'}.get('${{matrix.pkg-name}}', '${{matrix.pkg-name}}'))" >> $GITHUB_ENV
5961
- uses: ./.github/actions/pkg-install
6062
with:
63+
pkg-folder: dist/${{ env.PKG_DIR }}
6164
pkg-name: ${{ matrix.pkg-name }}
6265

63-
- name: Run CLI
64-
# todo: add testing for `lightning_app`
65-
if: ${{ matrix.pkg-name == 'lightning' }}
66+
- name: Run CLI (via python)
67+
if: ${{ matrix.pkg-name == 'lightning' || matrix.pkg-name == 'notset' }}
6668
run: python -m lightning --version
69+
- name: Run CLI (direct bash)
70+
if: ${{ matrix.pkg-name == 'lightning' || matrix.pkg-name == 'app' }}
71+
run: lightning --version
6772

73+
- name: Adjust code for Lit
74+
if: ${{ matrix.pkg-name == 'lightning' || matrix.pkg-name == 'notset' }}
75+
run: |
76+
pip install -q -r .actions/requirements.txt
77+
python .actions/assistant.py copy_replace_imports --source_dir="./src" \
78+
--source_import="pytorch_lightning,lightning_lite,lightning_app" \
79+
--target_import="lightning.pytorch,lightning.lite,lightning.app"
80+
rm -rf src/lightning
81+
- name: Rename src folders
82+
working-directory: src/
83+
run: |
84+
mv pytorch_lightning pl
85+
mv lightning_lite lit_lite
86+
mv lightning_app lit_app
87+
88+
- name: DocTests actions
89+
working-directory: .actions/
90+
run: |
91+
pip install -q pytest
92+
python -m pytest setup_tools.py
6893
- name: DocTest package
6994
env:
7095
LIGHTING_TESTING: 1 # path for require wrapper
7196
PY_IGNORE_IMPORTMISMATCH: 1
7297
run: |
7398
pip install -q "pytest-doctestplus>=0.9.0"
7499
pip list
75-
PKG_NAME=$(python -c "print({'app': 'lightning_app', 'lite': 'lightning_lite', 'pytorch': 'pytorch_lightning', 'lightning': 'lightning'}['${{matrix.pkg-name}}'])")
100+
PKG_NAME=$(python -c "print({'app': 'lit_app', 'lite': 'lit_lite', 'pytorch': 'pl'}.get('${{matrix.pkg-name}}', ''))")
76101
python -m pytest src/${PKG_NAME} --ignore-glob="**/cli/*-template/**"

‎.github/workflows/ci-pytorch-tests.yml

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -169,7 +169,10 @@ jobs:
169169
170170
- name: Adjust tests
171171
if: ${{ matrix.pkg-name == 'lightning' }}
172-
run: python .actions/assistant.py copy_replace_imports --source_dir="./tests" --source_import="pytorch_lightning,lightning_lite" --target_import="lightning.pytorch,lightning.lite"
172+
run: |
173+
python .actions/assistant.py copy_replace_imports --source_dir="./tests" \
174+
--source_import="pytorch_lightning,lightning_lite" \
175+
--target_import="lightning.pytorch,lightning.lite"
173176
174177
- name: Testing Warnings
175178
# the stacklevel can only be set on >=3.7

0 commit comments

Comments
 (0)