Skip to content

Commit cc205f6

Browse files
williamFalconwilliamFalcon
authored
0.9.0 readme (#3075)
* 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme * 0.9.0 readme
1 parent 58b1c90 commit cc205f6

File tree

4 files changed

+124
-115
lines changed

4 files changed

+124
-115
lines changed

‎README.md‎

Lines changed: 123 additions & 114 deletions
Original file line numberDiff line numberDiff line change
@@ -4,7 +4,7 @@
44

55
# PyTorch Lightning
66

7-
**The lightweight PyTorch wrapper for ML researchers. Scale your models. Write less boilerplate.**
7+
**The lightweight PyTorch wrapper for high-performance AI research. Scale your models, not the boilerplate.**
88

99

1010
[![PyPI Status](https://badge.fury.io/py/pytorch-lightning.svg)](https://badge.fury.io/py/pytorch-lightning)
@@ -22,6 +22,8 @@
2222
-->
2323
</div>
2424

25+
###### *Codecov is > 90%+ but build delays may show less
26+
2527
---
2628
## Trending contributors
2729

@@ -54,6 +56,8 @@
5456

5557
</center>
5658

59+
## Install
60+
5761
Simple installation from PyPI
5862
```bash
5963
pip install pytorch-lightning
@@ -67,42 +71,86 @@ conda install pytorch-lightning -c conda-forge
6771
## Docs
6872
- [master](https://pytorch-lightning.readthedocs.io/en/latest)
6973
- [stable](https://pytorch-lightning.readthedocs.io/en/stable)
74+
- [0.9.0](https://pytorch-lightning.readthedocs.io/en/0.9.0/)
7075
- [0.8.5](https://pytorch-lightning.readthedocs.io/en/0.8.5/)
7176
- [0.8.4](https://pytorch-lightning.readthedocs.io/en/0.8.4/)
7277
- [0.8.3](https://pytorch-lightning.readthedocs.io/en/0.8.3/)
7378
- [0.8.1](https://pytorch-lightning.readthedocs.io/en/0.8.1/)
74-
- [0.7.6](https://pytorch-lightning.readthedocs.io/en/0.7.6/)
7579

7680
## PyTorch Lightning is just organized PyTorch
77-
![PT to PL](https://github.com/PyTorchLightning/pytorch-lightning/blob/master/docs/source/_images/general/fast_2.gif)
78-
79-
Lightning is a way to organize your PyTorch code to decouple the science code from the engineering.
80-
It's more of a PyTorch style-guide than a framework.
81+
![PT to PL](/docs/source/_images/general/pl_quick_start_full_compressed.gif)
8182

82-
In Lightning, you organize your code into 3 distinct categories:
83+
Lightning disentangles PyTorch code to decouple the science from the engineering
84+
by organizing it into 4 categories:
8385

84-
1. Research code (goes in the LightningModule).
86+
1. Research code (the LightningModule).
8587
2. Engineering code (you delete, and is handled by the Trainer).
8688
3. Non-essential research code (logging, etc... this goes in Callbacks).
89+
4. Data (use PyTorch Dataloaders or organize them into a LightningDataModule)
8790

8891
Once you do this, you can train on multiple-GPUs, TPUs, CPUs and even in 16-bit precision without changing your code!
8992

9093
Get started with our [QUICK START PAGE](https://pytorch-lightning.readthedocs.io/en/stable/new-project.html)
9194

9295
---
93-
### [PyTorch Lightning Masterclass (new lessons weekly)](https://www.youtube.com/watch?v=DbESHcCoWbM&list=PLaMu-SDt_RB5NUm67hU2pdE75j6KaIOv2)
94-
[![IMAGE ALT TEXT HERE](docs/source/_images/general/PTL101_youtube_thumbnail.jpg)](https://www.youtube.com/watch?v=DbESHcCoWbM&list=PLaMu-SDt_RB5NUm67hU2pdE75j6KaIOv2)
96+
97+
## README Table of Contents
98+
- [Masterclass](https://github.com/PytorchLightning/pytorch-lightning#pytorch-lightning-masterclass-new-lessons-weekly)
99+
- [Demo](https://github.com/PytorchLightning/pytorch-lightning#demo)
100+
- [Advanced Examples](https://github.com/PytorchLightning/pytorch-lightning#advanced-examples)
101+
- [Testing Rigour](https://github.com/PytorchLightning/pytorch-lightning#testing-rigour)
102+
- [Does Lightning slow my PyTorch](https://github.com/PytorchLightning/pytorch-lightning#does-lightning-slow-my-pytorch)
103+
- [Flexibility](https://github.com/PytorchLightning/pytorch-lightning#how-flexible-is-it)
104+
- [What does Lightning control for me?](https://github.com/PytorchLightning/pytorch-lightning#what-does-lightning-control-for-me)
105+
- [Converting to Lightning](https://github.com/PytorchLightning/pytorch-lightning#how-much-effort-is-it-to-convert)
106+
- [New Project](https://github.com/PytorchLightning/pytorch-lightning#starting-a-new-project)
107+
- [Why do I need Lightning?](https://github.com/PytorchLightning/pytorch-lightning#why-do-i-want-to-use-lightning)
108+
- [Support](https://github.com/PytorchLightning/pytorch-lightning#support)
109+
- [Supported Research use cases](https://github.com/PytorchLightning/pytorch-lightning#what-types-of-research-works)
110+
- [Visualization](https://github.com/PytorchLightning/pytorch-lightning#visualization)
111+
- [Tutorials](https://github.com/PytorchLightning/pytorch-lightning#tutorials)
112+
- [Asking for help](https://github.com/PytorchLightning/pytorch-lightning#asking-for-help)
113+
- [FAQ](https://github.com/PytorchLightning/pytorch-lightning#faq)
114+
- [Bleeding edge install](https://github.com/PytorchLightning/pytorch-lightning#bleeding-edge)
115+
- [Lightning team](https://github.com/PytorchLightning/pytorch-lightning#lightning-team)
116+
- [BibTex](https://github.com/PytorchLightning/pytorch-lightning#bibtex)
117+
95118
---
119+
### [PyTorch Lightning Masterclass (new lessons weekly)](https://www.youtube.com/watch?v=DbESHcCoWbM&list=PLaMu-SDt_RB5NUm67hU2pdE75j6KaIOv2)
120+
121+
<div style="display: flex">
122+
<div>
123+
<p>From PyTorch to PyTorch Lightning</p>
124+
<a href="https://www.youtube.com/watch?v=DbESHcCoWbM&list=PLaMu-SDt_RB5NUm67hU2pdE75j6KaIOv2">
125+
<img alt="From PyTorch to PyTorch Lightning" src="https://github.com/PyTorchLightning/pytorch-lightning/blob/master/docs/source/_images/general/PTL101_youtube_thumbnail.jpg" width=250">
126+
</a>
127+
</div>
128+
<div style="margin-top: 5px">
129+
<p>Converting a VAE to PyTorch Lightning</p>
130+
<a href="https://www.youtube.com/watch?v=QHww1JH7IDU">
131+
<img alt="From PyTorch to PyTorch Lightning" src="https://github.com/PyTorchLightning/pytorch-lightning/blob/master/docs/source/_images/general/tutorial_cover.jpg" width=250">
132+
</a>
133+
</div>
134+
</div>
96135

97-
## [Refactoring your PyTorch code + benefits + full walk-through](https://www.youtube.com/watch?v=QHww1JH7IDU)
98-
[![Watch the video](docs/source/_images/general/tutorial_cover.jpg)](https://www.youtube.com/watch?v=QHww1JH7IDU)
136+
137+
---
99138

100139
## Demo
101-
Here's a minimal example without a validation or test loop.
140+
Here's a minimal example without a test loop.
102141

103142
```python
104-
# this is just a plain nn.Module with some structure
143+
import os
144+
import torch
145+
import torch.nn.functional as F
146+
from torchvision.datasets import MNIST
147+
from torch.utils.data import DataLoader, random_split
148+
from torchvision import transforms
149+
import pytorch_lightning as pl
150+
```
105151

152+
```python
153+
# this is just a plain nn.Module with some structure
106154
class LitClassifier(pl.LightningModule):
107155

108156
def __init__(self):
@@ -112,29 +160,65 @@ class LitClassifier(pl.LightningModule):
112160
def forward(self, x):
113161
return torch.relu(self.l1(x.view(x.size(0), -1)))
114162

115-
def training_step(self, batch, batch_nb):
163+
def training_step(self, batch, batch_idx):
164+
x, y = batch
165+
y_hat = self(x)
166+
loss = F.cross_entropy(y_hat, y)
167+
result = pl.TrainResult(loss)
168+
result.log('train_loss', loss, on_epoch=True)
169+
return result
170+
171+
def validation_step(self, batch, batch_idx):
116172
x, y = batch
117-
loss = F.cross_entropy(self(x), y)
118-
tensorboard_logs = {'train_loss': loss}
119-
return {'loss': loss, 'log': tensorboard_logs}
173+
y_hat = self(x)
174+
loss = F.cross_entropy(y_hat, y)
175+
result = pl.EvalResult(checkpoint_on=loss)
176+
result.log('val_loss', loss)
177+
return result
120178

121179
def configure_optimizers(self):
122180
return torch.optim.Adam(self.parameters(), lr=0.02)
123181

124182
# train!
125-
train_loader = DataLoader(MNIST(os.getcwd(), train=True, download=True, transform=transforms.ToTensor()), batch_size=32)
183+
dataset = MNIST(os.getcwd(), download=True, transform=transforms.ToTensor())
184+
train, val = random_split(dataset, [55000, 5000])
126185

127186
model = LitClassifier()
128-
trainer = pl.Trainer(gpus=8, precision=16)
129-
trainer.fit(model, train_loader)
187+
trainer = pl.Trainer()
188+
trainer.fit(model, DataLoader(train), DataLoader(val))
130189
```
131190

132-
Other examples:
191+
---
192+
193+
## Advanced Examples
194+
195+
###### Hello world
133196
[MNIST hello world](https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=gEulmrbxwaYL)
134-
[GAN](https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=P0bSmCw57aV5)
197+
[MNIST on TPUs](https://colab.research.google.com/drive/1-_LKx4HwAxl5M6xPJmqAAu444LTDQoa3)
198+
199+
###### Contrastive Learning
200+
[BYOL](https://pytorch-lightning-bolts.readthedocs.io/en/latest/self_supervised_models.html#byol)
201+
[CPC v2](https://pytorch-lightning-bolts.readthedocs.io/en/latest/self_supervised_models.html#cpc-v2)
202+
[Moco v2](https://pytorch-lightning-bolts.readthedocs.io/en/latest/self_supervised_models.html#moco-v2)
203+
[SIMCLR](https://pytorch-lightning-bolts.readthedocs.io/en/latest/self_supervised_models.html#simclr)
204+
205+
###### NLP
135206
[BERT](https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=7uQVI-xv9Ddj)
207+
[GPT-2](https://pytorch-lightning-bolts.readthedocs.io/en/latest/convolutional.html#gpt-2)
208+
209+
###### Reinforcement Learning
136210
[DQN](https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=NWvMLBDySQI5)
137-
[MNIST on TPUs](https://colab.research.google.com/drive/1-_LKx4HwAxl5M6xPJmqAAu444LTDQoa3)
211+
[Dueling-DQN](https://pytorch-lightning-bolts.readthedocs.io/en/latest/reinforce_learn.html#dueling-dqn)
212+
[Reinforce](https://pytorch-lightning-bolts.readthedocs.io/en/latest/reinforce_learn.html#reinforce)
213+
214+
###### Vision
215+
[GAN](https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=P0bSmCw57aV5)
216+
217+
###### Classic ML
218+
[Logistic Regression](https://pytorch-lightning-bolts.readthedocs.io/en/latest/classic_ml.html#logistic-regression)
219+
[Linear Regression](https://pytorch-lightning-bolts.readthedocs.io/en/latest/classic_ml.html#linear-regression)
220+
221+
---
138222

139223
## Testing Rigour
140224
All the automated code by the Trainer is [tested rigorously with every new PR](https://github.com/PyTorchLightning/pytorch-lightning/tree/master/tests).
@@ -145,7 +229,11 @@ For every PR we test all combinations of:
145229
- Linux, OSX, Windows
146230
- Multiple GPUs
147231

148-
**How does performance compare with vanilla PyTorch?**
232+
---
233+
234+
## Does Lightning Slow my PyTorch
235+
No! Lightning is meant for research/production cases that require high-performance.
236+
149237
We have tests to ensure we get the EXACT same results in under 600 ms difference per epoch. In reality, lightning adds about a 300 ms overhead per epoch.
150238
[Check out the parity tests here](https://github.com/PyTorchLightning/pytorch-lightning/tree/master/benchmarks).
151239

@@ -160,10 +248,6 @@ For example, here you could do your own backward pass without worrying about GPU
160248

161249
```python
162250
class LitModel(LightningModule):
163-
def optimizer_step(self, current_epoch, batch_idx, optimizer, optimizer_idx,
164-
second_order_closure=None, on_tpu=False, using_native_amp=False, using_lbfgs=False):
165-
optimizer.step()
166-
167251
def optimizer_zero_grad(self, current_epoch, batch_idx, optimizer, opt_idx):
168252
optimizer.zero_grad()
169253
```
@@ -203,64 +287,8 @@ Although your research/production project might start simple, once you add thing
203287
- 100+ community contributors.
204288

205289
Lightning is also part of the [PyTorch ecosystem](https://pytorch.org/ecosystem/) which requires projects to have solid testing, documentation and support.
206-
207290
---
208291

209-
## README Table of Contents
210-
- [How do I use it](https://github.com/PytorchLightning/pytorch-lightning#how-do-i-do-use-it)
211-
- [What lightning automates](https://github.com/PytorchLightning/pytorch-lightning#what-does-lightning-control-for-me)
212-
- [Tensorboard integration](https://github.com/PytorchLightning/pytorch-lightning#tensorboard)
213-
- [Lightning features](https://github.com/PytorchLightning/pytorch-lightning#lightning-automates-all-of-the-following-each-is-also-configurable)
214-
- [Examples](https://github.com/PytorchLightning/pytorch-lightning#examples)
215-
- [Tutorials](https://github.com/PytorchLightning/pytorch-lightning#tutorials)
216-
- [Asking for help](https://github.com/PytorchLightning/pytorch-lightning#asking-for-help)
217-
- [Contributing](https://github.com/PytorchLightning/pytorch-lightning/blob/master/.github/CONTRIBUTING.md)
218-
- [Bleeding edge install](https://github.com/PytorchLightning/pytorch-lightning#bleeding-edge)
219-
- [Lightning Design Principles](https://github.com/PytorchLightning/pytorch-lightning#lightning-design-principles)
220-
- [Lightning team](https://github.com/PytorchLightning/pytorch-lightning#lightning-team)
221-
- [FAQ](https://github.com/PytorchLightning/pytorch-lightning#faq)
222-
223-
---
224-
225-
## Realistic example
226-
Here's how you would organize a realistic PyTorch project into Lightning.
227-
228-
![PT to PL](docs/source/_images/mnist_imgs/pt_to_pl.jpg)
229-
230-
The LightningModule defines a *system* such as seq-2-seq, GAN, etc...
231-
It can ALSO define a simple classifier.
232-
233-
In summary, you:
234-
235-
1. Define a [LightningModule](https://pytorch-lightning.rtfd.io/en/latest/lightning-module.html)
236-
```python
237-
class LitSystem(pl.LightningModule):
238-
239-
def __init__(self):
240-
super().__init__()
241-
# not the best model...
242-
self.l1 = torch.nn.Linear(28 * 28, 10)
243-
244-
def forward(self, x):
245-
return torch.relu(self.l1(x.view(x.size(0), -1)))
246-
247-
def training_step(self, batch, batch_idx):
248-
...
249-
```
250-
251-
2. Fit it with a [Trainer](https://pytorch-lightning.rtfd.io/en/latest/pytorch_lightning.trainer.html)
252-
```python
253-
from pytorch_lightning import Trainer
254-
255-
model = LitSystem()
256-
257-
# most basic trainer, uses good defaults
258-
trainer = Trainer()
259-
trainer.fit(model)
260-
```
261-
262-
[Check out the COLAB demo here](https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=HOk9c4_35FKg)
263-
264292
## What types of research works?
265293
Anything! Remember, that this is just organized PyTorch code.
266294
The Training step defines the core complexity found in the training loop.
@@ -357,31 +385,6 @@ Lightning has out-of-the-box integration with the popular logging/visualizing fr
357385
- Experiment management
358386
- [Full list here](https://pytorch-lightning.readthedocs.io/en/latest/#common-use-cases)
359387

360-
361-
## Running speed
362-
Migrating to lightning does not mean compromising on speed! You can expect an overhead of about 300 ms per epoch compared with pure PyTorch.
363-
364-
365-
## Examples
366-
Check out this awesome list of research papers and implementations done with Lightning.
367-
368-
- [Contextual Emotion Detection (DoubleDistilBert)](https://github.com/PyTorchLightning/emotion_transformer)
369-
- [Generative Adversarial Network](https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=TyYOdg8g77P0)
370-
- [Hyperparameter optimization with Optuna](https://github.com/optuna/optuna/blob/master/examples/pytorch_lightning_simple.py)
371-
- [Hyperparameter optimization with Ray Tune](https://docs.ray.io/en/master/tune/tutorials/tune-pytorch-lightning.html)
372-
- [Image Inpainting using Partial Convolutions](https://github.com/ryanwongsa/Image-Inpainting)
373-
- [MNIST on TPU](https://colab.research.google.com/drive/1-_LKx4HwAxl5M6xPJmqAAu444LTDQoa3#scrollTo=BHBz1_AnamN_)
374-
- [NER (transformers, TPU, huggingface)](https://colab.research.google.com/drive/1dBN-wwYUngLYVt985wGs_OKPlK_ANB9D)
375-
- [NeuralTexture (CVPR)](https://github.com/PyTorchLightning/neuraltexture)
376-
- [Recurrent Attentive Neural Process](https://github.com/PyTorchLightning/attentive-neural-processes)
377-
- [Siamese Nets for One-shot Image Recognition](https://github.com/PyTorchLightning/Siamese-Neural-Networks)
378-
- [Speech Transformers](https://github.com/PyTorchLightning/speech-transformer-pytorch_lightning)
379-
- [Transformers transfer learning (Huggingface)](https://colab.research.google.com/drive/1F_RNcHzTfFuQf-LeKvSlud6x7jXYkG31#scrollTo=yr7eaxkF-djf)
380-
- [Transformers text classification](https://github.com/ricardorei/lightning-text-classification)
381-
- [VAE Library of over 18+ VAE flavors](https://github.com/AntixK/PyTorch-VAE)
382-
- [Transformers Question Answering (SQuAD)](https://github.com/tshrjn/Finetune-QA/)
383-
- [Pytorch-Lightning + Microsoft NNI with Docker](https://github.com/davinnovation/pytorch-boilerplate)
384-
385388
## Tutorials
386389
Check out our [introduction guide](https://pytorch-lightning.readthedocs.io/en/latest/introduction_guide.html) to get started.
387390
Or jump straight into [our tutorials](https://pytorch-lightning.readthedocs.io/en/latest/#tutorials).
@@ -400,10 +403,12 @@ If you have any questions, feel free to:
400403
---
401404

402405
## FAQ
403-
**How do I use Lightning for rapid research?**
406+
**How do I use Lightning for rapid research?**
407+
404408
[Here's a walk-through](https://pytorch-lightning.readthedocs.io/en/latest/introduction_guide.html)
405409

406410
**Why was Lightning created?**
411+
407412
Lightning has 3 goals in mind:
408413

409414
1. Maximal flexibility while abstracting out the common boilerplate across research projects.
@@ -459,15 +464,19 @@ pip install https://github.com/PytorchLightning/pytorch-lightning/archive/0.X.Y.
459464
- Justus Schock [(justusschock)](https://github.com/justusschock) (Former Core Member PyTorch Ignite)
460465

461466
#### Core Maintainers
462-
463-
- Nick Eggert [(neggert)](https://github.com/neggert)
464-
- Jeff Ling [(jeffling)](https://github.com/jeffling)
465467
- Jeremy Jordan [(jeremyjordan)](https://github.com/jeremyjordan)
466468
- Tullie Murrell [(tullie)](https://github.com/tullie)
467469
- Adrian Wälchli [(awaelchli)](https://github.com/awaelchli)
468470
- Nicki Skafte [(skaftenicki)](https://github.com/SkafteNicki)
469471
- Peter Yu [(yukw777)](https://github.com/yukw777)
470472
- Rohit Gupta [(rohitgr7)](https://github.com/rohitgr7)
473+
- Nathan Raw[(nateraw)](https://github.com/nateraw)
474+
- Ananya Harsh Jha [(ananyahjha93)](https://github.com/ananyahjha93)
475+
- Teddy Koker [(teddykoker)](https://github.com/teddykoker)
476+
477+
#### Alumni
478+
- Nick Eggert [(neggert)](https://github.com/neggert)
479+
- Jeff Ling [(jeffling)](https://github.com/jeffling)
471480

472481
---
473482

‎docs/source/_images/general/pl_quick_start_full_compressed.gif‎

25.2 MB
Loading

‎docs/source/index.rst‎

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -11,7 +11,7 @@ PyTorch Lightning Documentation
1111
:name: start
1212
:caption: Start Here
1313

14-
3_steps
14+
new-project
1515
introduction_guide
1616
performance
1717

‎docs/source/3_steps.rst‎ renamed to ‎docs/source/new-project.rst‎

File renamed without changes.

0 commit comments

Comments
 (0)