Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
8 changes: 4 additions & 4 deletions .github/CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -130,11 +130,11 @@ In case you are adding new dependencies, make sure that they are compatible with

### Documentation

To learn about development of docs, check out the docs [README.md](https://github.com/Lightning-AI/lightning/blob/master/docs/README.md).
To learn about development of docs, check out the docs [README.md](https://github.com/Lightning-AI/pytorch-lightning/blob/master/docs/README.md).

### Testing

To learn about tests, check out the tests [README.md](https://github.com/Lightning-AI/lightning/blob/master/tests/README.md).
To learn about tests, check out the tests [README.md](https://github.com/Lightning-AI/pytorch-lightning/blob/master/tests/README.md).

### Pull Request

Expand Down Expand Up @@ -165,8 +165,8 @@ We welcome any useful contribution! For your convenience here's a recommended wo

1. If any of the existing tests fail in your PR on our CI, refer to the following READMEs to identify what's failing and try to address it.

- [Test README](https://github.com/Lightning-AI/lightning/blob/master/tests/README.md)
- [CI/CD README](https://github.com/Lightning-AI/lightning/blob/master/.github/workflows/README.md)
- [Test README](https://github.com/Lightning-AI/pytorch-lightning/blob/master/tests/README.md)
- [CI/CD README](https://github.com/Lightning-AI/pytorch-lightning/tree/master/.github/workflows#readme)

1. When you feel ready for integrating your work, mark your PR "Ready for review".

Expand Down
2 changes: 1 addition & 1 deletion .github/PULL_REQUEST_TEMPLATE.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ Fixes #\<issue_number>
<summary><b>Before submitting</b></summary>

- Was this **discussed/agreed** via a GitHub issue? (not for typos and docs)
- [ ] Did you read the [contributor guideline](https://github.com/Lightning-AI/lightning/blob/master/.github/CONTRIBUTING.md), **Pull Request** section?
- [ ] Did you read the [contributor guideline](https://github.com/Lightning-AI/pytorch-lightning/blob/master/.github/CONTRIBUTING.md), **Pull Request** section?
- [ ] Did you make sure your **PR does only one thing**, instead of bundling different changes together?
- Did you make sure to **update the documentation** with your changes? (if necessary)
- Did you write any **new necessary tests**? (not for typos and docs)
Expand Down
6 changes: 6 additions & 0 deletions .github/markdown-links-config.json
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,12 @@
"ignorePatterns": [
{
"pattern": "^https://github.com/Lightning-AI/pytorch-lightning/pull/"
},
{
"pattern": "^https://dev.azure.com/Lightning-AI/lightning/_apis/build/status"
},
{
"pattern": "^https://codecov.io/gh/Lightning-AI/pytorch-lightning/graph/badge.svg"
}
],
"httpHeaders": [
Expand Down
5 changes: 3 additions & 2 deletions .github/workflows/ci-check-md-links.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,12 +9,13 @@ on:
types: [opened, reopened, ready_for_review, synchronize]
paths:
- ".github/workflows/ci-check-md-links.yml"
- ".github/workflows/markdown.links.config.json"
- ".github/markdown-links-config.json"
- "**/*.md"

jobs:
check-md-links:
uses: Lightning-AI/utilities/.github/workflows/check-md-links.yml@v0.14.3
uses: Lightning-AI/utilities/.github/workflows/check-md-links.yml@main # can be pin with >=0.14.4
with:
config-file: ".github/markdown-links-config.json"
base-branch: "master"
force-check-all: "yes"
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ ______________________________________________________________________

[![Discord](https://img.shields.io/discord/1077906959069626439?style=plastic)](https://discord.gg/VptPCZkGNa)
![GitHub commit activity](https://img.shields.io/github/commit-activity/w/lightning-ai/lightning)
[![license](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/Lightning-AI/lightning/blob/master/LICENSE)
[![license](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/Lightning-AI/pytorch-lightning/blob/master/LICENSE)

<!--
[![CodeFactor](https://www.codefactor.io/repository/github/Lightning-AI/lightning/badge)](https://www.codefactor.io/repository/github/Lightning-AI/lightning)
Expand Down
2 changes: 1 addition & 1 deletion docs/source-fabric/_templates/theme_variables.jinja
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{%- set external_urls = {
'github': 'https://github.com/Lightning-AI/lightning',
'github_issues': 'https://github.com/Lightning-AI/pytorch-lightning/issues',
'contributing': 'https://github.com/Lightning-AI/lightning/blob/master/.github/CONTRIBUTING.md',
'contributing': 'https://github.com/Lightning-AI/pytorch-lightning/blob/master/.github/CONTRIBUTING.md',
'governance': 'https://lightning.ai/docs/pytorch/latest/community/governance.html',
'docs': 'https://lightning.ai/docs/fabric/',
'twitter': 'https://twitter.com/LightningAI',
Expand Down
2 changes: 1 addition & 1 deletion docs/source-fabric/advanced/compile.rst
Original file line number Diff line number Diff line change
Expand Up @@ -221,7 +221,7 @@ needed to achieve the equivalent of `compile(distributed(quantized(model)))`:

model = fabric.setup(model)

For a full example, see our `FP8 Distributed Transformer example <https://github.com/Lightning-AI/lightning/blob/master/examples/fabric/fp8_distributed_transformer>`_.
For a full example, see our `FP8 Distributed Transformer example <https://github.com/Lightning-AI/pytorch-lightning/blob/master/examples/fabric/fp8_distributed_transformer>`_.

----

Expand Down
2 changes: 1 addition & 1 deletion docs/source-fabric/advanced/multiple_setup.rst
Original file line number Diff line number Diff line change
Expand Up @@ -113,4 +113,4 @@ You can pair up as many models and optimizers as you want. For example, two mode
# Set up discriminator
discriminator, optimizer_dis = fabric.setup(discriminator, optimizer_dis)

For a full example of this use case, see our `GAN example <https://github.com/Lightning-AI/lightning/blob/master/examples/fabric/dcgan>`_.
For a full example of this use case, see our `GAN example <https://github.com/Lightning-AI/pytorch-lightning/blob/master/examples/fabric/dcgan>`_.
10 changes: 5 additions & 5 deletions docs/source-fabric/examples/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -10,31 +10,31 @@ Examples
.. displayitem::
:header: Image Classification
:description: Train an image classifier on the MNIST dataset
:button_link: https://github.com/Lightning-AI/lightning/blob/master/examples/fabric/image_classifier
:button_link: https://github.com/Lightning-AI/pytorch-lightning/blob/master/examples/fabric/image_classifier
:col_css: col-md-4
:height: 200
:tag: basic

.. displayitem::
:header: Transformer Language Model
:description: A simple language model that learns to predict the next word in a sentence
:button_link: https://github.com/Lightning-AI/lightning/blob/master/examples/fabric/language_model
:button_link: https://github.com/Lightning-AI/pytorch-lightning/blob/master/examples/fabric/language_model
:col_css: col-md-4
:height: 200
:tag: basic

.. displayitem::
:header: GAN
:description: Train a GAN that generates realistic human faces
:button_link: https://github.com/Lightning-AI/lightning/blob/master/examples/fabric/dcgan
:button_link: https://github.com/Lightning-AI/pytorch-lightning/blob/master/examples/fabric/dcgan
:col_css: col-md-4
:height: 200
:tag: intermediate

.. displayitem::
:header: Meta-Learning
:description: Distributed training with the MAML algorithm on the Omniglot and MiniImagenet datasets
:button_link: https://github.com/Lightning-AI/lightning/blob/master/examples/fabric/meta_learning
:button_link: https://github.com/Lightning-AI/pytorch-lightning/blob/master/examples/fabric/meta_learning
:col_css: col-md-4
:height: 200
:tag: intermediate
Expand All @@ -50,7 +50,7 @@ Examples
.. displayitem::
:header: Reinforcement Learning
:description: Implementation of the Proximal Policy Optimization (PPO) algorithm with multi-GPU support
:button_link: https://github.com/Lightning-AI/lightning/blob/master/examples/fabric/reinforcement_learning
:button_link: https://github.com/Lightning-AI/pytorch-lightning/blob/master/examples/fabric/reinforcement_learning
:col_css: col-md-4
:height: 200
:tag: intermediate
Expand Down
2 changes: 1 addition & 1 deletion docs/source-fabric/fundamentals/convert.rst
Original file line number Diff line number Diff line change
Expand Up @@ -84,7 +84,7 @@ All steps combined, this is how your code will change:


That's it! You can now train on any device at any scale with a switch of a flag.
Check out our before-and-after example for `image classification <https://github.com/Lightning-AI/lightning/blob/master/examples/fabric/image_classifier/README.md>`_ and many more :doc:`examples <../examples/index>` that use Fabric.
Check out our before-and-after example for `image classification <https://github.com/Lightning-AI/pytorch-lightning/blob/master/examples/fabric/image_classifier/README.md>`_ and many more :doc:`examples <../examples/index>` that use Fabric.


----
Expand Down
2 changes: 1 addition & 1 deletion docs/source-pytorch/_templates/theme_variables.jinja
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
{%- set external_urls = {
'github': 'https://github.com/Lightning-AI/lightning',
'github_issues': 'https://github.com/Lightning-AI/pytorch-lightning/issues',
'contributing': 'https://github.com/Lightning-AI/lightning/blob/master/.github/CONTRIBUTING.md',
'contributing': 'https://github.com/Lightning-AI/pytorch-lightning/blob/master/.github/CONTRIBUTING.md',
'governance': 'https://lightning.ai/docs/pytorch/latest/community/governance.html',
'docs': 'https://lightning.ai/docs/pytorch/latest/',
'twitter': 'https://twitter.com/LightningAI',
Expand Down
2 changes: 1 addition & 1 deletion docs/source-pytorch/advanced/compile.rst
Original file line number Diff line number Diff line change
Expand Up @@ -251,7 +251,7 @@ needed to achieve the equivalent of `compile(distributed(quantized(model)))`:

self.model = torch.compile(model)

For a full example, see our `FP8 Distributed Transformer example <https://github.com/Lightning-AI/lightning/blob/master/examples/pytorch/fp8_distributed_transformer>`_.
For a full example, see our `FP8 Distributed Transformer example <https://github.com/Lightning-AI/pytorch-lightning/blob/master/examples/pytorch/fp8_distributed_transformer>`_.

----

Expand Down
2 changes: 1 addition & 1 deletion docs/source-pytorch/community/governance.rst
Original file line number Diff line number Diff line change
Expand Up @@ -67,6 +67,6 @@ and start tracking the development. It is possible that priorities change over t

Commits to the project are exclusively to be added by pull requests on GitHub and anyone in the community is welcome to
review them. However, reviews submitted by
`code owners <https://github.com/Lightning-AI/lightning/blob/master/.github/CODEOWNERS>`_
`code owners <https://github.com/Lightning-AI/pytorch-lightning/blob/master/.github/CODEOWNERS>`_
have higher weight and it is necessary to get the approval of code owners before a pull request can be merged.
Additional requirements may apply case by case.
2 changes: 1 addition & 1 deletion examples/fabric/meta_learning/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
## Meta-Learning - MAML

This is an example of a meta-learning algorithm called [MAML](https://arxiv.org/abs/1703.03400), trained on the
[Omniglot dataset](https://paperswithcode.com/dataset/omniglot-1) of handwritten characters from different alphabets.
[Omniglot dataset](https://github.com/brendenlake/omniglot) of handwritten characters from different alphabets.

The goal of meta-learning in this context is to learn a 'meta'-model trained on many different tasks, such that it can quickly adapt to a new task when trained with very few samples (few-shot learning).
If you are new to meta-learning, have a look at this short [introduction video](https://www.youtube.com/watch?v=ItPEBdD6VMk).
Expand Down
4 changes: 2 additions & 2 deletions src/lightning/data/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ We developed `StreamingDataset` to optimize training of large datasets stored on

Specifically crafted for multi-gpu & multi-node (with [DDP](https://lightning.ai/docs/pytorch/stable/accelerators/gpu_intermediate.html), [FSDP](https://lightning.ai/docs/pytorch/stable/advanced/model_parallel/fsdp.html), etc...), distributed training with large models, it enhances accuracy, performance, and user-friendliness. Now, training efficiently is possible regardless of the data's location. Simply stream in the required data when needed.

The `StreamingDataset` is compatible with any data type, including **images, text, video, audio, geo-spatial, and multimodal data** and it is a drop-in replacement for your PyTorch [IterableDataset](https://pytorch.org/docs/stable/data.html#torch.utils.data.IterableDataset) class. For example, it is used by [Lit-GPT](https://github.com/Lightning-AI/lit-gpt/blob/main/litgpt/data/tinyllama.py) to pretrain LLMs.
The `StreamingDataset` is compatible with any data type, including **images, text, video, audio, geo-spatial, and multimodal data** and it is a drop-in replacement for your PyTorch [IterableDataset](https://pytorch.org/docs/stable/data.html#torch.utils.data.IterableDataset) class. For example, it is used by [Lit-GPT](https://github.com/Lightning-AI/litgpt/blob/main/litgpt/data/tinyllama.py) to pretrain LLMs.

<br/>

Expand Down Expand Up @@ -284,7 +284,7 @@ for batch in tqdm(train_dataloader):

Lightning Data provides a stateful `StreamingDataLoader`. This simplifies resuming training over large datasets.

Note: The `StreamingDataLoader` is used by [Lit-GPT](https://github.com/Lightning-AI/lit-gpt/blob/main/litgpt/data/tinyllama.py) to pretrain LLMs. The statefulness still works when using a mixture of datasets with the `CombinedStreamingDataset`.
Note: The `StreamingDataLoader` is used by [Lit-GPT](https://github.com/Lightning-AI/litgpt/blob/main/litgpt/data/tinyllama.py) to pretrain LLMs. The statefulness still works when using a mixture of datasets with the `CombinedStreamingDataset`.

```python
import os
Expand Down
4 changes: 2 additions & 2 deletions src/pytorch_lightning/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ ______________________________________________________________________
<a href="#examples">Examples</a> •
<a href="#community">Community</a> •
<a href="https://lightning.ai/">Lightning AI</a> •
<a href="https://github.com/Lightning-AI/lightning/blob/master/LICENSE">License</a>
<a href="https://github.com/Lightning-AI/pytorch-lightning/blob/master/LICENSE">License</a>
</p>

<!-- DO NOT ADD CONDA DOWNLOADS... README CHANGES MUST BE APPROVED BY EDEN OR WILL -->
Expand All @@ -28,7 +28,7 @@ ______________________________________________________________________
[![codecov](https://codecov.io/gh/Lightning-AI/pytorch-lightning/graph/badge.svg?token=SmzX8mnKlA)](https://codecov.io/gh/Lightning-AI/pytorch-lightning)

[![ReadTheDocs](https://readthedocs.org/projects/pytorch-lightning/badge/?version=stable)](https://lightning.ai/docs/pytorch/stable/)[![Discord](https://img.shields.io/discord/1077906959069626439?style=plastic)](https://discord.gg/VptPCZkGNa)
[![license](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/Lightning-AI/lightning/blob/master/LICENSE)
[![license](https://img.shields.io/badge/License-Apache%202.0-blue.svg)](https://github.com/Lightning-AI/pytorch-lightning/blob/master/LICENSE)

<!--
[![CodeFactor](https://www.codefactor.io/repository/github/Lightning-AI/lightning/badge)](https://www.codefactor.io/repository/github/Lightning-AI/lightning)
Expand Down
4 changes: 2 additions & 2 deletions tests/README.md
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
# PyTorch-Lightning Tests

Most of the tests in PyTorch Lightning train a [BoringModel](https://github.com/Lightning-AI/lightning/blob/master/src/lightning/pytorch/demos/boring_classes.py) under various trainer conditions (ddp, amp, etc...). Want to add a new test case and not sure how? [Talk to us!](https://www.pytorchlightning.ai/community)
Most of the tests in PyTorch Lightning train a [BoringModel](https://github.com/Lightning-AI/pytorch-lightning/blob/master/src/lightning/pytorch/demos/boring_classes.py) under various trainer conditions (ddp, amp, etc...). Want to add a new test case and not sure how? [Talk to us!](https://www.pytorchlightning.ai/community)

## Running tests

Expand All @@ -26,7 +26,7 @@ Additionally, for testing backward compatibility with older versions of PyTorch
bash .actions/pull_legacy_checkpoints.sh
```

Note: These checkpoints are generated to set baselines for maintaining backward compatibility with legacy versions of PyTorch Lightning. Details of checkpoints for back-compatibility can be found [here](https://github.com/Lightning-AI/lightning/blob/master/tests/legacy/README.md).
Note: These checkpoints are generated to set baselines for maintaining backward compatibility with legacy versions of PyTorch Lightning. Details of checkpoints for back-compatibility can be found [here](https://github.com/Lightning-AI/pytorch-lightning/blob/master/tests/legacy/README.md).

You can run the full test suite in your terminal via this make script:

Expand Down