Tutorials and BlogPosts

[Borda]

Blog posts

• From PyTorch to PyTorch Lightning — A gentle introduction
  PyTorch is extremely easy to use to build complex AI models. But once the research gets complicated and things like multi-GPU training, 16-bit precision and TPU training get mixed in, users are likely to introduce bugs.
  PyTorch Lightning solves exactly this problem. Lightning structures your PyTorch code so it can abstract the details of training. This makes AI research scalable and fast to iterate on.
• 36 Ways Pytorch Lightning Can Supercharge Your AI Research
• Pytorch Lightning vs PyTorch Ignite vs Fast.ai
  PyTorch-lightning is a recently released library which is a Kera-like ML library for PyTorch. It leaves core training and validation logic to you and automates the rest. (BTW, by Keras I mean no boilerplate, not overly-simplified).
• Trivial Multi-Node Training With Pytorch-Lightning
• Converting From Keras To PyTorch Lightning -
  In this tutorial, we’ll convert a Keras model into a PyTorch Lightning model to add another capability to your deep-learning ninja skills.
• Simplify And Scale Your PyTorch Code With PyTorch Lighting -
  PyTorch Lightning is a very lightweight wrapper on PyTorch which is more like a coding standard than a framework. If you refactor your PyTorch code into the Lightning format you get the bells and whistles of top research teams without all the work. The format allows you to get rid of a ton of boilerplate code and makes it highly reproducible.
• Torchbearer Joins PyTorch Lightning
• PyTorch Lightning 0.7.1 Release and Venture Funding -
  The 0.7.1 release signals a new level of framework maturity. With major API changes behind us, this release paves the way to the major 1.0 milestone we aim to reach this year.
• En-Lightning Reinforcement Learning -
  This article looks at using PyTorch Lightning for the exciting domain of Reinforcement Learning (RL). Here we are going to build a standard Deep Q Network (DQN) model using the classic CartPole gym environment to illustrate how to start using Lightning to build your RL models.
• Custom classifier on top of BERT-like Language Model - guide
• Training Neural Networks for Leela Zero With PyTorch -
  A simple training pipeline for Leela Zero implemented with PyTorch, PyTorch Lightning and Hydra
• Log PyTorchLightning metrics to Neptune
• Use Pytorch Lightning with Weights & Biases
• Using Optuna to Optimize PyTorch Lightning Hyperparameters -
  Optuna is a hyperparameter optimization framework applicable to machine learning frameworks and black-box optimization solvers. PyTorch Lightning provides a lightweight PyTorch wrapper for better scaling with less code. Combining the two of them allows for automatic tuning of hyperparameters to find the best performing models.
• How to Keep Track of PyTorch Lightning Experiments with Neptune
• How I built a Face Mask Detector for COVID-19 using PyTorch Lightning
• Getting Started with PyTorch Lightning
• PyTorch Multi-GPU Metrics Library and More in PyTorch Lightning 0.8.1 -
  Today we released 0.8.1 which is a major milestone for PyTorch Lightning. With incredible user adoption and growth, we’re continuing to build tools to easily do AI research.
• PyTorch Lightning: Metrics
  With PyTorch Lightning 0.8.1 we added a feature that has been requested many times by our community: Metrics. This feature is designed to be used with PyTorch Lightning as well as with any other PyTorch based code. In this blog post, we’ll explain what Metrics is and how you can get started.
• fastai and PyTorch Lightning are democratising AI
• How To Tag Any Image Using Deep Learning
• Distributed Deep Learning with Ansible, AWS and Pytorch Lightning. Part 1
• Use Pytorch Lightning to Decouple Science science code from engineering code
• Productive NLP Experimentation with Python using Pytorch Lightning and Torchtext
• Video Prediction using Deep Learning and PyTorch-lightning
  In this guide, I will show you how to code a Convolutional Long Short-Term Memory (ConvLSTM) using an autoencoder (seq2seq) architecture for frame prediction using the MovingMNIST dataset (but custom datasets can also easily be integrated).
• Automate Your Neural Network Training With PyTorch Lightning
  PyTorch Lightning will automate your neural network training while staying your code simple, clean, and flexible. If you’re a researcher you will love this!
• TensorBoard with PyTorch Lightning
• Effective testing for machine learning systems
  Working as a core maintainer for PyTorch Lightning, I've grown a strong appreciation for the value of tests in software development. As I've been spinning up a new project at work, I've been spending a fair amount of time thinking about how we should test machine learning systems.
• Keeping Up with PyTorch Lightning and Hydra
  Last week, PyTorch Lightning 0.9.0 and Hydra’s fourth release candidate for 1.0.0 were released with a choke-full of new features and mostly final APIs. I thought it’d be a good time for me to revisit my side project Leela Zero PyTorch to see how these new versions can be integrated into it. In this post, I’ll talk about some of the new features of the two libraries, and how they helped Leela Zero PyTorch.
• EfficientDet Meets Pytorch-Lightning
  Welcome to this beginner friendly guide to object detection using EfficientDet. Similarly to what I have done in the NLP guide (check it here if you haven’t yet already), there will be a mix of theory, practice, and an application to the global wheat competition dataset.
• Automatically finding good learning rate for your network with PyTorch Lightning
  Among of all hyperparameters used in machine learning, learning rate is probably the very first one you hear about. It may also the one that you start tuning in the first place. You can find the right value with a bit of hyper parameter optimization, running tons of training sessions or you can let tools do it, much faster. Nowadays, many libraries implement LR Finder or “LR Range Test”.
• 3 Simple Tricks That Will Change the Way You Debug PyTorch
  Every Deep Learning project is different. No matter how much experience you bring with you, there will always be new challenges and unexpected behavior you will struggle with. The skill- and mindset that you bring to the project will determine how quickly you discover and adapt to the obstacles that stand in the way of success.