DeepLearning Excellent Open Course

[Bryce1010]

个人主页，个人学习生涯！

学习流程：

第一遍， 通读全文，了解内容

第二遍，针对性阅读，并记录心得

第三遍，理论结合实践

一点一点搬运到博客上

ref

• Deep learning papers reading roadmap

• Data science roadmap

• youtube-channels-for-deep-learning-and-computer-vision

• AI-CheetSheet

• DeelLearningTutorial AI完备路线

• AI学习路线 from 机器之心

Foreword:

• 已完成

• 已完成完成但需要多遍复习

• 还在计划学习

---

5. 深度学习

博客

• Videos & Images Theory and Analytics Laboratory (VITAL) of Sherbrooke University[laboratory] [blog]

  非常好的公共资源，有他们组的weekly-reading group& weekly machine learning videos.

• Lil’Log [blog]

• Berkeley Artificial Intelligence Research [Bair]

• Jay Alammar(NLP 大佬) [blog]

• Createmomo [blog]

  包含许多深度学习的基础

• Jonathan Hui blog [blog]

  关于深度学习，很久不更新了

• daiwk’s blog [blog]

  覆盖深度学习各方面

• AI Learn Notes [github]

• telesens.co[blog]

• 研究资源列表 A curated list of research resources [github]

• Yoshua Bengio [blog]

[Bryce1010]

公开课与资料

• 图像处理 Image and Video Processing: From Mars to Hollywood with a Stop at the Hospital [[couresa Duke]](Image and Video Processing: From Mars to Hollywood with a Stop at the Hospital)
• Deep Bayes 2019[url]
• DeepLearning.ai C4W1L01[url]
• 数学推荐[url]
• 2019 CIFAR DLRL Summer School Lectures & ZBengio深度学习&强化学习暑期班课程[url]
• Neral Networks: Tricks of the Trade[url]
• 迁移学习简明手册[url]
• 机器学习(Machine Learning)、深度学习(Deep Learning)、NLP面试中常考到的知识点和代码实现[url]
• PyData NYC python 和 Machine Learning[url]
• Hacker Tools程序员超实用工具课程[url]
• Cousera[url]
• The Age of A.I. （AI纪录片）[url]
• Learn TensorFlow in 3 Hours [url]
• CVSS 2019 Computational Vision Summer School [url]
• NIPS 2019 Hightlight papers [url]
  • Advanced AI Deep Reinforcement Learning in Python [url]
• 编程随想电子书 目录 [github]
• fun with graphics 图像学相关课程、书籍、教程[github]
• Deep Learning Papers Reading Roadmap [github]
• CMU2020深度学习进阶 [url]
• 【公开课】旷视x北大《深度学习实践》（28课时全) [bilibili]
• Introducing to Parallel Programming [udacity cs344] [nvidia]
• 纽约大学《深度学习(PyTorch)》课程(2020) by yann Lecun, Alfredo Canziani [bilibili]
• 机器学习进阶列表公开课 [reddit]
• 【2020伯克利】深度无监督学习 [bilibili]

书籍

书籍

• https://github.com/loveunk/Deep-learning-books

[Bryce1010]

顶会与期刊

• AI Conference Deadlines [url]

• 2019ICCV [all papers] [github]

• 2019CVPR [all papers] [github]

• 2020CVPR [github]

这一部分是DeepLearning的代码模板，主要任务

1. 收集deeplearning各方向的优秀代码
2. PyTorch, TensorFlow, Keras等框架的训练技巧
3. 优秀的知识点总结和review

[Bryce1010]

Python

• Python练习册，每天一个小程序 [github]

• python-cheatsheet [github]

• 自学Python——编程基础、科学计算及数据分析-中文python笔记 [github]

• What the fuck python [github]

• GitHub标星8100：Python中文资源，从新手到老司机，只要100天: [github]

• Python 进阶-Real Python tutorials [url]

• 《Intermediate Python》的中译版本 [doc]

• Python Data Science Handbook [book]

• Python有趣的小例子 [url]

• Pandas [url]

• Data-Science–Cheat-Sheet [github]

• 分享8点超级有用的Python编程建议 [weixin]

• 【个人项目】——Learning DataScience [github]

  • 存储DataScience的代码模板；
  • 存储DataScience的资料；

• python 文件处理 [zhihu]

• python magic method [url]

[Bryce1010]

PyTorch

• Pytorch 进阶
• Pytorch分布式训练 [zhihu]
• Apachecn:PyTorch 中文文档 & 教程
• pytorch入门与实践推荐
• pytorch-tutorial
• Awesome pytorch
• pytorch 模型训练实用教程推荐
• Pytorch 常用代码段整理
• 国外视频教程：另外一个国外大佬的视频教程，在YouTube上有很高的点击率: https://www.youtube.com/playlist?list=PLlMkM4tgfjnJ3I-dbhO9JTw7gNty6o_2m
• Awesome-Pytorch-Chinese
• Deep Learning Models
• DL_RL_with_pytorch强化学习
• Pytorch Hand book
• PyTorch Ignite 高级API [url]

TensorboardX

• tensorboardX github
• tensorboardX 官方教程
• PyTorch模型训练特征图可视化（TensorboardX）

Tensorflow

• TensorFlow 2.0官方教程 [url]

  Hands-on ML第二版[github]

• Data-Science-Study-Paths-March-2019 [github]

• GitHub：TensorFlow 最全资料集锦 [url

• tensorflow 中文社区 [url]

• tensorflow2.0 入门教程[zhihu]

• mofan tensorflow [url]

• Hands-on TensorFlow 2.0 from IJCAI19 [IJCAI]

Fast.ai

• Fast.ai深度学习实战课程(2019)学习笔记 [weibo]

OpenCV-Python

• OpenCV-Python图像处理源码及教程 [github]

• Youtube: OpenCV Python Tutorial For Beginners

• OpenCV Document [url]

[Bryce1010]

Writing & Reading

• Latex 笔记模板 [ref]

• Youtube: LaTex Tutorial [videos]

• How to read a paper [paper]

• Writing Technical Articles [ref]

• Research Skills [ref]

• Academic Writing [中文笔记]

5.1 Basic Network

1. Attention

• What is Attention?

• On the Relationship between Self-Attention and Convolutional Layers [paper] [pytorch]

  • Provides evidence that attention layers can perform convolution.
  • multi-head self-attention layer with sufficient number of heads is at least as powerful as any convolutional layer

• An overview of all the three metods CAM+Grad-CAM+Grad-CAM++[overview]

• Grad-CAM [overview][gradcam++ pytorch][gradcam pytorch] [3D grad-cam (theano) ]

• Grad-CAM++[[pytorch]](https://github.com/adityac94/Grad_CAM_plus_plus
  https://github.com/1Konny/gradcam_plus_plus-pytorch)

• (2020 Data Augmentation) GridMask Data Augmentation

2. Capsule Nets

• A curated list of awesome resources related to capsule networks[github]

• Understanding Hinton’s Capsule Networks. Part I,II,III,IV[Medium]

• Understanding Capsule Networks[Medium]

• CVPR2019 Capsule Network [video] [slides]

3. Loss Function

• Loss Function && Machine Learning Metrics 总结 [blog]

• Optimization Algorithms for Deep Learning[Medium]

• An overview of gradient descent optimization algorithms[url]

• On Optimization Methods for Deep Learning[ICML11]

• An Overview of gradient descent optimization algorithms [Arxiv]

• (2015 ICLR) ADAM: A METHOD FOR STOCHASTIC OPTIMIZATION [paper]

• 机器学习中的评价指标

• Understanding Optimizers

4. Classification Networks

• awesome image classification[url]

• Classification 综述整理 [bryce1010]

• 深入理解反向传播[url]

• 如何调参[url]

• (2019 ICML )EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks [paper]

• (2019 ICCV )Searching for MobileNetV3 [paper]

• (2018 CVPR )Squeeze-and-Excitation Networks(SENet) [paper][code][review]

• (2018 CVPR )Learning Transferable Architectures for Scalable Image Recognition(NASNet) [paper]

• (2017 CVPR )Aggregated Residual Transformations for Deep Neural Networks (ResNext) [paper]

• (2020 ICLR )Deforming kernels to adapt towards object deformation [github]

• pytorch-CIFAR10

torchvision datasets+densenet+dpn+efficientnet+googlenet+lenet+mobilenet+mobilenetv2

+pnasnet+preact_resnet+resnet+resnext+senet+shufflenet+shufflenetv2+vgg

7. Crowd Counting

• Awesome Crowd Counting[github]

• Crowd_counting_from_scratch[github]

• multi-scale convolutional neural network (CNN) for crowd counting[Arxiv2017]

• Counting Crowds and Lines with AI[url]

• datasets[github]

8. Domain Adaptation

• Awesome doamin adaptation[github]

9. Knowledge Distillation

• knowledge distillation papers[github]

• Distilling the Knowledge in a Neural Network[Medium]

• (实践)Teacher-Student Architecture in Plant Disease Classification [medium]

10. Transfer Learning

• 迁移学习简明手册 [github]

11. Person-Re-Identification

• Awesome -Person-Re-Identification [github]

[Bryce1010]

5.2 Medical Image Analysis

• State-of-the-art medical image segmentation methods based on various challenges[github]

• 2017 data science bowl dicom图像肺癌检测 [url]

• Medical-Classification-APTOS2019_Winner_Review

[Bryce1010]

5.3 Generative Adversarial Network

• SIGAI:生成式对抗网络模型综述[zhihu]

• 深入浅出：GAN原理与应用入门介绍[zhihu]

• github:YadiraF-GAN[github]
  Resources and Implementations of Generative Adversarial Nets: GAN, DCGAN, WGAN, CGAN, InfoGAN

• github:hindupuravinash-the-gan-zoo[github]
  A list of all named GANs!

• github: Zhangqianhui-AdversarialNetsPapers[github]
  The classical paper list with code about generative adversarial nets

• GAN万字长文综述（中文·下载）[weixin]

• Introduction to Turing Leanring and GANs (nice)[Medium]

• How to measure GAN performance?[Medium]

• Understanding Generative Adversarial Networks (GANs)[Medium]

• PyTorch implementations of Generative Adversarial Networks.[github]

• 视频教程

  • 台湾大学 李宏毅GAN教程
    https://www.bilibili.com/video/av24011528/
  • CS236: Deep Generative Models
    https://github.com/SKKSaikia/CS236_DGM

• (2020)A Review on Generative Adversarial Networks: Algorithms, Theory, and Applications [paper]

• VAE 教程

  • The Variational Autoencoder as a Two-Player Game PartI II III[Medium]

• Intuitively Understanding Variational Autoencoders[Medium]

  • VAE Careful Walkthrough[Medium]
  • From Autoencoder to Beta-VAE[LILI github]
  • Review on online VAE[[Review on github]]([https://grigorisg9gr.github.io/machine%20learning/neural%20networks/vae-review/](https://grigorisg9gr.github.io/machine learning/neural networks/vae-review/))
  • Understanding Variational Autoencoders(VAEs)[Medium]
  • (2019) An Introduction to Variational Autoencoders [paper]

• (CVPR2019 StyleGAN v1)A Style-Based Generator Architecture for Generative Adversarial Networks (StyleGAN) [paper]

• (arXiv2019 StyleGAN v2)Analyzing and Improving the Image Quality of StyleGAN [paper]

• (ArXiv 2019 + overview) An Introduciton to Variational Autoencoders [paper]

• [StarGAN v2] StarGAN v2: Diverse Image Synthesis for Multiple Domains [paper] [code]

• (ICLR 2020)Overcoming the Disentanglement vs Reconstruction Trade-off via Jacobian Supervision [paper] [github]

• (ICLR2020) RealnessGAN [github]

  it enables the basic DCGAN architecture to generate realistic images at 10241024 resolution when trained from scratch

[Bryce1010]

5.4 OCR

• [TPAMI 2016] (CRNN) An end-to-end trainable neural network for image-based sequence recognition and its application to scene text recognition [paper] [crnn torch7/PyTorch]
• 使用TensorFlow Attention API进行OCR文本图像提取 [blog]
• OCR 文本检测综述
• awesome-deep-text-detection-recognition [github]
• OCR project [github]

[Bryce1010]

5.5 Object Detection

• 目标检测综述 [blog]

• [ICLR 2020]Computation Reallocation for Object Detection [paper]

  • 对目标检测框架，学习针对不同特征尺度和像素空间位置的计算重分配策略

• (2019 NIPS)DetNAS: Backbone Search for Object Detection [paper]

• (2019 ICCV)CenterNet: Keypoint Triplets for Object Detection [paper]

• (2019 ICCV) ThunderNet: Towards Real-time Generic Object Detection on Mobile Devices

• (2019 ICCV Attention) GCNet: Non-local Networks Meet Squeeze-Excitation Networks and Beyond [github]

  Non-Local Network + SENet

• (2019 CVPR)NAS-FPN: Learning Scalable Feature Pyramid Architecture for Object Detection [paper]

• (2019 CVPR) HRNet: Deep High-Resolution Representation Learning for Human Pose Estimation [paper]

  keypoint detection

• (CVPR 2019)Hybrid Task Cascade for Instance Segmentation [paper] [mmdetection]

• (ECCV 2018) CornerNet: Detecting Objects as Paired Keypoints [paper]

• (CVPR2018) Cascade R-CNN: Delving Into High Quality Object Detection[paper]

• (2018)YOLOv3: An Incremental Improvement [paper]

• (ICLR2018)Mixed Precision Training [paper] [论文笔记]

• (ICCV 2017)Mask R-CNN [paper]

• (ICCV 2017) soft-nms : Improving Object Detection With One Line of Code [paper] [论文笔记]

• (ICCV 2017) Feature Pyramid Networks for Object Detection(FPN) [paper]

• (ICCV 2017 Best Student Paper) Focal Loss for Dense Object Detection (RetinaNet) [paper]

• (ECCV 2016) SSD: Single Shot MultiBox Detector [paper]

• (CVPR2016) OHEM : Training Region-based Object Detectors with Online Hard Example Mining [paper] [论文笔记]

• (NIPS 2015) Faster R-CNN: Towards Real-Time Object Detection with Region Proposal Networks [paper] [pytorch]

• Detectron项目[url]

• 目标检测中不平衡样本综述 [github]

• MMDetection [url]

  RPN + Faster-RCNN + Mask-RCNN + Fast-RCNN + RetinaNet + Cascade R-CNN

  +Cascade Mask-RCNN + Hybrid Task Cascade + SSD + Group Normalization +

  Weight Standardization + Deformable Convolution v2 + Instaboost + Libra R-CNN +

  Guided Anchoring + FCOS + FoveaBox + RepPoints + FreeAnchors + Grid R-CNN(plus)+

  GHM + GCNet + Mask Scoring RCNN + Train from Scratch + NAS-FPN + ATSS +

• Imbalance Problems in Object Detection [[github]]( Imbalance Problems in Object Detection)

[Bryce1010]

5.6 Video Analysis

• (CVPR 2018) Non-local Neural Networks [paper][code]

• TSM: Temporal Shift Module for Efficient Video Understanding [paper]

• [ArXiv 2019] Depth-Aware Video Frame Interpolation [paper]

• (Skill) PyTorch : Video Dataset (Pytorch载入视频数据的dataset.py) [repo]

• (Skill) Getting started with Videos from OpenCV [url]

• PySlowFast is an open source video understanding codebase from FAIR that provides state-of-the-art video classification models [github]

[Bryce1010]

5.7 Segmentation

• DeepLearning Segmentation 总结 [博客]

• github: awesome semantic segmentation[github]

• Resources of semantic segmantation based on Deep Learning model[github]

• 2019语义分割综述[weixin]

• PointRend: Image Segmentation as Rendering [paper][review]

  把语义分割问题以及实例分割问题当做一个图像渲染问题来解决。本质是一个新型的上采样方法，针对物体边缘的分割进行优化，使得其在难以分割的物体边缘部分有更好的表现。

• PyTorch for Semantic Segmentation [github]

[Bryce1010]

5.8 Graphic Neural Network

• awesome graph classification[github]
• Must-read papers and continuous track on Graph Neural Network[github]
• 【图神经网络：模型与应用】《Graph Neural Networks: Models and Applications(AAAI 2020 Tutorial)》 [ppt]

5.9 Auto MachineLearning

• Survey on Automated Machine Learning:[paper]

• Auto Machine Learning 学习笔记[weixin]

• AutoML: A Sruvey of the State-of-the-Art[paper]

• Attention[paper]

• Everything you need to know about AutoML and Neural Architecture Search[Medium]

• LITERATURE ON NEURAL ARCHITECTURE SEARCH[url]

• Awesome NAS[github]

• Awesome ML Model Compression[github]

• awesome-AutoML-and-Lightweight-Models[github]

• [ArXiv 2019] Network Pruning via Transformable Architecture Search [paper]

5.10 Meta Learning

• Learning to Learn [berkeley bari]

• Stanford Deep Multi-Task and Meta Learning[weixin]

5.11 3D Object

• Review: deep learning on 3D point clouds [paper]
• PyTorch3D is FAIR's library of reusable components for deep learning with 3D data [github]

[Bryce1010]

5.12 Unsupervised Learning

• 伯克利课程：深度无监督学习(2020) [course homepage]

5.13 Few-shot Learning

• Awesome few-shot from Duan[url]

• Awesome few-shot from bryce1010[url]

• Few-shot Classification 综述 [bryce1010]

• Few-shot Learning detection综述 [bryce1010]

• (ArXiv 2020)Continual Local Replacement for Few-shot Image Recognition [paper] [github]

[Bryce1010]

6. 强化学习

• 英文教程：A Free course in Deep Reinforcemet Learning form beginner to expert[github]
• 综述论文140页：An Introduction to Deep Reinforcement Learning[paper]
• 增强学习笔记[zhihu]
• Reinforcement Learning: An Introduction (Richard S. Sutton and Andrew G. Barto) (http://incompleteideas.net/book/the-book-2nd.html)
• Reinforcement Learning - David Silver[B站地址(中文字幕)] [youtube] [课程PPT] [课程笔记]
• 李宏毅 《深度强化学习》 [B站地址(中文字幕)] [课程原地址] [课程PP] [课程笔记]
• A(Long) Peek into Reinforcement Learning [LiLi github]
• Deep Reinforcement Learning Algorithms with PyTorch [github]
• Deep Reinforcement LAB [github]
• 强化学习开源资源大列表：框架、环境和数据(任务) [google sheets]
• 【deeppavlov.ai的深度强化学习进阶课程】“Course: Advanced Topics in Deep Reinforcement learning | deeppavlov.ai [url]
• (2017) Deep Reinforcement Learning: An Overview [paper]

[Bryce1010]

7. 自然语言处理（NLP）

• 李纪为-出入NLP领域的一些小建议[zhihu]

• 斯坦福2019深度学习NLP课程 视频+PPT+参考资料+优秀项目 [zhihu]

• Overview of Modern Deep Learning Techniques Applied to Natural Language Processing[github]

• 深度学习之自然语言处理斯坦福大学CS224n课程集训营[github]

• 邱锡鹏： 五个练习上手深度学习 [Medium]

• CS224n 斯坦福深度自然语言处理课 [2019版课程主页] [B站英文字幕版]

• CS224n 2019一周通关攻略 [zhihu]

• 自然语言NLP基本概念大全 [blog]

• NLP 中文综述整理[blog]

• NLP Paper Summary [github]

• [XLNet] XLNet: Generalized Autoregressive Pretraining for Language Understanding [paper] [code]

• Understanding BERT [medium]

• Pytorch Transformers [github]

  BERT, GPT, GPT-2, Transformer-XL, XLNet, et.

• Transformers: State-of-the-art Natural Language Processing for TensorFlow 2.0 and PyTorch [github]

• Papers & presentations from Hugging Face's weekly science day [github]

• an Open Course Platform for Stanford CS224n (2020 Winter) [github]

• Google QUEST Q&A Labeling 1st place solution with code [kaggle]