General

General

• Net2Net
• Can Active Memory Replace Attention?
• Extensions and Limitations of the Neural GPU
• Evolution Strategies
• Deep Forest: Towards An Alternative to Deep Neural Networks
• A Regularized Framework for Sparse and Structured Neural Attention
• Self-Normalizing Neural Networks
• FreezeOut: Accelerate Training by Progressively Freezing Layers
• Super-Convergence: Very Fast Training of Residual Networks Using Large Learning Rates
• Memory-based Parameter Adaptation
• Meta-Learning via Feature-Label Memory Network
• Adversarial Contrastive Estimation

Dropout

• Variational Dropout and the Local Reparameterization Trick
• Dropout as a Bayesian Approximation: Representing Model Uncertainty in Deep Learning
• Concrete Dropout

Variational / Bayesian

• Improving Variational Auto-Encodersusing Householder Flow
• Multiplicative Normalizing Flows for Variational Bayesian Neural Networks
• Variational Attention for Sequence-to-Sequence Models

Recurrent Neural Networks

• Semi-Supervised Learning with Ladder Networks

• On Multiplicative Integration with Recurrent Neural Networks

• Recurrent Highway Networks

• HyperNetworks

• Using Fast Weights to Attend to the Recent Past

• Recurrent Weighted Average

• Recurrent Additive Networks

• Efficiently applying attention to sequential data with the Recurrent Discounted Attention unit

• Gated Recurrent Neural Tensor Network

• Training RNNs as Fast as CNNs

• Recurrent Residual Attention

• Recurrent Discounted Attention unit

• Lattice Recurrent Unit: Improving Convergence and Statistical Efficiency for Sequence Modeling

• Deep Echo State Network (DeepESN): A Brief Survey