AgentNet

A lightweight library to build and train neural networks for reinforcement learning using Theano+Lasagne

Installation

Here's an installation guide

In short,

Manual install

1. Install bleeding edge theano/lasagne
2. [sudo] pip install --upgrade https://github.com/yandexdataschool/AgentNet/archive/master.zip

Docker container

1. install Docker,
2. make sure docker daemon is running (sudo service docker start)
3. make sure no application is using port 1234 (this is the default port that can be changed)
4. [sudo] docker run -d -p 1234:8888 justheuristic/agentnet
5. Access via localhost:1234 or whatever port you chose

Documentation and tutorials

AgentNet is using embedded documentation, so calling help(some_function_or_object) or pressing shift+tab in Ipython will yield description of what that thing is supposed to do.

A standard pipeline of AgentNet experiment can be found among examples

• Playing Atari SpaceInvaders with Convolutional NN via OpenAI Gym
  • Step-by-step explaination of what you need to do to recreate DeepMind Atari DQN
  • Written in a generic way, so that adding recurrent memory or changing learning algorithm could be done in a couple of lines
• Simple Deep Recurrent Reinforcement Learning setup
  • Most basic demo, if a bit boring. Covers the problem of learning "If X1 than Y1 Else Y2".
  • Only required if SpaceInvaders left you confused.

Demos

If you wish to get acquainted with the current library state, view some of the ./examples

• Playing Atari with Convolutional NN via OpenAI Gym
  • Can switch to any visual game thanks to their awesome interface
  • Very simplistic, non-recurrent suffering from atari flickering, etc.
• Deep Recurrent Kung-Fu training with GRUs and actor-critic
  • Uses the "Playing atari" example with minor changes
  • Trains via Advantage actor-critic (value+policy-based)
• Simple Deep Recurrent Reinforcement Learning setup
  • Trying to guess the interconnected hidden factors on a synthetic problem setup
• Stack-augmented GRU generator
  • Reproducing http://arxiv.org/abs/1503.01007 with less code
• MOAR deep recurrent value-based LR for wikipedia facts guessing
  • Trying to figure a policy on guessing musician attributes (genres, decades active, instruments, etc)
  • Using several hidden layers and 3-step Q-learning
• More to come

If you wish to join the development, we would be eager to accept your help. Current priority development anchors are maintained at the bottom of this readme.