Update readme

Author: MartinBraquet

README.md

@@ -43,13 +43,54 @@ If some dependencies release changes that break the code, you can install the pr
 pip install -r requirements.txt
 ```
 
+## Background
+
+> [!NOTE]  
+> This section is highly technical; feel free to skip it and play with the code right away.
+
+#### Agents
+
+Most computer algorithms discretize the game into states and actions. Here, the state is the position of the pawns and the available actions are the possible moves of the pawns.
+
+Squadro is a finite state machine, meaning that the next state of the game is completely determined by the current state and the action played. With this definition, one can see that the game is a Markov Decision Process (MDP). At each state, the current player can play different actions, which lead to different states. Then the next player can play different actions from any of those new states, etc. The future of the game can be represented as a tree, whose branches are the actions that lead to different states.
+
+An algorithm can explore that space of possibilities to infer the best move to play now. As the tree is huge, it is not possible to explore all the possible paths until the end of the game. Typically, they explore only a small fraction of the tree and then use the information gathered from those states to make a decision. More precisely, those two phases are:
+
+* **State exploration**: exploring the space of states by a careful choice of actions. The most common exploration methods are Minimax and Monte Carlo Tree Search (MCTS). Minimax explores all the states up to a specific depth, while MCTS navigates until it finds a state that has not been visited yet. Minimax can be sped up by skipping the search in the parts of the tree that won't affect the final decision; this method is called alpha-beta pruning.
+* **State evaluation**: evaluating a state. If we have a basic understanding of the game and how to win, one can design a heuristic (state evaluation function) that gives an estimate of how good it is to be in that state / position. Otherwise, it can often be better to use a computer algorithm to evaluate the state.
+  * The simplest algorithm to estimate the state is to randomly let the game play until it is over (i.e., pick random actions for both players). When played enough times, it can give the probability to win in that state.
+  * More complex, and hence accurate, algorithms are using reinforcement learning (AI). They learn from experience by storing information about each state/action in one of:
+    * Q value function, a lookup table for each state and action;
+    * deep Q network (DQN), a neural network that approximates the Q value function, which is necessary when the state space is huge (i.e., cannot be stored in memory).
+
+List of available agents:
+
+* _human_: another local human player (i.e., both playing on the same computer)
+* _random_: a computer that plays randomly among all available moves
+* _ab_relative_advancement_: a computer that lists the possible moves from the current position and evaluates them directly (i.e., it "thinks" only one move ahead), where the evaluation function is the player's advancement
+* _relative_advancement_: a computer that lists the possible moves from the current position and evaluates them directly (i.e., it "thinks" only one move ahead), where the evaluation function is the player's advancement compared to the other player
+* _ab_relative_advancement_: a computer that plays minimax with alpha-beta pruning (depth ~4), where the evaluation function is the player's advancement compared to the other player
+* _mcts_advancement_: Monte Carlo tree search, where the evaluation function is the player's advancement compared to the other player
+* _mcts_rollout_: Monte Carlo tree search, where the evaluation function is determined by a random playout until the end of the game
+* _mcts_q_learning_: Monte Carlo tree search, where the evaluation function is determined by a lookup table
+* _mcts_deep_q_learning_: Monte Carlo tree search, where the evaluation function is determined by a convolutional neural network 
+
+You can also access the most updated list of available agents with:
+
+```python
+import squadro
+
+print(squadro.AVAILABLE_AGENTS)
+```
+
+
 ## Usage
 
-This package can be used in the following ways:
+This package can be used in many interesting ways. You can play the game, train an AI agent, run simulations, and analyze animations.
 
 ### Play
 
-You can play against someone else or many different types of computer algorithms. See the [Agents](#Agents) section below for more details.
+You can play against someone else or many different types of computer algorithms.
 
 > [!TIP]
 > If you run into the following error on a Linux machine when launching the game:
@@ -127,43 +168,6 @@ agent_dql = MonteCarloDeepQLearningAgent()  # Q-Learning
 
 ```
 
-#### Agents
-
-Most computer algorithms discretize the game into states and actions. Here, the state is the position of the pawns and the available actions are the possible moves of the pawns.
-
-Squadro is a finite state machine, meaning that the next state of the game is completely determined by the current state and the action played. With this definition, one can see that the game is a Markov Decision Process (MDP). At each state, the current player can play different actions, which lead to different states. Then the next player can play different actions from any of those new states, etc. The future of the game can be represented as a tree, whose branches are the actions that lead to different states.
-
-An algorithm can explore that space of possibilities to infer the best move to play now. As the tree is huge, it is not possible to explore all the possible paths until the end of the game. Typically, they explore only a small fraction of the tree and then use the information gathered from those states to make a decision. More precisely, those two phases are:
-
-* **State exploration**: exploring the space of states by a careful choice of actions. The most common exploration methods are Minimax and Monte Carlo Tree Search (MCTS). Minimax explores all the states up to a specific depth, while MCTS navigates until it finds a state that has not been visited yet. Minimax can be sped up by skipping the search in the parts of the tree that won't affect the final decision; this method is called alpha-beta pruning.
-* **State evaluation**: evaluating a state. If we have a basic understanding of the game and how to win, one can design a heuristic (state evaluation function) that gives an estimate of how good it is to be in that state / position. Otherwise, it can often be better to use a computer algorithm to evaluate the state.
-  * The simplest algorithm to estimate the state is to randomly let the game play until it is over (i.e., pick random actions for both players). When played enough times, it can give the probability to win in that state.
-  * More complex, and hence accurate, algorithms are using reinforcement learning (AI). They learn from experience by storing information about each state/action in one of:
-    * Q value function, a lookup table for each state and action;
-    * deep Q network (DQN), a neural network that approximates the Q value function, which is necessary when the state space is huge (i.e., cannot be stored in memory).
-
-List of available agents:
-
-* _human_: another local human player (i.e., both playing on the same computer)
-* _random_: a computer that plays randomly among all available moves
-* _ab_relative_advancement_: a computer that lists the possible moves from the current position and evaluates them directly (i.e., it "thinks" only one move ahead), where the evaluation function is the player's advancement
-* _relative_advancement_: a computer that lists the possible moves from the current position and evaluates them directly (i.e., it "thinks" only one move ahead), where the evaluation function is the player's advancement compared to the other player
-* _ab_relative_advancement_: a computer that plays minimax with alpha-beta pruning (depth ~4), where the evaluation function is the player's advancement compared to the other player
-* _mcts_advancement_: Monte Carlo tree search, where the evaluation function is the player's advancement compared to the other player
-* _mcts_rollout_: Monte Carlo tree search, where the evaluation function is determined by a random playout until the end of the game
-* _mcts_q_learning_: Monte Carlo tree search, where the evaluation function is determined by a lookup table
-* _mcts_deep_q_learning_: Monte Carlo tree search, where the evaluation function is determined by a convolutional neural network 
-
-
-You can also access the most updated list of available agents with:
-
-```python
-import squadro
-
-print(squadro.AVAILABLE_AGENTS)
-```
-
-
 ### Training
 
 One can train a model using reinforcement learning (RL) algorithms. Currently, Squadro supports two such algorithms: