Several, small documentation improvements (#3903)

* Several, small documentation improvements

- Re-organize main repo README
- Minor clean-ups to Python package-specific readme files
- Clean-up to Unity Inference Engine page
- Update to the docs README
- Added a specific cross-platform section in ML-Agents Overview to amplify Barracuda
- Updated the links in Limitations.md to point to the specific subsections
- Cleaned up the Designing a Learning Environment page. Added an intro paragraph.
- Updated the installation guide to specifically call out local installation
- A few minor formatting, spelling errors fixed.

Author: Marwan Mattar

README.md

@@ -3,6 +3,7 @@
 <img src="docs/images/image-banner.png" align="middle" width="3000"/>
 
 # Unity ML-Agents Toolkit (Beta)
+
 [![docs badge](https://img.shields.io/badge/docs-reference-blue.svg)](https://github.com/Unity-Technologies/ml-agents/tree/release_1_docs/docs/)
 
 [![license badge](https://img.shields.io/badge/license-Apache--2.0-green.svg)](LICENSE)
@@ -27,20 +28,25 @@ developer communities.
 
 ## Features
 
-- Unity environment control from Python
-- 15+ sample Unity environments
-- Two deep reinforcement learning algorithms, Proximal Policy Optimization (PPO)
-  and Soft Actor-Critic (SAC)
+- 15+ [example Unity environments](docs/Learning-Environment-Examples.md)
 - Support for multiple environment configurations and training scenarios
-- Self-play mechanism for training agents in adversarial scenarios
-- Train memory-enhanced agents using deep reinforcement learning
-- Easily definable Curriculum Learning and Generalization scenarios
+- Flexible Unity SDK that can be integrated into your game or custom Unity scene
+- Training using two deep reinforcement learning algorithms, Proximal Policy
+  Optimization (PPO) and Soft Actor-Critic (SAC)
 - Built-in support for Imitation Learning through Behavioral Cloning or
   Generative Adversarial Imitation Learning
+- Self-play mechanism for training agents in adversarial scenarios
+- Easily definable Curriculum Learning scenarios for complex tasks
+- Train robust agents using environment randomization
 - Flexible agent control with On Demand Decision Making
-- Wrap learning environments as a gym
-- Utilizes the Unity Inference Engine
-- Train using concurrent Unity environment instances
+- Train using multiple concurrent Unity environment instances
+- Utilizes the [Unity Inference Engine](docs/Unity-Inference-Engine.md) to
+  provide native cross-platform support
+- Unity environment [control from Python](docs/Python-API.md)
+- Wrap Unity learning environments as a [gym](gym-unity/README.md)
+
+See our [ML-Agents Overview](docs/ML-Agents-Overview.md) page for detailed
+descriptions of all these features.
 
 ## Releases & Documentation
 

com.unity.ml-agents/Documentation~/com.unity.ml-agents.md

@@ -1,79 +1,88 @@
 # About ML-Agents package (`com.unity.ml-agents`)
 
-The Unity ML-Agents package contains the C# SDK for the [Unity ML-Agents Toolkit].
-
-The package allows you to convert any Unity scene to into a learning
-environment and train character behaviors using a variety of machine learning
-algorithms. Additionally, it allows you to embed these trained behaviors back into
-Unity scenes to control your characters. More specifically, the package provides
- the following core functionalities:
-
-* Define Agents: entities, or characters, whose behavior will be learned. Agents are entities
-  that generate observations (through sensors), take actions, and receive rewards from
-  the environment.
-* Define Behaviors: entities that specifiy how an agent should act. Multiple agents can
-  share the same Behavior and a scene may have multiple Behaviors.
-* Record demonstrations of an agent within the Editor. You can use demonstrations
-  to help train a behavior for that agent.
-* Embedding a trained behavior into the scene via the [Unity Inference Engine].
-  Embedded behaviors allow you to switch an Agent between learning and inference.
-
-Note that the *ML-Agents* package does not contain the machine learning algorithms for training
-behaviors. The *ML-Agents* package only supports instrumenting a Unity scene, setting it up for
-training, and then embedding the trained model back into your Unity scene. The machine learning
-algorithms that orchestrate training are part of the companion [Python package].
-
+The Unity ML-Agents package contains the C# SDK for the [Unity ML-Agents
+Toolkit].
+
+The package allows you to convert any Unity scene to into a learning environment
+and train character behaviors using a variety of machine learning algorithms.
+Additionally, it allows you to embed these trained behaviors back into Unity
+scenes to control your characters. More specifically, the package provides the
+following core functionalities:
+
+- Define Agents: entities, or characters, whose behavior will be learned. Agents
+  are entities that generate observations (through sensors), take actions, and
+  receive rewards from the environment.
+- Define Behaviors: entities that specifiy how an agent should act. Multiple
+  agents can share the same Behavior and a scene may have multiple Behaviors.
+- Record demonstrations of an agent within the Editor. You can use
+  demonstrations to help train a behavior for that agent.
+- Embedding a trained behavior into the scene via the [Unity Inference Engine].
+  Embedded behaviors allow you to switch an Agent between learning and
+  inference.
+
+Note that the _ML-Agents_ package does not contain the machine learning
+algorithms for training behaviors. The _ML-Agents_ package only supports
+instrumenting a Unity scene, setting it up for training, and then embedding the
+trained model back into your Unity scene. The machine learning algorithms that
+orchestrate training are part of the companion [Python package].
 
 ## Package contents
 
 The following table describes the package folder structure:
 
-|**Location**|**Description**|
-|---|---|
-|*Documentation~*|Contains the documentation for the Unity package.|
-|*Editor*|Contains utilities for Editor windows and drawers.|
-|*Plugins*|Contains third-party DLLs.|
-|*Runtime*|Contains core C# APIs for integrating ML-Agents into your Unity scene. |
-|*Tests*|Contains the unit tests for the package.|
-
+| **Location**     | **Description**                                                        |
+| ---------------- | ---------------------------------------------------------------------- |
+| _Documentation~_ | Contains the documentation for the Unity package.                      |
+| _Editor_         | Contains utilities for Editor windows and drawers.                     |
+| _Plugins_        | Contains third-party DLLs.                                             |
+| _Runtime_        | Contains core C# APIs for integrating ML-Agents into your Unity scene. |
+| _Tests_          | Contains the unit tests for the package.                               |
 
 <a name="Installation"></a>
+
 ## Installation
 
-To install this *ML-Agents* package, follow the instructions in the [Package Manager documentation].
+To install this _ML-Agents_ package, follow the instructions in the [Package
+Manager documentation].
 
 To install the companion Python package to enable training behaviors, follow the
 [installation instructions] on our [GitHub repository].
 
-
 ## Requirements
 
-This version of the Unity ML-Agents package is compatible with the following versions of the
-Unity Editor:
+This version of the Unity ML-Agents package is compatible with the following
+versions of the Unity Editor:
 
-* 2018.4 and later
+- 2018.4 and later
 
-
-## Known limitations
+## Known Limitations
 
 ### Training
-Training is limited to the Unity Editor and Standalone builds on Windows, MacOS, and Linux with the
-Mono scripting backend. Currently, training does not work with the IL2CPP scripting backend.  Your
-environment will default to inference mode if training is not supported or is not currently running.
+
+Training is limited to the Unity Editor and Standalone builds on Windows, MacOS,
+and Linux with the Mono scripting backend. Currently, training does not work
+with the IL2CPP scripting backend. Your environment will default to inference
+mode if training is not supported or is not currently running.
 
 ### Inference
-Inference is executed via the [Unity Inference Engine](https://docs.unity3d.com/Packages/com.unity.barracuda@latest/index.html).
+
+Inference is executed via the
+[Unity Inference Engine](https://docs.unity3d.com/Packages/com.unity.barracuda@latest/index.html).
 
 **CPU**
-- All platforms supported.
+
+All platforms supported.
 
 **GPU**
-- All platforms supported except:
-  - WebGL and GLES 3/2 on Android / iPhone
 
- **NOTE:** Mobile platform support includes:
- - Vulkan for Android
- - Metal for iOS.
+All platforms supported except:
+
+- WebGL and GLES 3/2 on Android / iPhone
+
+**NOTE:** Mobile platform support includes:
+
+- Vulkan for Android
+- Metal for iOS.
 
 ### Headless Mode
 
@@ -92,24 +101,27 @@ You can control the frequency of Academy stepping by calling
 `Academy.Instance.EnvironmentStep()`
 
 ### Unity Inference Engine Models
+
 Currently, only models created with our trainers are supported for running
 ML-Agents with a neural network behavior.
 
-
 ## Helpful links
 
 If you are new to the Unity ML-Agents package, or have a question after reading
-the documentation, you can checkout our
-[GitHUb Repository], which also includes a number of ways to [connect with us]
-including our [ML-Agents Forum].
-
-
-[Unity ML-Agents Toolkit]: https://github.com/Unity-Technologies/ml-agents
-[Unity Inference Engine]: https://docs.unity3d.com/Packages/com.unity.barracuda@latest/index.html
-[Package Manager documentation]: https://docs.unity3d.com/Manual/upm-ui-install.html
-[installation instructions]: https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/docs/Installation.md
-[GitHUb Repository]: https://github.com/Unity-Technologies/ml-agents
-[Python package]: https://github.com/Unity-Technologies/ml-agents
-[Execution Order of Event Functions]: https://docs.unity3d.com/Manual/ExecutionOrder.html
-[connect with us]: https://github.com/Unity-Technologies/ml-agents#community-and-feedback
-[ML-Agents Forum]: https://forum.unity.com/forums/ml-agents.453/
+the documentation, you can checkout our [GitHUb Repository], which also includes
+a number of ways to [connect with us] including our [ML-Agents Forum].
+
+[unity ML-Agents Toolkit]: https://github.com/Unity-Technologies/ml-agents
+[unity inference engine]:
+  https://docs.unity3d.com/Packages/com.unity.barracuda@latest/index.html
+[package manager documentation]:
+  https://docs.unity3d.com/Manual/upm-ui-install.html
+[installation instructions]:
+  https://github.com/Unity-Technologies/ml-agents/blob/release_1_docs/docs/Installation.md
+[github repository]: https://github.com/Unity-Technologies/ml-agents
+[python package]: https://github.com/Unity-Technologies/ml-agents
+[execution order of event functions]:
+  https://docs.unity3d.com/Manual/ExecutionOrder.html
+[connect with us]:
+  https://github.com/Unity-Technologies/ml-agents#community-and-feedback
+[ml-agents forum]: https://forum.unity.com/forums/ml-agents.453/

docs/Background-TensorFlow.md

@@ -14,9 +14,9 @@ that we leverage within the ML-Agents Toolkit.
 performing computations using data flow graphs, the underlying representation of
 deep learning models. It facilitates training and inference on CPUs and GPUs in
 a desktop, server, or mobile device. Within the ML-Agents Toolkit, when you
-train the behavior of an agent, the output is a TensorFlow model (.nn) file that
-you can then associate with an Agent. Unless you implement a new algorithm, the
-use of TensorFlow is mostly abstracted away and behind the scenes.
+train the behavior of an agent, the output is a model (.nn) file that you can
+then associate with an Agent. Unless you implement a new algorithm, the use of
+TensorFlow is mostly abstracted away and behind the scenes.
 
 ## TensorBoard
 

docs/Custom-SideChannels.md

@@ -8,58 +8,70 @@ inappropriate as an agent observation.
 
 ## Overview
 
-In order to use a side channel, it must be implemented as both Unity and Python classes.
+In order to use a side channel, it must be implemented as both Unity and Python
+classes.
 
 ### Unity side
-The side channel will have to implement the `SideChannel` abstract class and the following method.
 
- * `OnMessageReceived(IncomingMessage msg)` : You must implement this method and read the data from IncomingMessage.
-  The data must be read in the order that it was written.
+The side channel will have to implement the `SideChannel` abstract class and the
+following method.
 
-The side channel must also assign a `ChannelId` property in the constructor. The `ChannelId` is a Guid
-(or UUID in Python) used to uniquely identify a side channel. This Guid must be the same on C# and Python.
-There can only be one side channel of a certain id during communication.
+- `OnMessageReceived(IncomingMessage msg)` : You must implement this method and
+  read the data from IncomingMessage. The data must be read in the order that it
+  was written.
 
-To send data from C# to Python, create an `OutgoingMessage` instance, add data to it, call the
-`base.QueueMessageToSend(msg)` method inside the side channel, and call the
-`OutgoingMessage.Dispose()` method.
+The side channel must also assign a `ChannelId` property in the constructor. The
+`ChannelId` is a Guid (or UUID in Python) used to uniquely identify a side
+channel. This Guid must be the same on C# and Python. There can only be one side
+channel of a certain id during communication.
 
-To register a side channel on the Unity side, call `SideChannelManager.RegisterSideChannel` with the side channel
-as only argument.
+To send data from C# to Python, create an `OutgoingMessage` instance, add data
+to it, call the `base.QueueMessageToSend(msg)` method inside the side channel,
+and call the `OutgoingMessage.Dispose()` method.
+
+To register a side channel on the Unity side, call
+`SideChannelManager.RegisterSideChannel` with the side channel as only argument.
 
 ### Python side
-The side channel will have to implement the `SideChannel` abstract class. You must implement :
 
- * `on_message_received(self, msg: "IncomingMessage") -> None` : You must implement this method and read the data
-  from IncomingMessage. The data must be read in the order that it was written.
+The side channel will have to implement the `SideChannel` abstract class. You
+must implement :
+
+- `on_message_received(self, msg: "IncomingMessage") -> None` : You must
+  implement this method and read the data from IncomingMessage. The data must be
+  read in the order that it was written.
 
-The side channel must also assign a `channel_id` property in the constructor. The `channel_id` is a UUID
-(referred in C# as Guid) used to uniquely identify a side channel. This number must be the same on C# and
-Python. There can only be one side channel of a certain id during communication.
+The side channel must also assign a `channel_id` property in the constructor.
+The `channel_id` is a UUID (referred in C# as Guid) used to uniquely identify a
+side channel. This number must be the same on C# and Python. There can only be
+one side channel of a certain id during communication.
 
-To assign the `channel_id` call the abstract class constructor with the appropriate `channel_id` as follows:
+To assign the `channel_id` call the abstract class constructor with the
+appropriate `channel_id` as follows:
 
 ```python
 super().__init__(my_channel_id)
 ```
 
-To send a byte array from Python to C#, create an `OutgoingMessage` instance, add data to it, and call the
- `super().queue_message_to_send(msg)` method inside the side channel.
+To send a byte array from Python to C#, create an `OutgoingMessage` instance,
+add data to it, and call the `super().queue_message_to_send(msg)` method inside
+the side channel.
 
-To register a side channel on the Python side, pass the side channel as argument when creating the
-`UnityEnvironment` object. One of the arguments of the constructor (`side_channels`) is a list of side channels.
+To register a side channel on the Python side, pass the side channel as argument
+when creating the `UnityEnvironment` object. One of the arguments of the
+constructor (`side_channels`) is a list of side channels.
 
 ## Example implementation
 
-Below is a simple implementation of a side channel that will exchange ascii encoded
-strings between a Unity environment and Python.
+Below is a simple implementation of a side channel that will exchange ASCII
+encoded strings between a Unity environment and Python.
 
 ### Example Unity C# code
 
-The first step is to create the `StringLogSideChannel` class within the Unity project.
-Here is an implementation of a `StringLogSideChannel` that will listen for messages
-from python and print them to the Unity debug log, as well as send error messages
-from Unity to python.
+The first step is to create the `StringLogSideChannel` class within the Unity
+project. Here is an implementation of a `StringLogSideChannel` that will listen
+for messages from python and print them to the Unity debug log, as well as send
+error messages from Unity to python.
 
 ```csharp
 using UnityEngine;
@@ -100,10 +112,10 @@ Once we have defined our custom side channel class, we need to ensure that it is
 instantiated and registered. This can typically be done wherever the logic of
 the side channel makes sense to be associated, for example on a MonoBehaviour
 object that might need to access data from the side channel. Here we show a
-simple MonoBehaviour object which instantiates and registeres the new side
-channel. If you have not done it already, make sure that the MonoBehaviour
-which registers the side channel is attached to a gameobject which will
-be live in your Unity scene.
+simple MonoBehaviour object which instantiates and registers the new side
+channel. If you have not done it already, make sure that the MonoBehaviour which
+registers the side channel is attached to a GameObject which will be live in
+your Unity scene.
 
 ```csharp
 using UnityEngine;
@@ -148,7 +160,8 @@ public class RegisterStringLogSideChannel : MonoBehaviour
 
 ### Example Python code
 
-Now that we have created the necessary Unity C# classes, we can create their Python counterparts.
+Now that we have created the necessary Unity C# classes, we can create their
+Python counterparts.
 
 ```python
 from mlagents_envs.environment import UnityEnvironment
@@ -183,10 +196,9 @@ class StringLogChannel(SideChannel):
         super().queue_message_to_send(msg)
 ```
 
-
-We can then instantiate the new side channel,
-launch a `UnityEnvironment` with that side channel active, and send a series of
-messages to the Unity environment from Python using it.
+We can then instantiate the new side channel, launch a `UnityEnvironment` with
+that side channel active, and send a series of messages to the Unity environment
+from Python using it.
 
 ```python
 # Create the channel
@@ -210,7 +222,7 @@ for i in range(1000):
 env.close()
 ```
 
-Now, if you run this script and press `Play` the Unity Editor when prompted,
-the console in the Unity Editor will display a message at every Python step.
+Now, if you run this script and press `Play` the Unity Editor when prompted, the
+console in the Unity Editor will display a message at every Python step.
 Additionally, if you press the Space Bar in the Unity Engine, a message will
 appear in the terminal.