You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: README.md
+2Lines changed: 2 additions & 0 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -73,6 +73,8 @@ If you run into any problems using the ML-Agents toolkit,
73
73
[submit an issue](https://github.com/Unity-Technologies/ml-agents/issues) and
74
74
make sure to include as much detail as possible.
75
75
76
+
Your opinion matters a great deal to us. Only by hearing your thoughts on the Unity ML-Agents Toolkit can we continue to improve and grow. Please take a few minutes to [let us know about it](https://github.com/Unity-Technologies/ml-agents/issues/1454).
77
+
76
78
For any other questions or feedback, connect directly with the ML-Agents
We've included pre-trained models for the 3D Ball example.
53
54
54
-
1. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Scenes` folder
55
+
1. In the **Project** window, go to the `Assets/ML-Agents/Examples/3DBall/Scenes` folder
55
56
and open the `3DBall` scene file.
56
-
2. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Prefabs` folder
57
-
and select the `Game/Platform` prefab.
58
-
3. In the `Ball 3D Agent` Component: Drag the **3DBallLearning** Brain located in
59
-
`Assets/ML-Agents/Examples/3DBall/Brains` into the `Brain` property of
60
-
the `Ball 3D Agent`.
61
-
4. Make sure that all of the Agents in the Scene now have **3DBallLearning** as `Brain`.
62
-
__Note__ : You can modify multiple game objects in a scene by selecting them all at
57
+
2. In the **Project** window, go to the `Assets/ML-Agents/Examples/3DBall/Prefabs` folder.
58
+
Expand `Game` and click on the `Platform` prefab. You should see the `Platform` prefab in the **Inspector** window.
59
+
60
+
**Note**: The platforms in the `3DBall` scene were created using the `Platform` prefab. Instead of updating all 12 platforms individually, you can update the `Platform` prefab instead.
61
+
62
+
63
+
64
+
3. In the **Project** window, drag the **3DBallLearning** Brain located in
65
+
`Assets/ML-Agents/Examples/3DBall/Brains` into the `Brain` property under `Ball 3D Agent (Script)` component in the **Inspector** window.
4. You should notice that each `Platform` under each `Game` in the **Hierarchy** windows now contains **3DBallLearning** as `Brain`. __Note__ : You can modify multiple game objects in a scene by selecting them all at
63
70
once using the search bar in the Scene Hierarchy.
64
-
5. In the **Project** window, locate the `Assets/ML-Agents/Examples/3DBall/TFModels`
71
+
5. In the **Project** window, click on the **3DBallLearning** Brain located in
72
+
`Assets/ML-Agents/Examples/3DBall/Brains`. You should see the properties in the **Inspector** window.
73
+
6. In the **Project** window, open the `Assets/ML-Agents/Examples/3DBall/TFModels`
65
74
folder.
66
-
6. Drag the `3DBallLearning` model file from the `Assets/ML-Agents/Examples/3DBall/TFModels`
67
-
folder to the **Model** field of the **3DBallLearning** Brain.
68
-
7. Click the **Play** button and you will see the platforms balance the balls
75
+
7. Drag the `3DBallLearning` model file from the `Assets/ML-Agents/Examples/3DBall/TFModels`
76
+
folder to the **Model** field of the **3DBallLearning** Brain in the **Inspector** window. __Note__ : All of the brains should now have `3DBallLearning` as the TensorFlow model in the `Model` property
77
+
8. Click the **Play** button and you will see the platforms balance the balls
69
78
using the pretrained model.
70
79
71
-
80
+
72
81
73
82
## Using the Basics Jupyter Notebook
74
83
@@ -84,30 +93,26 @@ More information and documentation is provided in the
84
93
85
94
## Training the Brain with Reinforcement Learning
86
95
87
-
### Adding a Brain to the training session
96
+
### Setting up the enviornment for training
88
97
89
98
To set up the environment for training, you will need to specify which agents are contributing
90
99
to the training and which Brain is being trained. You can only perform training with
91
100
a `Learning Brain`.
92
101
93
-
1. Assign the **3DBallLearning** Brain to the agents you would like to train.
94
-
__Note:__ You can assign the same Brain to multiple agents at once : To do so, you can
95
-
use the prefab system. When an agent is created from a prefab, modifying the prefab
96
-
will modify the agent as well. If the agent does not synchronize with the prefab, you
97
-
can hit the Revert button on top of the Inspector.
98
-
Alternatively, you can select multiple agents in the scene and modify their `Brain`
99
-
property all at once.
100
-
2. Select the **Ball3DAcademy** GameObject and make sure the **3DBallLearning** Brain
101
-
is in the Broadcast Hub. In order to train, you need to toggle the
102
-
`Control` checkbox.
103
-
104
-
__Note:__ Assigning a Brain to an agent (dragging a Brain into the `Brain` property of
102
+
1. Each platform agent needs an assigned `Learning Brain`. In this example, each platform agent was created using a prefab. To update all of the brains in each platform agent at once, you only need to update the platform agent prefab. In the **Project** window, go to the `Assets/ML-Agents/Examples/3DBall/Prefabs` folder. Expand `Game` and click on the `Platform` prefab. You should see the `Platform` prefab in the **Inspector** window. In the **Project** window, drag the **3DBallLearning** Brain located in `Assets/ML-Agents/Examples/3DBall/Brains` into the `Brain` property under `Ball 3D Agent (Script)` component in the **Inspector** window.
103
+
104
+
**Note**: The Unity prefab system will modify all instances of the agent properties in your scene. If the agent does not synchronize automatically with the prefab, you can hit the Revert button in the top of the **Inspector** window.
105
+
106
+
2. In the **Hierarchy** window, select `Ball3DAcademy`.
107
+
3. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Brains` folder and drag the **3DBallLearning** Brain to the `Brains` property under `Braodcast Hub` in the `Ball3DAcademy` object in the **Inspector** window. In order to train, make sure the `Control` checkbox is selected.
108
+
109
+
**Note:** Assigning a Brain to an agent (dragging a Brain into the `Brain` property of
105
110
the agent) means that the Brain will be making decision for that agent. Whereas dragging
106
111
a Brain into the Broadcast Hub means that the Brain will be exposed to the Python process.
107
112
The `Control` checkbox means that in addition to being exposed to Python, the Brain will
108
113
be controlled by the Python process (required for training).
109
114
110
-
115
+
111
116
112
117
### Training the environment
113
118
@@ -134,7 +139,7 @@ be controlled by the Python process (required for training).
134
139
Editor"_ is displayed on the screen, you can press the :arrow_forward: button
135
140
in Unity to start training in the Editor.
136
141
137
-
**Note**: Alternatively, you can use an executable rather than the Editor to
142
+
**Note**: Alternatively, you can use an executable rather than the Editor to
138
143
perform training. Please refer to [this
139
144
page](Learning-Environment-Executable.md) for instructions on how to build and
Copy file name to clipboardExpand all lines: docs/Getting-Started-with-Balance-Ball.md
+19-23Lines changed: 19 additions & 23 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -28,7 +28,7 @@ Let's get started!
28
28
In order to install and set up the ML-Agents toolkit, the Python dependencies
29
29
and Unity, see the [installation instructions](Installation.md).
30
30
31
-
## Understanding a Unity Environment (3D Balance Ball)
31
+
## Understanding the Unity Environment (3D Balance Ball)
32
32
33
33
An agent is an autonomous actor that observes and interacts with an
34
34
_environment_. In the context of Unity, an environment is a scene containing an
@@ -53,26 +53,26 @@ to speed up training since all twelve agents contribute to training in parallel.
53
53
The Academy object for the scene is placed on the Ball3DAcademy GameObject. When
54
54
you look at an Academy component in the inspector, you can see several
55
55
properties that control how the environment works.
56
-
The **Broadcast Hub** keeps track of which Brains will send data during training,
57
-
If a Brain is added to the hub, his data will be sent to the external training
56
+
The **Broadcast Hub** keeps track of which Brains will send data during training.
57
+
If a Brain is added to the hub, the data from this Brain will be sent to the external training
58
58
process. If the `Control` checkbox is checked, the training process will be able to
59
-
control the agents linked to the Brain to train them.
60
-
The **Training** and **Inference Configuration** properties
59
+
control and train the agents linked to the Brain.
60
+
The **Training Configuration** and **Inference Configuration** properties
61
61
set the graphics and timescale properties for the Unity application.
62
62
The Academy uses the **Training Configuration** during training and the
63
63
**Inference Configuration** when not training. (*Inference* means that the
64
64
Agent is using a trained model or heuristics or direct control — in other
65
65
words, whenever **not** training.)
66
-
Typically, you set low graphics quality and a high time scale for the **Training
67
-
configuration** and a high graphics quality and the timescale to `1.0` for the
66
+
Typically, you would set a low graphics quality and timescale to greater `1.0` for the **Training
67
+
Configuration** and a high graphics quality and timescale to `1.0` for the
68
68
**Inference Configuration** .
69
69
70
70
**Note:** if you want to observe the environment during training, you can adjust
71
71
the **Training Configuration** settings to use a larger window and a timescale
72
72
closer to 1:1. Be sure to set these parameters back when training in earnest;
73
73
otherwise, training can take a very long time.
74
74
75
-
Another aspect of an environment to look at is the Academy implementation. Since
75
+
Another aspect of an environment is the Academy implementation. Since
76
76
the base Academy class is abstract, you must always define a subclass. There are
77
77
three functions you can implement, though they are all optional:
78
78
@@ -88,28 +88,24 @@ environment around the Agents.
88
88
89
89
### Brain
90
90
91
-
Brains are assets that exist in your project folder. The Ball3DAgents are connected
92
-
to a Brain, for example : the **3DBallLearning**.
93
-
A Brain doesn't store any information about an Agent, it just
91
+
As of v0.6, a Brain is a Unity asset and exists within the `UnitySDK` folder. These brains (ex. **3DBallLearning.asset**) are loaded into each Agent object (ex. **Ball3DAgents**). A Brain doesn't store any information about an Agent, it just
94
92
routes the Agent's collected observations to the decision making process and
95
-
returns the chosen action to the Agent. Thus, all Agents can share the same
96
-
Brain, but act independently. The Brain settings tell you quite a bit about how
93
+
returns the chosen action to the Agent. All Agents can share the same
94
+
Brain, but would act independently. The Brain settings tell you quite a bit about how
97
95
an Agent works.
98
96
99
-
You can create Brain objects by selecting `Assets ->
100
-
Create -> ML-Agents -> Brain`. There are 3 kinds of Brains :
101
-
The **Learning Brain** is a Brain that uses a Neural Network to take decisions.
102
-
When the Brain is checked as `Control` in the Academy **Broadcast Hub**, the
103
-
external process will be taking decisions for the agents
104
-
and generate a neural network when the training is over. You can also use the
97
+
You can create new Brain assets by selecting `Assets ->
98
+
Create -> ML-Agents -> Brain`. There are 3 types of Brains.
99
+
The **Learning Brain** is a Brain that uses a trained neural network to make decisions.
100
+
When the `Control` box is checked in the Brains property under the **Broadcast Hub** in the Academy, the external process that is training the neural network will take over decision making for the agents
101
+
and ultimately generate a trained neural network. You can also use the
105
102
**Learning Brain** with a pre-trained model.
106
-
The **Heuristic** Brain allows you to hand-code the Agent's logic by extending
103
+
The **Heuristic** Brain allows you to hand-code the Agent logic by extending
107
104
the Decision class.
108
105
Finally, the **Player** Brain lets you map keyboard commands to actions, which
109
-
can be useful when testing your agents and environment. If none of these types
110
-
of Brains do what you need, you can implement your own Brain.
106
+
can be useful when testing your agents and environment. You can also implement your own type of Brain.
111
107
112
-
In this tutorial, you will use a**Learning Brain** for training.
108
+
In this tutorial, you will use the**Learning Brain** for training.
0 commit comments