Skip to content

Commit 4387060

Browse files
authored
Update AWS page (#478)
* Update AWS page
1 parent c73e044 commit 4387060

File tree

1 file changed

+41
-26
lines changed

1 file changed

+41
-26
lines changed
Lines changed: 41 additions & 26 deletions
Original file line numberDiff line numberDiff line change
@@ -1,53 +1,68 @@
11
# Training on Amazon Web Service
22

3-
This page contains instructions for setting up an EC2 instance on Amazon Web Service for use in training ML-Agents environments. Current limitations of the Unity Engine require that a screen be available to render to. In order to make this possible when training on a remote server, a virtual screen is required. We can do this by installing Xorg and creating a virtual screen. Once installed and created, we can display the Unity environment in the virtual environment, and train as we would on a local machine.
3+
This page contains instructions for setting up an EC2 instance on Amazon Web Service for training ML-Agents environments. You can run "headless" training if none of the agents in the environment use visual observations.
44

55
## Pre-Configured AMI
66
A public pre-configured AMI is available with the ID: `ami-30ec184a` in the `us-east-1` region. It was created as a modification of the Amazon Deep Learning [AMI](https://aws.amazon.com/marketplace/pp/B01M0AXXQB).
77

88
## Configuring your own Instance
9-
Instructions here are adapted from this [Medium post](https://medium.com/towards-data-science/how-to-run-unity-on-amazon-cloud-or-without-monitor-3c10ce022639) on running general Unity applications in the cloud.
109

1110
1. To begin with, you will need an EC2 instance which contains the latest Nvidia drivers, CUDA8, and cuDNN. There are a number of external tutorials which describe this, such as:
1211
* [Getting CUDA 8 to Work With openAI Gym on AWS and Compiling TensorFlow for CUDA 8 Compatibility](https://davidsanwald.github.io/2016/11/13/building-tensorflow-with-gpu-support.html)
1312
* [Installing TensorFlow on an AWS EC2 P2 GPU Instance](http://expressionflow.com/2016/10/09/installing-tensorflow-on-an-aws-ec2-p2-gpu-instance/)
1413
* [Updating Nvidia CUDA to 8.0.x in Ubuntu 16.04 – EC2 Gx instance](https://aichamp.wordpress.com/2016/11/09/updating-nvidia-cuda-to-8-0-x-in-ubuntu-16-04-ec2-gx-instance/)
15-
2. Move `python` to remote instance.
14+
15+
## Installing ML-Agents
16+
17+
2. Move `python` sub-folder of this ml-agents repo to the remote ECS instance, and set it as the working directory.
1618
2. Install the required packages with `pip3 install .`.
17-
3. Run the following commands to install Xorg:
19+
20+
## Testing
21+
22+
To verify that all steps worked correctly:
23+
24+
1. In the Unity Editor, load a project containing an ML-Agents environment (you can use one of the example environments if you have not created your own).
25+
2. Open the Build Settings window (menu: File > Build Settings).
26+
3. Select Linux as the Target Platform, and x64_86 as the target architecture.
27+
4. Check Headless Mode (unless you have enabled a virtual screen following the instructions below).
28+
5. Click Build to build the Unity environment executable.
29+
6. Upload the executable to your EC2 instance.
30+
7. Test the instance setup from Python using:
31+
32+
```python
33+
from unityagents import UnityEnvironment
34+
35+
env = UnityEnvironment(<your_env>)
36+
```
37+
Where `<your_env>` corresponds to the path to your environment executable.
38+
39+
You should receive a message confirming that the environment was loaded successfully.
40+
41+
## (Optional) Enabling a virtual screen
42+
43+
_Instructions here are adapted from this [Medium post](https://medium.com/towards-data-science/how-to-run-unity-on-amazon-cloud-or-without-monitor-3c10ce022639) on running general Unity applications in the cloud._
44+
45+
Current limitations of the Unity Engine require that a screen be available to render to when using visual observations. In order to make this possible when training on a remote server, a virtual screen is required. We can do this by installing Xorg and creating a virtual screen. Once installed and created, we can display the Unity environment in the virtual environment, and train as we would on a local machine. Ensure that `headless` mode is disabled when building linux executables which use visual observations.
46+
47+
1. Run the following commands to install Xorg:
48+
1849
```
1950
sudo apt-get update
2051
sudo apt-get install -y xserver-xorg mesa-utils
2152
sudo nvidia-xconfig -a --use-display-device=None --virtual=1280x1024
2253
```
23-
4. Restart the EC2 instance.
2454
25-
## Launching your instance
55+
2. Restart the EC2 instance.
2656
27-
1. Make sure there are no Xorg processes running. To kill the Xorg processes, run `sudo killall Xorg`.
57+
3. Make sure there are no Xorg processes running. To kill the Xorg processes, run `sudo killall Xorg`.
2858
Note that you might have to run this command multiple times depending on how Xorg is configured.
2959
If you run `nvidia-smi`, you will have a list of processes running on the GPU, Xorg should not be in the list.
3060
31-
2. Run:
61+
4. Run:
62+
3263
```
3364
sudo /usr/bin/X :0 &
3465
export DISPLAY=:0
3566
```
36-
3. To ensure the installation was successful, run `glxgears`. If there are no errors, then Xorg is correctly configured.
37-
4. There is a bug in _Unity 2017.1_ which requires the uninstallation of `libxrandr2`, which can be removed with :
38-
```
39-
sudo apt-get remove --purge libwxgtk3.0-0v5
40-
sudo apt-get remove --purge libxrandr2
41-
```
42-
This is scheduled to be fixed in 2017.3.
43-
44-
## Testing
45-
46-
If all steps worked correctly, upload an example binary built for Linux to the instance, and test it from Python with:
47-
```python
48-
from unityagents import UnityEnvironment
49-
50-
env = UnityEnvironment(your_env)
51-
```
52-
53-
You should receive a message confirming that the environment was loaded successfully.
67+
68+
5. To ensure the installation was successful, run `glxgears`. If there are no errors, then Xorg is correctly configured.

0 commit comments

Comments
 (0)