# Training on Microsoft Azure (works with ML-Agents toolkit v0.3)

This page contains instructions for setting up training on Microsoft Azure
through either
[Azure Container Instances](https://azure.microsoft.com/services/container-instances/)
or Virtual Machines. Non "headless" training has not yet been tested to verify
support.

## Pre-Configured Azure Virtual Machine

A pre-configured virtual machine image is available in the Azure Marketplace and
is nearly completely ready for training.  You can start by deploying the
[Data Science Virtual Machine for Linux (Ubuntu)](https://azuremarketplace.microsoft.com/en-us/marketplace/apps/microsoft-dsvm.linux-data-science-vm-ubuntu)
into your Azure subscription.  Once your VM is deployed, SSH into it and run the
following command to complete dependency installation:

```sh
pip3 install docopt
```

Note that, if you choose to deploy the image to an
[N-Series GPU optimized VM](https://docs.microsoft.com/azure/virtual-machines/linux/sizes-gpu),
training will, by default, run on the GPU.  If you choose any other type of VM,
training will run on the CPU.

## Configuring your own Instance

Setting up your own instance requires a number of package installations.  Please
view the documentation for doing so
[here](Training-on-Microsoft-Azure-Custom-Instance.md).

## Installing ML-Agents

1. [Move](https://docs.microsoft.com/en-us/azure/virtual-machines/linux/copy-files-to-linux-vm-using-scp)
   the `ml-agents` sub-folder of this ml-agents repo to the remote Azure
   instance, and set it as the working directory.
2. Install the required packages with `pip3 install .`.

## Testing

To verify that all steps worked correctly:

1. In the Unity Editor, load a project containing an ML-Agents environment (you
   can use one of the example environments if you have not created your own).
2. Open the Build Settings window (menu: File > Build Settings).
3. Select Linux as the Target Platform, and x86_64 as the target architecture.
4. Check Headless Mode.
5. Click Build to build the Unity environment executable.
6. Upload the resulting files to your Azure instance.
7. Test the instance setup from Python using:

```python
from mlagents.envs import UnityEnvironment

env = UnityEnvironment(<your_env>)
```

Where `<your_env>` corresponds to the path to your environment executable.

You should receive a message confirming that the environment was loaded
successfully.

## Running Training on your Virtual Machine

To run your training on the VM:

1. [Move](https://docs.microsoft.com/en-us/azure/virtual-machines/linux/copy-files-to-linux-vm-using-scp)
    your built Unity application to your Virtual Machine.
2. Set the directory where the ML-Agents Toolkit was installed to your
   working directory.
3. Run the following command:

```sh
mlagents-learn <trainer_config> --env=<your_app> --run-id=<run_id> --train
```

Where `<your_app>` is the path to your app (i.e.
`~/unity-volume/3DBallHeadless`) and `<run_id>` is an identifier you would like
to identify your training run with.

If you've selected to run on a N-Series VM with GPU support, you can verify that
the GPU is being used by running `nvidia-smi` from the command line.

## Monitoring your Training Run with TensorBoard

Once you have started training, you can [use TensorBoard to observe the
training](Using-Tensorboard.md).  

1. Start by [opening the appropriate port for web traffic to connect to your VM](https://docs.microsoft.com/en-us/azure/virtual-machines/windows/nsg-quickstart-portal).  

    * Note that you don't need to generate a new `Network Security Group` but
      instead, go to the **Networking** tab under **Settings** for your VM.
    * As an example, you could use the following settings to open the Port with
      the following Inbound Rule settings:
      * Source: Any
      * Source Port Ranges: *
      * Destination: Any
      * Destination Port Ranges: 6006
      * Protocol: Any
      * Action: Allow
      * Priority: (Leave as default)

2. Unless you started the training as a background process, connect to your VM
   from another terminal instance.
3. Run the following command from your terminal
   `tensorboard --logdir=summaries --host 0.0.0.0`
4. You should now be able to open a browser and navigate to
   `<Your_VM_IP_Address>:6060` to view the TensorBoard report.

## Running on Azure Container Instances

[Azure Container Instances](https://azure.microsoft.com/services/container-instances/)
allow you to spin up a container, on demand, that will run your training and
then be shut down.  This ensures you aren't leaving a billable VM running when
it isn't needed.  You can read more about
[The ML-Agents toolkit support for Docker containers here](Using-Docker.md).
Using ACI enables you to offload training of your models without needing to
install Python and TensorFlow on your own computer.  You can find instructions,
including a pre-deployed image in DockerHub for you to use, available
[here](https://github.com/druttka/unity-ml-on-azure).