ml-agents/docs/Installation.md

Installation

The ML-Agents Toolkit contains several components:

• Unity package (com.unity.ml-agents) contains the Unity C# SDK that will be integrated into your Unity project. This package contains a sample to help you get started with ML-Agents.
• Unity package (com.unity.ml-agents.extensions) contains experimental C#/Unity components that are not yet ready to be part of the base com.unity.ml-agents package. com.unity.ml-agents.extensions has a direct dependency on com.unity.ml-agents.
• Three Python packages:
  • mlagents contains the machine learning algorithms that enables you to train behaviors in your Unity scene. Most users of ML-Agents will only need to directly install mlagents.
  • mlagents_envs contains a Python API to interact with a Unity scene. It is a foundational layer that facilitates data messaging between Unity scene and the Python machine learning algorithms. Consequently, mlagents depends on mlagents_envs.
  • gym_unity provides a Python-wrapper for your Unity scene that supports the OpenAI Gym interface.
• Unity Project that contains several example environments that highlight the various features of the toolkit to help you get started.

Consequently, to install and use the ML-Agents Toolkit you will need to:

• Install Unity (2019.4 or later)
• Install Python (3.6.1 or higher)
• Clone this repository (Optional)
  • Note: If you do not clone the repository, then you will not be able to access the example environments and training configurations or the com.unity.ml-agents.extensions package. Additionally, the Getting Started Guide assumes that you have cloned the repository.
• Install the com.unity.ml-agents Unity package
• Install the com.unity.ml-agents.extensions Unity package (Optional)
• Install the mlagents Python package

Install Unity 2019.4 or Later

Download and install Unity. We strongly recommend that you install Unity through the Unity Hub as it will enable you to manage multiple Unity versions.

Install Python 3.6.1 or Higher

We recommend installing Python 3.6 or 3.7. If you are using Windows, please install the x86-64 version and not x86. If your Python environment doesn't include pip3, see these instructions on installing it.

Clone the ML-Agents Toolkit Repository (Optional)

Now that you have installed Unity and Python, you can now install the Unity and Python packages. You do not need to clone the repository to install those packages, but you may choose to clone the repository if you'd like download our example environments and training configurations to experiment with them (some of our tutorials / guides assume you have access to our example environments).

NOTE: There are samples shipped with the Unity Package. You only need to clone the repository if you would like to explore more examples.

git clone --branch release_16 https://github.com/Unity-Technologies/ml-agents.git

The --branch release_16 option will switch to the tag of the latest stable release. Omitting that will get the main branch which is potentially unstable.

Advanced: Local Installation for Development

You will need to clone the repository if you plan to modify or extend the ML-Agents Toolkit for your purposes. If you plan to contribute those changes back, make sure to clone the main branch (by omitting --branch release_16 from the command above). See our Contributions Guidelines for more information on contributing to the ML-Agents Toolkit.

Install the com.unity.ml-agents Unity package

The Unity ML-Agents C# SDK is a Unity Package. You can install the com.unity.ml-agents package directly from the Package Manager registry. Please make sure you enable 'Preview Packages' in the 'Advanced' dropdown in order to find the latest Preview release of the package.

NOTE: If you do not see the ML-Agents package listed in the Package Manager please follow the advanced installation instructions below.

Advanced: Local Installation for Development

You can add the local com.unity.ml-agents package (from the repository that you just cloned) to your project by:

1. navigating to the menu Window -> Package Manager.
2. In the package manager window click on the + button on the top left of the packages list).
3. Select Add package from disk...
4. Navigate into the com.unity.ml-agents folder.
5. Select the package.json file.

If you are going to follow the examples from our documentation, you can open the Project folder in Unity and start tinkering immediately.

Install the com.unity.ml-agents.extensions Unity package (Optional)

To install the com.unity.ml-agents.extensions package, you need to first clone the repo and then complete a local installation similar to what was outlined in the previous Advanced: Local Installation for Development section. Complete installation steps can be found in the package documentation.

Install the mlagents Python package

Installing the mlagents Python package involves installing other Python packages that mlagents depends on. So you may run into installation issues if your machine has older versions of any of those dependencies already installed. Consequently, our supported path for installing mlagents is to leverage Python Virtual Environments. Virtual Environments provide a mechanism for isolating the dependencies for each project and are supported on Mac / Windows / Linux. We offer a dedicated guide on Virtual Environments.

(Windows) Installing PyTorch

On Windows, you'll have to install the PyTorch package separately prior to installing ML-Agents. Activate your virtual environment and run from the command line:

pip3 install torch~=1.7.1 -f https://download.pytorch.org/whl/torch_stable.html

Note that on Windows, you may also need Microsoft's Visual C++ Redistributable if you don't have it already. See the PyTorch installation guide for more installation options and versions.

Installing mlagents

To install the mlagents Python package, activate your virtual environment and run from the command line:

python -m pip install mlagents==0.25.1

Note that this will install mlagents from PyPi, not from the cloned repository. If you installed this correctly, you should be able to run mlagents-learn --help, after which you will see the command line parameters you can use with mlagents-learn.

By installing the mlagents package, the dependencies listed in the setup.py file are also installed. These include PyTorch (Requires a CPU w/ AVX support).

Advanced: Local Installation for Development

If you intend to make modifications to mlagents or mlagents_envs, you should install the packages from the cloned repository rather than from PyPi. To do this, you will need to install mlagents and mlagents_envs separately. From the repository's root directory, run:

pip3 install torch -f https://download.pytorch.org/whl/torch_stable.html
pip3 install -e ./ml-agents-envs
pip3 install -e ./ml-agents

Running pip with the -e flag will let you make changes to the Python files directly and have those reflected when you run mlagents-learn. It is important to install these packages in this order as the mlagents package depends on mlagents_envs, and installing it in the other order will download mlagents_envs from PyPi.

Next Steps

The Getting Started guide contains several short tutorials on setting up the ML-Agents Toolkit within Unity, running a pre-trained model, in addition to building and training environments.

Help

If you run into any problems regarding ML-Agents, refer to our FAQ and our Limitations pages. If you can't find anything please submit an issue and make sure to cite relevant information on OS, Python version, and exact error message (whenever possible).