### Start by installing **Unity 2017.1** or later (required)
Download link available [here](https://store.unity.com/download?ref=update).
If you are new to using the Unity Editor, you can find the general documentation [here](https://docs.unity3d.com/Manual/index.html).
## Installation
### Clone the repository
Once installed, you will want to clone the Agents GitHub repository. References will be made throughout to `unity-environment` and `python` directories. Both are located at the root of the repository.
In order to install and set-up the Python and Unity environments, see the instructions [here](installation.md).
## Building Unity Environment
Launch the Unity Editor, and log in, if necessary.
3. Expand the `Ball3DAcademy` GameObject and locate its child object `Ball3DBrain` within the Scene hierarchy in the editor. Ensure Type of Brain for this object is set to `External`.
4. *File -> Build Settings*
5. Choose your target platform:
- (optional) Select “Developer Build” to log debug messages.
- (optional) Select “Development Build” to log debug messages.
## Installing Python API
In order to train an agent within the framework, you will need to install Python 2 or 3, and the dependencies described below.
### Windows Users
If you are a Windows user who is new to Python/TensorFlow, follow [this guide](https://nitishmutha.github.io/tensorflow/2017/01/22/TensorFlow-with-gpu-for-windows.html) to set up your Python environment.
### Requirements
* Jupyter
* Matplotlib
* numpy
* Pillow
* Python (2 or 3)
* TensorFlow (1.0+)
### Installing Dependencies
To install dependencies, go into the `python` directory and run:
`pip install .`
or
`pip3 install .`
If your Python environment doesn't include `pip`, see these [instructions](https://packaging.python.org/guides/installing-using-linux-tools/#installing-pip-setuptools-wheel-with-linux-package-managers) on installing it.
Once dependencies are installed, you are ready to test the Ball balance environment from Python.
## Training the Brain with Reinforcement Learning
### Testing Python API
To ensure that your environment and the Python API work as expected, you can use the `python/Basics` Jupyter notebook. This notebook contains a simple walkthrough of the functionality of the API. Within `Basics`, be sure to set `env_name` to the name of the environment file you built earlier.
## Training the Brain with Reinforcement Learning
### Training with PPO
In order to train an agent to correctly balance the ball, we will use a Reinforcement Learning algorithm called Proximal Policy Optimization (PPO). This is a method that has been shown to be safe, efficient, and more general purpose than many other RL algorithms, as such we have chosen it as the example algorithm for use with ML Agents. For more information on PPO, OpenAI has a recent [blog post](https://blog.openai.com/openai-baselines-ppo/) explaining it.
### Observing Training Progress
In order to observe the training process in more detail, you can use Tensorboard.
In your command line, run :
In your command line, enter into `python` and then run :
`tensorboard --logdir='summaries`
`tensorboard --logdir=summaries`
Then navigate to `localhost:6006`.
4. Select the `3DBallBrain` object from the Scene hierarchy.
5. Change the `Type of Brain` to `Internal`.
6. Drag the `<env_name>.bytes` file from the Project window of the Editor to the `Graph Model` placeholder in the `3DBallBrain` inspector window.
7. Set the `Graph Placeholder` size to 1.
7. Set the `Graph Placeholder` size to 1 (_Note that step 7 and 8 are done because 3DBall is a continuous control environment, and the TensorFlow model requires a noise parameter to decide actions. In cases with discrete control, epsilon is not needed_).
8. Add a placeholder called `epsilon` with a type of `floating point` and a range of values from 0 to 0.
9. Press the Play button at the top of the editor.
- `train_model` indicates whether to run the environment in train (`True`) or test (`False`) mode.
- `config` is an optional dictionary of configuration flags specific to the environment. For more information on adding optional config flags to an environment, see [here](Making-a-new-Unity-Environment.md#implementing-yournameacademy). For generic environments, `config` can be ignored. `config` is a dictionary of strings to floats where the keys are the names of the `resetParameters` and the values are their corresponding float values.
- **Step : `env.step(action, memory=None, value = None)`**
Sends a step signal to the environment using the actions. Note that if you have more than one brain in the environment, you must provide a dictionary from brain names to actions.
Sends a step signal to the environment using the actions. For each brain :
- `value` is an optional input that be used to send a single float per agent to be displayed if and `AgentMonitor.cs` component is attached to the agent.
- `value` is an optional input that be used to send a single float per agent to be displayed if and `AgentMonitor.cs` component is attached to the agent.
Note that if you have more than one external brain in the environment, you must provide dictionaries from brain names to arrays for `action`, `memory` and `value`. For example: If you have two external brains named `brain1` and `brain2` each with one agent taking two continuous actions, then you can have:
vincentpierre
7 年前