[submit an issue](https://github.com/Unity-Technologies/ml-agents/issues) and
make sure to include as much detail as possible.
Your opinion matters a great deal to us. Only by hearing your thoughts on the Unity ML-Agents Toolkit can we continue to improve and grow. Please take a few minutes to [let us know about it](https://github.com/Unity-Technologies/ml-agents/issues/1454).
For any other questions or feedback, connect directly with the ML-Agents
We've included pre-trained models for the 3D Ball example.
1. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Scenes` folder
1. In the **Project** window, go to the `Assets/ML-Agents/Examples/3DBall/Scenes` folder
2. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Prefabs` folder
and select the `Game/Platform` prefab.
3. In the `Ball 3D Agent` Component: Drag the **3DBallLearning** Brain located in
`Assets/ML-Agents/Examples/3DBall/Brains` into the `Brain` property of
the `Ball 3D Agent`.
4. Make sure that all of the Agents in the Scene now have **3DBallLearning** as `Brain`.
__Note__ : You can modify multiple game objects in a scene by selecting them all at
2. In the **Project** window, go to the `Assets/ML-Agents/Examples/3DBall/Prefabs` folder.
Expand `Game` and click on the `Platform` prefab. You should see the `Platform` prefab in the **Inspector** window.
**Note**: The platforms in the `3DBall` scene were created using the `Platform` prefab. Instead of updating all 12 platforms individually, you can update the `Platform` prefab instead.
3. In the **Project** window, drag the **3DBallLearning** Brain located in
`Assets/ML-Agents/Examples/3DBall/Brains` into the `Brain` property under `Ball 3D Agent (Script)` component in the **Inspector** window.
4. You should notice that each `Platform` under each `Game` in the **Hierarchy** windows now contains **3DBallLearning** as `Brain`. __Note__ : You can modify multiple game objects in a scene by selecting them all at
5. In the **Project** window, locate the `Assets/ML-Agents/Examples/3DBall/TFModels`
5. In the **Project** window, click on the **3DBallLearning** Brain located in
`Assets/ML-Agents/Examples/3DBall/Brains`. You should see the properties in the **Inspector** window.
6. In the **Project** window, open the `Assets/ML-Agents/Examples/3DBall/TFModels`
6. Drag the `3DBallLearning` model file from the `Assets/ML-Agents/Examples/3DBall/TFModels`
folder to the **Model** field of the **3DBallLearning** Brain.
7. Click the **Play** button and you will see the platforms balance the balls
7. Drag the `3DBallLearning` model file from the `Assets/ML-Agents/Examples/3DBall/TFModels`
folder to the **Model** field of the **3DBallLearning** Brain in the **Inspector** window. __Note__ : All of the brains should now have `3DBallLearning` as the TensorFlow model in the `Model` property
8. Click the **Play** button and you will see the platforms balance the balls
## Using the Basics Jupyter Notebook
## Training the Brain with Reinforcement Learning
### Adding a Brain to the training session
### Setting up the enviornment for training
1. Assign the **3DBallLearning** Brain to the agents you would like to train.
__Note:__ You can assign the same Brain to multiple agents at once : To do so, you can
use the prefab system. When an agent is created from a prefab, modifying the prefab
will modify the agent as well. If the agent does not synchronize with the prefab, you
can hit the Revert button on top of the Inspector.
Alternatively, you can select multiple agents in the scene and modify their `Brain`
property all at once.
2. Select the **Ball3DAcademy** GameObject and make sure the **3DBallLearning** Brain
is in the Broadcast Hub. In order to train, you need to toggle the
`Control` checkbox.
__Note:__ Assigning a Brain to an agent (dragging a Brain into the `Brain` property of
1. Each platform agent needs an assigned `Learning Brain`. In this example, each platform agent was created using a prefab. To update all of the brains in each platform agent at once, you only need to update the platform agent prefab. In the **Project** window, go to the `Assets/ML-Agents/Examples/3DBall/Prefabs` folder. Expand `Game` and click on the `Platform` prefab. You should see the `Platform` prefab in the **Inspector** window. In the **Project** window, drag the **3DBallLearning** Brain located in `Assets/ML-Agents/Examples/3DBall/Brains` into the `Brain` property under `Ball 3D Agent (Script)` component in the **Inspector** window.
**Note**: The Unity prefab system will modify all instances of the agent properties in your scene. If the agent does not synchronize automatically with the prefab, you can hit the Revert button in the top of the **Inspector** window.
2. In the **Hierarchy** window, select `Ball3DAcademy`.
3. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Brains` folder and drag the **3DBallLearning** Brain to the `Brains` property under `Braodcast Hub` in the `Ball3DAcademy` object in the **Inspector** window. In order to train, make sure the `Control` checkbox is selected.
**Note:** Assigning a Brain to an agent (dragging a Brain into the `Brain` property of
### Training the environment
Editor"_ is displayed on the screen, you can press the :arrow_forward: button
in Unity to start training in the Editor.
**Note**: Alternatively, you can use an executable rather than the Editor to
**Note**: Alternatively, you can use an executable rather than the Editor to
perform training. Please refer to [this
page](Learning-Environment-Executable.md) for instructions on how to build and
In order to install and set up the ML-Agents toolkit, the Python dependencies
and Unity, see the [installation instructions](Installation.md).
## Understanding a Unity Environment (3D Balance Ball)
## Understanding the Unity Environment (3D Balance Ball)
An agent is an autonomous actor that observes and interacts with an
_environment_. In the context of Unity, an environment is a scene containing an
The Academy object for the scene is placed on the Ball3DAcademy GameObject. When
you look at an Academy component in the inspector, you can see several
properties that control how the environment works.
The **Broadcast Hub** keeps track of which Brains will send data during training,
If a Brain is added to the hub, his data will be sent to the external training
The **Broadcast Hub** keeps track of which Brains will send data during training.
If a Brain is added to the hub, the data from this Brain will be sent to the external training
control the agents linked to the Brain to train them.
The **Training** and **Inference Configuration** properties
control and train the agents linked to the Brain.
The **Training Configuration** and **Inference Configuration** properties
Typically, you set low graphics quality and a high time scale for the **Training
configuration** and a high graphics quality and the timescale to `1.0` for the
Typically, you would set a low graphics quality and timescale to greater `1.0` for the **Training
Configuration** and a high graphics quality and timescale to `1.0` for the
**Inference Configuration** .
**Note:** if you want to observe the environment during training, you can adjust
Another aspect of an environment to look at is the Academy implementation. Since
Another aspect of an environment is the Academy implementation. Since
the base Academy class is abstract, you must always define a subclass. There are
three functions you can implement, though they are all optional:
### Brain
Brains are assets that exist in your project folder. The Ball3DAgents are connected
to a Brain, for example : the **3DBallLearning**.
A Brain doesn't store any information about an Agent, it just
As of v0.6, a Brain is a Unity asset and exists within the `UnitySDK` folder. These brains (ex. **3DBallLearning.asset**) are loaded into each Agent object (ex. **Ball3DAgents**). A Brain doesn't store any information about an Agent, it just
returns the chosen action to the Agent. Thus, all Agents can share the same
Brain, but act independently. The Brain settings tell you quite a bit about how
returns the chosen action to the Agent. All Agents can share the same
Brain, but would act independently. The Brain settings tell you quite a bit about how
You can create Brain objects by selecting `Assets ->
Create -> ML-Agents -> Brain`. There are 3 kinds of Brains :
The **Learning Brain** is a Brain that uses a Neural Network to take decisions.
When the Brain is checked as `Control` in the Academy **Broadcast Hub**, the
external process will be taking decisions for the agents
and generate a neural network when the training is over. You can also use the
You can create new Brain assets by selecting `Assets ->
Create -> ML-Agents -> Brain`. There are 3 types of Brains.
The **Learning Brain** is a Brain that uses a trained neural network to make decisions.
When the `Control` box is checked in the Brains property under the **Broadcast Hub** in the Academy, the external process that is training the neural network will take over decision making for the agents
and ultimately generate a trained neural network. You can also use the
The **Heuristic** Brain allows you to hand-code the Agent's logic by extending
The **Heuristic** Brain allows you to hand-code the Agent logic by extending
can be useful when testing your agents and environment. If none of these types
of Brains do what you need, you can implement your own Brain.
can be useful when testing your agents and environment. You can also implement your own type of Brain.
In this tutorial, you will use a **Learning Brain** for training.
In this tutorial, you will use the **Learning Brain** for training.
GitHub
6 年前
