unity-tech-cn
/
ml-agents
46
14
派生 39
代码 合并请求 版本发布
浏览代码

Release v0.6 docs (#1325) 

* Fix Typo #1323

* First update to the docs

* Addressed comments

* remove references to TF#

* Replaced the references to TF# with new document.

* Edditied the FAQ
/develop-generalizationTraining-TrainerController
GitHub 6 年前
当前提交
0702f677
共有 14 个文件被更改，包括 81 次插入 和 433 次删除
  1. 19
      docs/Background-TensorFlow.md
  2. 73
      docs/Basic-Guide.md
  3. 32
      docs/FAQ.md
  4. 16
      docs/Getting-Started-with-Balance-Ball.md
  5. 16
      docs/Learning-Environment-Design-Learning-Brains.md
  6. 38
      docs/Learning-Environment-Executable.md
  7. 3
      docs/Limitations.md
  8. 5
      docs/ML-Agents-Overview.md
  9. 4
      docs/Readme.md
  10. 4
      docs/Training-Curriculum-Learning.md
  11. 4
      docs/Training-Imitation-Learning.md
  12. 2
      docs/Training-ML-Agents.md
  13. 117
      docs/images/imported-tensorflowsharp.png
  14. 181
      docs/Using-TensorFlow-Sharp-in-Unity.md

19
docs/Background-TensorFlow.md
查看文件


performing computations using data flow graphs, the underlying representation of
deep learning models. It facilitates training and inference on CPUs and GPUs in
a desktop, server, or mobile device. Within the ML-Agents toolkit, when you
train the behavior of an agent, the output is a TensorFlow model (.bytes) file
train the behavior of an agent, the output is a TensorFlow model (.tf) file
that you can then embed within a Learning Brain. Unless you implement a new
algorithm, the use of TensorFlow is mostly abstracted away and behind the
scenes.

recommend this
[tutorial](https://github.com/dandelionmane/tf-dev-summit-tensorboard-tutorial).
## TensorFlowSharp
## Tensorflow Model Inference
This means that the Learning Brain is not natively supported since Unity scripts
are written in C#. Consequently, to enable the Learning Brain, we leverage a
third-party library
[TensorFlowSharp](https://github.com/migueldeicaza/TensorFlowSharp) which
provides .NET bindings to TensorFlow. Thus, when a Unity environment that
contains a Learning Brain is built, inference is performed via TensorFlowSharp.
We provide an additional in-depth overview of how to leverage
[TensorFlowSharp within Unity](Using-TensorFlow-Sharp-in-Unity.md)
which will become more
relevant once you install and start training behaviors within the ML-Agents
toolkit. Given the reliance on TensorFlowSharp, the Learning Brain is currently
marked as experimental.
We have are using the [Unity Machine Learning Inference SDK](Inference-Engine.md) to
run the models inside of Unity. In order to use it, you will need to have an
appropriate backend downloaded.

73
docs/Basic-Guide.md
查看文件


## Setting up the ML-Agents Toolkit within Unity
In order to use the ML-Agents toolkit within Unity, you need to change some
Unity settings first. Also [TensorFlowSharp
plugin](https://s3.amazonaws.com/unity-ml-agents/0.5/TFSharpPlugin.unitypackage)
is needed for you to use pre-trained model within Unity, which is based on the
[TensorFlowSharp repo](https://github.com/migueldeicaza/TensorFlowSharp).
Unity settings first. Youy will also need to have appropriate inference backends
installed in order to run your models inside of Unity. See [here](Inference-Engine.md)
for more information.
1. Launch Unity
2. On the Projects dialog, choose the **Open** option at the top of the window.

1. Option the **Other Settings** section.
2. Select **Scripting Runtime Version** to **Experimental (.NET 4.6
Equivalent or .NET 4.x Equivalent)**
3. In **Scripting Defined Symbols**, add the flag `ENABLE_TENSORFLOW`. After
typing in the flag name, press Enter.
![Project Settings](images/project-settings.png)
[Download](https://s3.amazonaws.com/unity-ml-agents/0.5/TFSharpPlugin.unitypackage)
the TensorFlowSharp plugin. Then import it into Unity by double clicking the
downloaded file. You can check if it was successfully imported by checking the
TensorFlow files in the Project window under **Assets** > **ML-Agents** >
**Plugins** > **Computer**.
**Note**: If you don't see anything under **Assets**, drag the
`UnitySDK/Assets/ML-Agents` folder under **Assets** within Project window.
![Imported TensorFlowsharp](images/imported-tensorflowsharp.png)
## Running a Pre-trained Model
1. In the **Project** window, go to `Assets/ML-Agents/Examples/3DBall/Scenes` folder

3. In the `Ball 3D Agent` Component: Drag the **Ball3DBrain** located into
3. In the `Ball 3D Agent` Component: Drag the **3DBallLearning** located into
4. Make sure that all of the Agents in the Scene now have **Ball3DBrain** as `Brain`.
4. Make sure that all of the Agents in the Scene now have **3DBallLearning** as `Brain`.
folder to the **Model** field of the **Ball3DBrain**.
folder to the **Model** field of the **3DBallLearning**.
7. Click the **Play** button and you will see the platforms balance the balls
using the pretrained model.

the Brain to the training session. This allows the Agents linked to that Brain
to communicate with the external training process when making their decisions.
1. Assign the **Ball3DBrain** to the agents you would like to train.
1. Assign the **3DBallLearning** to the agents you would like to train and the **3DBallPlayer** Brain to the agents you want to control manually.
2. Select the **Ball3DAcademy** GameObject and add the **Ball3DBrain**
to the Broadcast Hub and toggle the `Control` checkbox.
2. Select the **Ball3DAcademy** GameObject and make sure the **3DBallLearning** Brain
is in the Broadcast Hub. In order to train, you need to toggle the
`Control` checkbox.
![Set Brain to External](images/mlagents-SetBrainToTrain.png)

'Ball3DAcademy' started successfully!
Unity Academy name: Ball3DAcademy
Number of Brains: 1
Number of External Brains : 1
Number of Training Brains : 1
Unity brain name: Ball3DBrain
Unity brain name: 3DBallLearning
Number of Visual Observations (per agent): 0
Vector Observation space size (per agent): 8
Number of stacked Vector Observation: 1

INFO:mlagents.envs:Hyperparameters for the PPO Trainer of brain Ball3DBrain:
INFO:mlagents.envs:Hyperparameters for the PPO Trainer of brain 3DBallLearning:
batch_size: 64
beta: 0.001
buffer_size: 12000

use_curiosity: False
curiosity_strength: 0.01
curiosity_enc_size: 128
model_path: ./models/first-run-0/Ball3DBrain
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 1000. Mean Reward: 1.242. Std of Reward: 0.746. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 2000. Mean Reward: 1.319. Std of Reward: 0.693. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 3000. Mean Reward: 1.804. Std of Reward: 1.056. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 4000. Mean Reward: 2.151. Std of Reward: 1.432. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 5000. Mean Reward: 3.175. Std of Reward: 2.250. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 6000. Mean Reward: 4.898. Std of Reward: 4.019. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 7000. Mean Reward: 6.716. Std of Reward: 5.125. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 8000. Mean Reward: 12.124. Std of Reward: 11.929. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 9000. Mean Reward: 18.151. Std of Reward: 16.871. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 10000. Mean Reward: 27.284. Std of Reward: 28.667. Training.
model_path: ./models/first-run-0/3DBallLearning
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 1000. Mean Reward: 1.242. Std of Reward: 0.746. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 2000. Mean Reward: 1.319. Std of Reward: 0.693. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 3000. Mean Reward: 1.804. Std of Reward: 1.056. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 4000. Mean Reward: 2.151. Std of Reward: 1.432. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 5000. Mean Reward: 3.175. Std of Reward: 2.250. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 6000. Mean Reward: 4.898. Std of Reward: 4.019. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 7000. Mean Reward: 6.716. Std of Reward: 5.125. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 8000. Mean Reward: 12.124. Std of Reward: 11.929. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 9000. Mean Reward: 18.151. Std of Reward: 16.871. Training.
INFO:mlagents.trainers: first-run-0: 3DBallLearning: Step: 10000. Mean Reward: 27.284. Std of Reward: 28.667. Training.
`models/<run-identifier>/editor_<academy_name>_<run-identifier>.bytes` where
`<academy_name>` is the name of the Academy GameObject in the current scene.
`models/<run-identifier>/<brain_name>.tf` where
`<brain_name>` is the name of the Brain corresponding to the model.
(**Note:** There is a known bug on Windows that causes the saving of the model to
fail when you early terminate the training, it's recommended to wait until Step
has reached the max_steps parameter you set in trainer_config.yaml.) This file

1. Move your model file into
`UnitySDK/Assets/ML-Agents/Examples/3DBall/TFModels/`.
2. Open the Unity Editor, and select the **3DBall** scene as described above.
3. Select the **Ball3DBrain** Learning Brain from the Scene hierarchy.
5. Drag the `<env_name>_<run-identifier>.bytes` file from the Project window of
the Editor to the **Graph Model** placeholder in the **Ball3DBrain**
3. Select the **3DBallLearning** Learning Brain from the Scene hierarchy.
5. Drag the `<brain_name>.tf` file from the Project window of
the Editor to the **Model** placeholder in the **3DBallLearning**
inspector window.
6. Press the :arrow_forward: button at the top of the Editor.

32
docs/FAQ.md
查看文件


to [Setting Up The ML-Agents Toolkit Within
Unity](Installation.md#setting-up-ml-agent-within-unity) for solution.
## TensorFlowSharp flag not turned on
If you have already imported the TensorFlowSharp plugin, but haven't set
ENABLE_TENSORFLOW flag for your scripting define symbols, you will see the
following error message:
```console
You need to install and enable the TensorFlowSharp plugin in order to use the Learning Brain.
```
This error message occurs because the TensorFlowSharp plugin won't be usage
without the ENABLE_TENSORFLOW flag, refer to [Setting Up The ML-Agents Toolkit
Within Unity](Installation.md#setting-up-ml-agent-within-unity) for solution.
## Instance of CoreBrainInternal couldn't be created
If you try to use ML-Agents in Unity versions 2017.1 - 2017.3, you might
encounter an error that looks like this:
## Cannot drag Model into Learning Brain
```console
Instance of CoreBrainInternal couldn't be created. The the script
class needs to derive from ScriptableObject.
UnityEngine.ScriptableObject:CreateInstance(String)
```
You migh not have the appropriate backend required to import the model. Refer to the
[Inference Engine](Inference-Engine.md) for more information on how to import backends
and reimport the asset.
You can fix the error by removing `CoreBrain` from CoreBrainInternal.cs:16,
clicking on your Brain Gameobject to let the scene recompile all the changed
C# scripts, then adding the `CoreBrain` back. Make sure your brain is in
Internal mode, your TensorFlowSharp plugin is imported and the
ENABLE_TENSORFLOW flag is set. This fix is only valid locally and unstable.
## Environment Permission Error

16
docs/Getting-Started-with-Balance-Ball.md
查看文件


### Brain
Brains are assets that exist in your project folder. The Ball3DAgents are connected
to a brain, for example : the **Ball3DBrain**.
to a brain, for example : the **3DBallLearning**.
A Brain doesn't store any information about an Agent, it just
routes the Agent's collected observations to the decision making process and
returns the chosen action to the Agent. Thus, all Agents can share the same

use it with Agents having a **Learning Brain**.
__Note:__ Do not just close the Unity Window once the `Saved Model` message appears.
Either wait for the training process to close the window or press Ctrl+C at the
command-line prompt. If you simply close the window manually, the .bytes file
command-line prompt. If you simply close the window manually, the `.tf` file
### Setting up TensorFlowSharp Support
Because TensorFlowSharp support is still experimental, it is disabled by
default. In order to enable it, you must follow these steps. Please note that
the `Learning` Brain mode will only be available once completing these steps.
### Setting up Inference Support
To set up the TensorFlowSharp Support, follow [Setting up ML-Agents Toolkit
within Unity](Basic-Guide.md#setting-up-ml-agents-within-unity) section. of the
Basic Guide page.
In order to run neural network models inside of Unity, you will need to setup the
Inference Engine with an appropriate backend. See [here](Inference-Engine.md) for more
information.
### Embedding the trained model into Unity

16
docs/Learning-Environment-Design-Learning-Brains.md
查看文件


To use a graph model:
1. Select the **Learning Brain** asset in the **Project** window of the Unity Editor.
**Note:** In order to see the **Learning** Brain Type option, you must
[enable TensorFlowSharp](Using-TensorFlow-Sharp-in-Unity.md).
3. Import the `environment_run-id.bytes` file produced by the PPO training
program. (Where `environment_run-id` is the name of the model file, which is
**Note:** In order to use the **Learning** Brain, you have appropriate backend for the
Inference Engine. See [here](Inference-Engine.md).
2. Import the `model_name` file produced by the PPO training
program. (Where `model_name` is the name of the model file, which is
constructed from the name of your Unity environment executable and the run-id
value you assigned when running the training process.)

**Project** window and drop it into an appropriate folder.
4. Once the `environment.bytes` file is imported, drag it from the **Project**
3. Once the `model_name.tf` file is imported, drag it from the **Project**
window to the **Model** field of the Brain component.
If you are using a model produced by the ML-Agents `mlagents-learn` command, use

The default values of the TensorFlow graph parameters work with the model
produced by the PPO and BC training code in the ML-Agents SDK. To use a default
ML-Agents model, the only parameter that you need to set is the `Graph Model`,
which must be set to the .bytes file containing the trained model itself.
ML-Agents model, the only parameter that you need to set is the `Model`,
which must be set to the `.tf` file containing the trained model itself.
* `Model` : This must be the `bytes` file corresponding to the pre-trained
* `Model` : This must be the `.tf` file corresponding to the pre-trained
TensorFlow graph. (You must first drag this file into your Project window
and then from the Resources folder into the inspector)

38
docs/Learning-Environment-Executable.md
查看文件


'Ball3DAcademy' started successfully!
Unity Academy name: Ball3DAcademy
Number of Brains: 1
Number of External Brains : 1
Number of Training Brains : 1
Unity brain name: Ball3DBrain
Unity brain name: Ball3DLearning
Number of Visual Observations (per agent): 0
Vector Observation space size (per agent): 8
Number of stacked Vector Observation: 1

INFO:mlagents.envs:Hyperparameters for the PPO Trainer of brain Ball3DBrain:
INFO:mlagents.envs:Hyperparameters for the PPO Trainer of brain Ball3DLearning:
batch_size: 64
beta: 0.001
buffer_size: 12000

use_curiosity: False
curiosity_strength: 0.01
curiosity_enc_size: 128
model_path: ./models/first-run-0/Ball3DBrain
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 1000. Mean Reward: 1.242. Std of Reward: 0.746. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 2000. Mean Reward: 1.319. Std of Reward: 0.693. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 3000. Mean Reward: 1.804. Std of Reward: 1.056. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 4000. Mean Reward: 2.151. Std of Reward: 1.432. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 5000. Mean Reward: 3.175. Std of Reward: 2.250. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 6000. Mean Reward: 4.898. Std of Reward: 4.019. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 7000. Mean Reward: 6.716. Std of Reward: 5.125. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 8000. Mean Reward: 12.124. Std of Reward: 11.929. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 9000. Mean Reward: 18.151. Std of Reward: 16.871. Training.
INFO:mlagents.trainers: first-run-0: Ball3DBrain: Step: 10000. Mean Reward: 27.284. Std of Reward: 28.667. Training.
model_path: ./models/first-run-0/Ball3DLearning
INFO:mlagents.trainers: first-run-0: Ball3DLearning: Step: 1000. Mean Reward: 1.242. Std of Reward: 0.746. Training.
INFO:mlagents.trainers: first-run-0: Ball3DLearning: Step: 2000. Mean Reward: 1.319. Std of Reward: 0.693. Training.
INFO:mlagents.trainers: first-run-0: Ball3DLearning: Step: 3000. Mean Reward: 1.804. Std of Reward: 1.056. Training.
INFO:mlagents.trainers: first-run-0: Ball3DLearning: Step: 4000. Mean Reward: 2.151. Std of Reward: 1.432. Training.
INFO:mlagents.trainers: first-run-0: Ball3DLearning: Step: 5000. Mean Reward: 3.175. Std of Reward: 2.250. Training.
INFO:mlagents.trainers: first-run-0: Ball3DLearning: Step: 6000. Mean Reward: 4.898. Std of Reward: 4.019. Training.
INFO:mlagents.trainers: first-run-0: Ball3DLearning: Step: 7000. Mean Reward: 6.716. Std of Reward: 5.125. Training.
INFO:mlagents.trainers: first-run-0: Ball3DLearning: Step: 8000. Mean Reward: 12.124. Std of Reward: 11.929. Training.
INFO:mlagents.trainers: first-run-0: Ball3DLearning: Step: 9000. Mean Reward: 18.151. Std of Reward: 16.871. Training.
INFO:mlagents.trainers: first-run-0: Ball3DLearning: Step: 10000. Mean Reward: 27.284. Std of Reward: 28.667. Training.
`models/<run-identifier>/<env_name>_<run-identifier>.bytes`, which corresponds
`models/<run-identifier>/<brain_name>.tf`, which corresponds
to your model's latest checkpoint. (**Note:** There is a known bug on Windows
that causes the saving of the model to fail when you early terminate the
training, it's recommended to wait until Step has reached the max_steps

1. Move your model file into
`UnitySDK/Assets/ML-Agents/Examples/3DBall/TFModels/`.
2. Open the Unity Editor, and select the **3DBall** scene as described above.
3. Select the **Ball3DBrain** object from the Project window.
5. Drag the `<env_name>_<run-identifier>.bytes` file from the Project window of
the Editor to the **Model** placeholder in the **Ball3DBrain**
3. Select the **Ball3DLearning** object from the Project window.
5. Drag the `<brain_name>.tf` file from the Project window of
the Editor to the **Model** placeholder in the **Ball3DLearning**
6. Remove the **Ball3DBrain** from the Academy's `Broadcast Hub`
6. Remove the **Ball3DLearning** from the Academy's `Broadcast Hub`
7. Press the Play button at the top of the editor.

3
docs/Limitations.md
查看文件


### TensorFlow support
Currently the Ml-Agents toolkit uses TensorFlow 1.7.1 due to the version of the
TensorFlowSharp plugin we are using.
Currently the Ml-Agents toolkit uses TensorFlow 1.7.1 only.

5
docs/ML-Agents-Overview.md
查看文件


To summarize: our built-in implementations are based on TensorFlow, thus, during
training the Python API uses the observations it receives to learn a TensorFlow
model. This model is then embedded within the Learning Brain during inference to
generate the optimal actions for all Agents linked to that Brain. **Note that
our Learning Brain is currently experimental as it is limited to TensorFlow
models and leverages the third-party
[TensorFlowSharp](https://github.com/migueldeicaza/TensorFlowSharp) library.**
generate the optimal actions for all Agents linked to that Brain.
The
[Getting Started with the 3D Balance Ball Example](Getting-Started-with-Balance-Ball.md)

4
docs/Readme.md
查看文件


* [Learning Environment Best Practices](Learning-Environment-Best-Practices.md)
* [Using the Monitor](Feature-Monitor.md)
* [Using an Executable Environment](Learning-Environment-Executable.md)
* [TensorFlowSharp in Unity (Experimental)](Using-TensorFlow-Sharp-in-Unity.md)
## Training

* [Training on the Cloud with Amazon Web Services](Training-on-Amazon-Web-Service.md)
* [Training on the Cloud with Microsoft Azure](Training-on-Microsoft-Azure.md)
* [Using TensorBoard to Observe Training](Using-Tensorboard.md)
## Inference
* [Unity Machine Learning Inference SDK](Inference-Engine.md)
## Help

4
docs/Training-Curriculum-Learning.md
查看文件


corresponding Brain. For example, in the Wall Jump environment, there are two
Brains---BigWallBrain and SmallWallBrain. If we want to define a curriculum for
the BigWallBrain, we will save `BigWallBrain.json` into
`curricula/wall-jump/`.
`config/curricula/wall-jump/`.
### Training with a Curriculum

agents in the Wall Jump environment with curriculum learning, we can run
```sh
mlagents-learn config/trainer_config.yaml --curriculum=curricula/wall-jump/ --run-id=wall-jump-curriculum --train
mlagents-learn config/trainer_config.yaml --curriculum=config/curricula/wall-jump/ --run-id=wall-jump-curriculum --train
```
We can then keep track of the current lessons and progresses via TensorBoard.

4
docs/Training-Imitation-Learning.md
查看文件


similarly to the demonstrations.
9. Once the Student Agents are exhibiting the desired behavior, end the training
process with `CTL+C` from the command line.
10. Move the resulting `*.bytes` file into the `TFModels` subdirectory of the
10. Move the resulting `*.tf` file into the `TFModels` subdirectory of the
Assets folder (or a subdirectory within Assets of your choosing) , and use
with `Learning` Brain.

2. Reset the training buffer. This enables you to instruct the agents to forget
their buffer of recent experiences. This is useful if you'd like to get them
to quickly learn a new behavior. The default command to reset the buffer is
to press `C` on the keyboard.
to press `C` on the keyboard.

2
docs/Training-ML-Agents.md
查看文件


When training is finished, you can find the saved model in the `models` folder
under the assigned run-id — in the cats example, the path to the model would be
`models/cob_1/CatsOnBicycles_cob_1.bytes`.
`models/cob_1/CatsOnBicycles_cob_1.tf`.
While this example used the default training hyperparameters, you can edit the
[training_config.yaml file](#training-config-file) with a text editor to set

117
docs/images/imported-tensorflowsharp.png
查看文件

之前 之后

181
docs/Using-TensorFlow-Sharp-in-Unity.md
查看文件


# Using TensorFlowSharp in Unity (Experimental)
The ML-Agents toolkit allows you to use pre-trained
[TensorFlow graphs](https://www.tensorflow.org/programmers_guide/graphs)
inside your Unity
games. This support is possible thanks to the
[TensorFlowSharp project](https://github.com/migueldeicaza/TensorFlowSharp).
The primary purpose for this support is to use the TensorFlow models produced by
the ML-Agents toolkit's own training programs, but a side benefit is that you
can use any TensorFlow model.
_Notice: This feature is still experimental. While it is possible to embed
trained models into Unity games, Unity Technologies does not officially support
this use-case for production games at this time. As such, no guarantees are
provided regarding the quality of experience. If you encounter issues regarding
battery life, or general performance (especially on mobile), please let us
know._
## Supported devices
* Linux 64 bits
* Mac OS X 64 bits
* Windows 64 bits
* iOS (Requires additional steps)
* Android
## Requirements
* Unity 2017.4 or above
* Unity TensorFlow Plugin ([Download here](https://s3.amazonaws.com/unity-ml-agents/0.5/TFSharpPlugin.unitypackage))
## Using TensorFlowSharp with ML-Agents
Go to `Edit` -> `Player Settings` and add `ENABLE_TENSORFLOW` to the `Scripting
Define Symbols` for each type of device you want to use (**`PC, Mac and Linux
Standalone`**, **`iOS`** or **`Android`**).
Set the Brain you used for training to `Learning`. Drag `your_name_graph.bytes`
into Unity and then drag it into The `Model` field in the Brain.
## Using your own trained graphs
The TensorFlow data graphs produced by the ML-Agents training programs work
without any additional settings.
In order to use a TensorFlow data graph in Unity, make sure the nodes of your
graph have appropriate names. You can assign names to nodes in TensorFlow :
```python
variable= tf.identity(variable, name="variable_name")
```
We recommend using the following naming conventions:
* Name the batch size input placeholder `batch_size`
* Name the input vector observation placeholder `state`
* Name the output node `action`
* Name the recurrent vector (memory) input placeholder `recurrent_in` (if any)
* Name the recurrent vector (memory) output node `recurrent_out` (if any)
* Name the observations placeholders input placeholders `visual_observation_i`
where `i` is the index of the observation (starting at 0)
You can have additional placeholders for float or integers but they must be
placed in placeholders of dimension 1 and size 1. (Be sure to name them.)
It is important that the inputs and outputs of the graph are exactly the ones
you receive and return when training your model with an External Brain. This
means you cannot have any operations such as reshaping outside of the graph. The
object you get by calling `step` or `reset` has fields `vector_observations`,
`visual_observations` and `memories` which must correspond to the placeholders
of your graph. Similarly, the arguments `action` and `memory` you pass to `step`
must correspond to the output nodes of your graph.
While training your Agent using the Python API, you can save your graph at any
point of the training. Note that the argument `output_node_names` must be the
name of the tensor your graph outputs (separated by a coma if using multiple
outputs). In this case, it will be either `action` or `action,recurrent_out` if
you have recurrent outputs.
```python
from tensorflow.python.tools import freeze_graph
freeze_graph.freeze_graph(input_graph = model_path +'/raw_graph_def.pb',
input_binary = True,
input_checkpoint = last_checkpoint,
output_node_names = "action",
output_graph = model_path +'/your_name_graph.bytes' ,
clear_devices = True, initializer_nodes = "",input_saver = "",
restore_op_name = "save/restore_all", filename_tensor_name = "save/Const:0")
```
Your model will be saved with the name `your_name_graph.bytes` and will contain
both the graph and associated weights. Note that you must save your graph as a
.bytes file so Unity can load it.
See
[Learning Brain](Learning-Environment-Design-Learning-Brains.md#learning-brain)
for more information about using Learning Brains.
If you followed these instructions well, the Agents in your environment that use
this Brain will use your fully trained network to make decisions.
## iOS additional instructions for building
* Before build your game against iOS platform, make sure you've set the
flag `ENABLE_TENSORFLOW` for it.
* Once you build the project for iOS in the editor, open the .xcodeproj file
within the project folder using Xcode.
* Set up your ios account following the
[iOS Account setup page](https://docs.unity3d.com/Manual/iphone-accountsetup.html).
* In **Build Settings** > **Linking** > **Other Linker Flags**:
* Double click on the flag list to expand the list
* Add `-force_load`
* Drag the library `libtensorflow-core.a` from the **Project Navigator** on
the left under `Libraries/ML-Agents/Plugins/iOS` into the flag list, after
`-force_load`.
## Using TensorFlowSharp without ML-Agents
Beyond controlling an in-game agent, you can also use TensorFlowSharp for more
general computation. The following instructions describe how to generally embed
TensorFlow models without using the ML-Agents framework.
You must have a TensorFlow graph, such as `your_name_graph.bytes`, made using
TensorFlow's `freeze_graph.py`. The process to create such graph is explained in
the [Using your own trained graphs](#using-your-own-trained-graphs) section.
### Inside of Unity
To load and use a TensorFlow data graph in Unity:
1. Put the file, `your_name_graph.bytes`, into Resources.
2. At the top off your C# script, add the line:
```csharp
using TensorFlow;
```
3. If you will be building for android, you must add this block at the start of
your code :
```csharp
#if UNITY_ANDROID && !UNITY_EDITOR
TensorFlowSharp.Android.NativeBinding.Init();
#endif
```
4. Load your graph as a text asset into a variable, such as `graphModel`:
```csharp
TextAsset graphModel = Resources.Load (your_name_graph) as TextAsset;
```
5. You then must instantiate the graph in Unity by adding the code :
```csharp graph = new TFGraph ();
graph.Import (graphModel.bytes);
session = new TFSession (graph);
```
6. Assign the input tensors for the graph. For example, the following code
assigns a one dimensional input tensor of size 2:
```csharp
var runner = session.GetRunner ();
runner.AddInput (graph ["input_placeholder_name"] [0], new float[]{ placeholder_value1, placeholder_value2 });
```
You must provide all required inputs to the graph. Supply one input per
TensorFlow placeholder.
7. To calculate and access the output of your graph, run the following code.
```csharp
runner.Fetch (graph["output_placeholder_name"][0]);
float[,] recurrent_tensor = runner.Run () [0].GetValue () as float[,];
```
Note that this example assumes the output array is a two-dimensional tensor of
floats. Cast to a long array if your outputs are integers.
撰写 预览
正在加载...
取消
保存