浏览代码

Merge pull request #1451 from Unity-Technologies/release-v0.6-revertTF1

Release v0.6 revert tf1
/develop-generalizationTraining-TrainerController
GitHub 6 年前
当前提交
e9121bb5
共有 108 个文件被更改,包括 1283 次插入261 次删除
  1. 2
      UnitySDK/Assets/ML-Agents/Editor/Tests/EditModeTestInternalBrainTensorApplier.cs
  2. 2
      UnitySDK/Assets/ML-Agents/Editor/Tests/EditModeTestInternalBrainTensorGenerator.cs
  3. 2
      UnitySDK/Assets/ML-Agents/Examples/3DBall/Brains/3DBallHardLearning.asset
  4. 2
      UnitySDK/Assets/ML-Agents/Examples/3DBall/Brains/3DBallLearning.asset
  5. 10
      UnitySDK/Assets/ML-Agents/Examples/BananaCollectors/Brains/BananaLearning.asset
  6. 2
      UnitySDK/Assets/ML-Agents/Examples/Basic/Brains/BasicLearning.asset
  7. 10
      UnitySDK/Assets/ML-Agents/Examples/Bouncer/Brains/BouncerLearning.asset
  8. 2
      UnitySDK/Assets/ML-Agents/Examples/Crawler/Brains/CrawlerDynamicLearning.asset
  9. 2
      UnitySDK/Assets/ML-Agents/Examples/Crawler/Brains/CrawlerStaticLearning.asset
  10. 2
      UnitySDK/Assets/ML-Agents/Examples/GridWorld/Brains/GridWorldLearning.asset
  11. 2
      UnitySDK/Assets/ML-Agents/Examples/Hallway/Brains/HallwayLearning.asset
  12. 2
      UnitySDK/Assets/ML-Agents/Examples/PushBlock/Brains/PushBlockLearning.asset
  13. 2
      UnitySDK/Assets/ML-Agents/Examples/Pyramids/Brains/PyramidsLearning.asset
  14. 2
      UnitySDK/Assets/ML-Agents/Examples/Reacher/Brains/ReacherLearning.asset
  15. 2
      UnitySDK/Assets/ML-Agents/Examples/Soccer/Brains/GoalieLearning.asset
  16. 2
      UnitySDK/Assets/ML-Agents/Examples/Soccer/Brains/StrikerLearning.asset
  17. 2
      UnitySDK/Assets/ML-Agents/Examples/Tennis/Brains/TennisLearning.asset
  18. 10
      UnitySDK/Assets/ML-Agents/Examples/Walker/Brains/WalkerLearning.asset
  19. 2
      UnitySDK/Assets/ML-Agents/Examples/WallJump/Brains/BigWallJumpLearning.asset
  20. 2
      UnitySDK/Assets/ML-Agents/Examples/WallJump/Brains/SmallWallJumpLearning.asset
  21. 6
      UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/ApplierImpl.cs
  22. 5
      UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/GeneratorImpl.cs
  23. 12
      UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/ModelParamLoader.cs
  24. 5
      UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/TensorApplier.cs
  25. 6
      UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/TensorGenerator.cs
  26. 36
      UnitySDK/Assets/ML-Agents/Scripts/LearningBrain.cs
  27. 19
      docs/Background-TensorFlow.md
  28. 38
      docs/Basic-Guide.md
  29. 14
      docs/FAQ.md
  30. 14
      docs/Getting-Started-with-Balance-Ball.md
  31. 10
      docs/Learning-Environment-Design-Learning-Brains.md
  32. 4
      docs/Learning-Environment-Executable.md
  33. 4
      docs/ML-Agents-Overview.md
  34. 12
      docs/Migrating.md
  35. 2
      docs/Readme.md
  36. 2
      docs/Training-Imitation-Learning.md
  37. 2
      docs/Training-ML-Agents.md
  38. 2
      ml-agents/mlagents/trainers/policy.py
  39. 224
      UnitySDK/Assets/ML-Agents/Editor/Tests/MultinomialTest.cs
  40. 122
      UnitySDK/Assets/ML-Agents/Editor/Tests/RandomNormalTest.cs
  41. 7
      UnitySDK/Assets/ML-Agents/Examples/3DBall/TFModels/3DBallHardLearning.bytes.meta
  42. 7
      UnitySDK/Assets/ML-Agents/Examples/3DBall/TFModels/3DBallLearning.bytes.meta
  43. 7
      UnitySDK/Assets/ML-Agents/Examples/BananaCollectors/TFModels/BananaLearning.bytes.meta
  44. 7
      UnitySDK/Assets/ML-Agents/Examples/Basic/TFModels/BasicLearning.bytes.meta
  45. 7
      UnitySDK/Assets/ML-Agents/Examples/Bouncer/TFModels/BouncerLearning.bytes.meta
  46. 7
      UnitySDK/Assets/ML-Agents/Examples/Crawler/TFModels/CrawlerDynamicLearning.bytes.meta
  47. 7
      UnitySDK/Assets/ML-Agents/Examples/Crawler/TFModels/CrawlerStaticLearning.bytes.meta
  48. 7
      UnitySDK/Assets/ML-Agents/Examples/GridWorld/TFModels/GridWorldLearning.bytes.meta
  49. 7
      UnitySDK/Assets/ML-Agents/Examples/Hallway/TFModels/HallwayLearning.bytes.meta
  50. 7
      UnitySDK/Assets/ML-Agents/Examples/PushBlock/TFModels/PushBlockLearning.bytes.meta
  51. 7
      UnitySDK/Assets/ML-Agents/Examples/Pyramids/TFModels/PyramidsLearning.bytes.meta
  52. 7
      UnitySDK/Assets/ML-Agents/Examples/Reacher/TFModels/ReacherLearning.bytes.meta
  53. 7
      UnitySDK/Assets/ML-Agents/Examples/Soccer/TFModels/GoalieLearning.bytes.meta
  54. 7
      UnitySDK/Assets/ML-Agents/Examples/Soccer/TFModels/StrikerLearning.bytes.meta
  55. 7
      UnitySDK/Assets/ML-Agents/Examples/Tennis/TFModels/TennisLearning.bytes.meta
  56. 7
      UnitySDK/Assets/ML-Agents/Examples/Walker/TFModels/WalkerLearning.bytes.meta
  57. 7
      UnitySDK/Assets/ML-Agents/Examples/WallJump/TFModels/BigWallJumpLearning.bytes.meta
  58. 7
      UnitySDK/Assets/ML-Agents/Examples/WallJump/TFModels/SmallWallJumpLearning.bytes.meta
  59. 213
      UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/TFSharpInferenceEngine.cs
  60. 12
      UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/TFSharpInferenceEngine.cs.meta
  61. 40
      UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/Tensor.cs
  62. 11
      UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/Tensor.cs.meta
  63. 8
      UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/Utils.meta
  64. 55
      docs/Using-TensorFlow-Sharp-in-Unity.md
  65. 117
      docs/images/imported-tensorflowsharp.png
  66. 98
      UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/Utils/Multinomial.cs
  67. 11
      UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/Utils/Multinomial.cs.meta
  68. 105
      UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/Utils/RandomNormal.cs
  69. 11
      UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/Utils/RandomNormal.cs.meta
  70. 7
      UnitySDK/Assets/ML-Agents/Examples/3DBall/TFModels/3DBallLearning.tf.meta
  71. 7
      UnitySDK/Assets/ML-Agents/Examples/3DBall/TFModels/3DBallHardLearning.tf.meta
  72. 7
      UnitySDK/Assets/ML-Agents/Examples/BananaCollectors/TFModels/BananaLearning.tf.meta
  73. 7
      UnitySDK/Assets/ML-Agents/Examples/Basic/TFModels/BasicLearning.tf.meta
  74. 7
      UnitySDK/Assets/ML-Agents/Examples/Bouncer/TFModels/BouncerLearning.tf.meta
  75. 7
      UnitySDK/Assets/ML-Agents/Examples/Crawler/TFModels/CrawlerDynamicLearning.tf.meta
  76. 7
      UnitySDK/Assets/ML-Agents/Examples/Crawler/TFModels/CrawlerStaticLearning.tf.meta
  77. 7
      UnitySDK/Assets/ML-Agents/Examples/GridWorld/TFModels/GridWorldLearning.tf.meta
  78. 7
      UnitySDK/Assets/ML-Agents/Examples/Hallway/TFModels/HallwayLearning.tf.meta
  79. 7
      UnitySDK/Assets/ML-Agents/Examples/PushBlock/TFModels/PushBlockLearning.tf.meta
  80. 7
      UnitySDK/Assets/ML-Agents/Examples/Pyramids/TFModels/PyramidsLearning.tf.meta
  81. 7
      UnitySDK/Assets/ML-Agents/Examples/Reacher/TFModels/ReacherLearning.tf.meta
  82. 7
      UnitySDK/Assets/ML-Agents/Examples/Soccer/TFModels/GoalieLearning.tf.meta
  83. 7
      UnitySDK/Assets/ML-Agents/Examples/Soccer/TFModels/StrikerLearning.tf.meta
  84. 7
      UnitySDK/Assets/ML-Agents/Examples/Tennis/TFModels/TennisLearning.tf.meta
  85. 7
      UnitySDK/Assets/ML-Agents/Examples/Walker/TFModels/WalkerLearning.tf.meta
  86. 7
      UnitySDK/Assets/ML-Agents/Examples/WallJump/TFModels/BigWallJumpLearning.tf.meta
  87. 7
      UnitySDK/Assets/ML-Agents/Examples/WallJump/TFModels/SmallWallJumpLearning.tf.meta
  88. 8
      UnitySDK/Assets/ML-Agents/Plugins/InferenceEngine.meta
  89. 0
      /UnitySDK/Assets/ML-Agents/Examples/3DBall/TFModels/3DBallLearning.bytes
  90. 0
      /UnitySDK/Assets/ML-Agents/Examples/3DBall/TFModels/3DBallHardLearning.bytes
  91. 0
      /UnitySDK/Assets/ML-Agents/Examples/BananaCollectors/TFModels/BananaLearning.bytes
  92. 0
      /UnitySDK/Assets/ML-Agents/Examples/Basic/TFModels/BasicLearning.bytes
  93. 0
      /UnitySDK/Assets/ML-Agents/Examples/Bouncer/TFModels/BouncerLearning.bytes
  94. 0
      /UnitySDK/Assets/ML-Agents/Examples/Crawler/TFModels/CrawlerStaticLearning.bytes
  95. 0
      /UnitySDK/Assets/ML-Agents/Examples/Crawler/TFModels/CrawlerDynamicLearning.bytes
  96. 0
      /UnitySDK/Assets/ML-Agents/Examples/GridWorld/TFModels/GridWorldLearning.bytes
  97. 0
      /UnitySDK/Assets/ML-Agents/Examples/Hallway/TFModels/HallwayLearning.bytes
  98. 0
      /UnitySDK/Assets/ML-Agents/Examples/PushBlock/TFModels/PushBlockLearning.bytes

2
UnitySDK/Assets/ML-Agents/Editor/Tests/EditModeTestInternalBrainTensorApplier.cs


using System.Linq;
using NUnit.Framework;
using UnityEngine;
using UnityEngine.MachineLearning.InferenceEngine;
using UnityEngine.MachineLearning.InferenceEngine.Util;
using System.Reflection;
using MLAgents.InferenceBrain;

2
UnitySDK/Assets/ML-Agents/Editor/Tests/EditModeTestInternalBrainTensorGenerator.cs


using System.Linq;
using NUnit.Framework;
using UnityEngine;
using UnityEngine.MachineLearning.InferenceEngine;
using UnityEngine.MachineLearning.InferenceEngine.Util;
using MLAgents.InferenceBrain;
namespace MLAgents.Tests

2
UnitySDK/Assets/ML-Agents/Examples/3DBall/Brains/3DBallHardLearning.asset


-
-
vectorActionSpaceType: 1
model: {fileID: 11400000, guid: c2381a94783b14d5fa1c56dc947a9578, type: 3}
model: {fileID: 4900000, guid: 8a2da2218425f46e9921caefda4b7813, type: 3}

2
UnitySDK/Assets/ML-Agents/Examples/3DBall/Brains/3DBallLearning.asset


-
-
vectorActionSpaceType: 1
model: {fileID: 0}
model: {fileID: 4900000, guid: 9f58800fa9d54477aa01ee258842f6b3, type: 3}

10
UnitySDK/Assets/ML-Agents/Examples/BananaCollectors/Brains/BananaLearning.asset


-
-
vectorActionSpaceType: 0
graphModel: {fileID: 0}
graphPlaceholders: []
BatchSizePlaceholderName: batch_size
VectorObservationPlacholderName: vector_observation
RecurrentInPlaceholderName: recurrent_in
RecurrentOutPlaceholderName: recurrent_out
VisualObservationPlaceholderName: []
ActionPlaceholderName: action
PreviousActionPlaceholderName: prev_action
model: {fileID: 4900000, guid: 69bd818d72b944849916d2fda9fe471b, type: 3}

2
UnitySDK/Assets/ML-Agents/Examples/Basic/Brains/BasicLearning.asset


vectorActionDescriptions:
-
vectorActionSpaceType: 0
model: {fileID: 0}
model: {fileID: 4900000, guid: 503ce1e8257904bd0b5be8f7fb4b5d28, type: 3}

10
UnitySDK/Assets/ML-Agents/Examples/Bouncer/Brains/BouncerLearning.asset


-
-
vectorActionSpaceType: 1
graphModel: {fileID: 4900000, guid: f6197a66bd242464e8b654ef1ac01faf, type: 3}
graphPlaceholders: []
BatchSizePlaceholderName: batch_size
VectorObservationPlacholderName: vector_observation
RecurrentInPlaceholderName: recurrent_in
RecurrentOutPlaceholderName: recurrent_out
VisualObservationPlaceholderName: []
ActionPlaceholderName: action
PreviousActionPlaceholderName: prev_action
model: {fileID: 4900000, guid: 760d2b8347b4b46e3a44d9b989e1304e, type: 3}

2
UnitySDK/Assets/ML-Agents/Examples/Crawler/Brains/CrawlerDynamicLearning.asset


-
-
vectorActionSpaceType: 1
model: {fileID: 0}
model: {fileID: 4900000, guid: 9482a8782450a4d87b20942c4523176b, type: 3}

2
UnitySDK/Assets/ML-Agents/Examples/Crawler/Brains/CrawlerStaticLearning.asset


-
-
vectorActionSpaceType: 1
model: {fileID: 0}
model: {fileID: 4900000, guid: e256bd37f98f246e5be72618766d0a93, type: 3}

2
UnitySDK/Assets/ML-Agents/Examples/GridWorld/Brains/GridWorldLearning.asset


vectorActionDescriptions:
-
vectorActionSpaceType: 0
model: {fileID: 0}
model: {fileID: 4900000, guid: 0fd168a0ea1d04ef9a68c80cf452ce3d, type: 3}

2
UnitySDK/Assets/ML-Agents/Examples/Hallway/Brains/HallwayLearning.asset


vectorActionDescriptions:
-
vectorActionSpaceType: 0
model: {fileID: 0}
model: {fileID: 4900000, guid: 84588668e6ea948d3ab55bb813cc769b, type: 3}

2
UnitySDK/Assets/ML-Agents/Examples/PushBlock/Brains/PushBlockLearning.asset


vectorActionDescriptions:
-
vectorActionSpaceType: 0
model: {fileID: 0}
model: {fileID: 4900000, guid: e22850d2072904a0ab06069cda2599e5, type: 3}

2
UnitySDK/Assets/ML-Agents/Examples/Pyramids/Brains/PyramidsLearning.asset


vectorActionDescriptions:
-
vectorActionSpaceType: 0
model: {fileID: 0}
model: {fileID: 4900000, guid: 7d1c7f27447234c3a81169de00dcaa8a, type: 3}

2
UnitySDK/Assets/ML-Agents/Examples/Reacher/Brains/ReacherLearning.asset


-
-
vectorActionSpaceType: 1
model: {fileID: 0}
model: {fileID: 4900000, guid: 5fb4a3624e9ca4e1c81b51b5117cb31e, type: 3}

2
UnitySDK/Assets/ML-Agents/Examples/Soccer/Brains/GoalieLearning.asset


vectorActionDescriptions:
-
vectorActionSpaceType: 0
model: {fileID: 0}
model: {fileID: 4900000, guid: 890ab8f03425c4a80a52ba674ddec3f3, type: 3}

2
UnitySDK/Assets/ML-Agents/Examples/Soccer/Brains/StrikerLearning.asset


vectorActionDescriptions:
-
vectorActionSpaceType: 0
model: {fileID: 0}
model: {fileID: 4900000, guid: 23410257d39d44616bfefdff59c7fbc9, type: 3}

2
UnitySDK/Assets/ML-Agents/Examples/Tennis/Brains/TennisLearning.asset


-
-
vectorActionSpaceType: 1
model: {fileID: 0}
model: {fileID: 4900000, guid: 6d4281b70d41f48cb83d663b84f78c9a, type: 3}

10
UnitySDK/Assets/ML-Agents/Examples/Walker/Brains/WalkerLearning.asset


-
-
vectorActionSpaceType: 1
graphModel: {fileID: 0}
graphPlaceholders: []
BatchSizePlaceholderName: batch_size
VectorObservationPlacholderName: vector_observation
RecurrentInPlaceholderName: recurrent_in
RecurrentOutPlaceholderName: recurrent_out
VisualObservationPlaceholderName: []
ActionPlaceholderName: action
PreviousActionPlaceholderName: prev_action
model: {fileID: 4900000, guid: 48ab33cf9fbee4883948187618027835, type: 3}

2
UnitySDK/Assets/ML-Agents/Examples/WallJump/Brains/BigWallJumpLearning.asset


-
-
vectorActionSpaceType: 0
model: {fileID: 0}
model: {fileID: 4900000, guid: c118879bb5db84f269e4da23ba8c4f61, type: 3}

2
UnitySDK/Assets/ML-Agents/Examples/WallJump/Brains/SmallWallJumpLearning.asset


-
-
vectorActionSpaceType: 0
model: {fileID: 0}
model: {fileID: 4900000, guid: 92cd96b2c34334db692e93af25b64d2a, type: 3}

6
UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/ApplierImpl.cs


using UnityEngine.MachineLearning.InferenceEngine;
using System.Collections.Generic;
using UnityEngine;
using UnityEngine.MachineLearning.InferenceEngine.Util;
using System.Collections.Generic;
using MLAgents.InferenceBrain.Utils;
namespace MLAgents.InferenceBrain
{

5
UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/GeneratorImpl.cs


using UnityEngine.MachineLearning.InferenceEngine;
using System.Collections.Generic;
using System.Collections.Generic;
using UnityEngine.MachineLearning.InferenceEngine.Util;
using MLAgents.InferenceBrain.Utils;
namespace MLAgents.InferenceBrain
{

12
UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/ModelParamLoader.cs


using UnityEngine.MachineLearning.InferenceEngine;
using UnityEngine;
#if ENABLE_TENSORFLOW
using System;
using System.Collections.Generic;
using System.Linq;

Continuous
}
private const long ApiVersion = 1;
private InferenceEngine _engine;
private TFSharpInferenceEngine _engine;
private BrainParameters _brainParameters;
private List<string> _failedModelChecks = new List<string>();

/// <param name="brainParameters"> The BrainParamters that are used verify the
/// compatibility with the InferenceEngine</param>
/// <returns></returns>
public static ModelParamLoader GetLoaderAndCheck(InferenceEngine engine,
public static ModelParamLoader GetLoaderAndCheck(TFSharpInferenceEngine engine,
BrainParameters brainParameters)
{
ModelParamLoader modelParamLoader = new ModelParamLoader(engine, brainParameters);

private ModelParamLoader(InferenceEngine engine, BrainParameters brainParameters)
private ModelParamLoader(TFSharpInferenceEngine engine, BrainParameters brainParameters)
{
_engine = engine;
_brainParameters = brainParameters;

{
_engine.ExecuteGraph(new Tensor[0], outputs);
}
catch (Exception e)
catch
{
return -1;
}

}
}
}
#endif

5
UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/TensorApplier.cs


using UnityEngine.MachineLearning.InferenceEngine;
using System.Collections.Generic;
using UnityEngine.MachineLearning.InferenceEngine.Util;
using System;
using System.Collections.Generic;
namespace MLAgents.InferenceBrain
{

6
UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/TensorGenerator.cs


using UnityEngine.MachineLearning.InferenceEngine;
using System.Collections.Generic;
using UnityEngine.MachineLearning.InferenceEngine.Util;
using System.Linq;
using System;
using System.Collections.Generic;
namespace MLAgents.InferenceBrain
{

36
UnitySDK/Assets/ML-Agents/Scripts/LearningBrain.cs


using UnityEngine;
using System.Linq;
using MLAgents.InferenceBrain;
using UnityEngine.MachineLearning.InferenceEngine;
using UnityEngine.Profiling;
namespace MLAgents

{
private TensorGenerator _tensorGenerator;
private TensorApplier _tensorApplier;
#if ENABLE_TENSORFLOW
public Model model;
#endif
public TextAsset model;
private InferenceEngine _engine;
#if ENABLE_TENSORFLOW
private TFSharpInferenceEngine _engine;
#endif
private IEnumerable<Tensor> _inferenceInputs;
private IEnumerable<Tensor> _inferenceOutputs;

/// </exception>
public void ReloadModel(int seed = 0)
{
#if ENABLE_TENSORFLOW
var config = new InferenceEngineConfig
{
Device = InferenceEngineConfig.DeviceType.CPU
};
_engine = InferenceAPI.LoadModel(model, config);
_engine = new TFSharpInferenceEngine();
_engine.PrepareModel(model.bytes);
}
else
{

_inferenceOutputs = _modelParamLoader.GetOutputTensors();
_tensorGenerator = new TensorGenerator(brainParameters, seed);
_tensorApplier = new TensorApplier(brainParameters, seed);
#endif
}
/// <summary>

/// Brain Parameters</returns>
public IEnumerable<string> GetModelFailedChecks()
{
#if ENABLE_TENSORFLOW
#else
return new List<string>(){
"You need to install the TensorflowSharp plugin and add the ENABLE_TENSORFLOW " +
"flag in your Player Settings in order to use inference. "};
#endif
}
/// <inheritdoc />

{
return;
}
#if ENABLE_TENSORFLOW
if (_engine == null)
{
Debug.LogError($"No model was present for the Brain {name}.");

// Update the outputs
_tensorApplier.ApplyTensors(_inferenceOutputs, agentInfos);
#else
if (agentInfos.Count > 0)
{
Debug.LogError(string.Format(
"The brain {0} was set to inference mode but the Tensorflow library is not " +
"present in the Unity project.",
name));
}
#endif
agentInfos.Clear();
}
}

19
docs/Background-TensorFlow.md


performing computations using data flow graphs, the underlying representation of
deep learning models. It facilitates training and inference on CPUs and GPUs in
a desktop, server, or mobile device. Within the ML-Agents toolkit, when you
train the behavior of an agent, the output is a TensorFlow model (.tf) file
train the behavior of an agent, the output is a TensorFlow model (.bytes) file
that you can then embed within a Learning Brain. Unless you implement a new
algorithm, the use of TensorFlow is mostly abstracted away and behind the
scenes.

recommend our guide on [using Tensorboard with ML-Agents](Using-Tensorboard.md) or
this [tutorial](https://github.com/dandelionmane/tf-dev-summit-tensorboard-tutorial).
## Tensorflow Model Inference
## TensorflowSharp
We have are using the [Unity Machine Learning Inference SDK](Inference-Engine.md) to
run the models inside of Unity. In order to use it, you will need to have an
appropriate backend downloaded.
This means that the Learning Brain is not natively supported since Unity scripts
are written in C#. Consequently, to enable the Learning Brain, we leverage a
third-party library
[TensorFlowSharp](https://github.com/migueldeicaza/TensorFlowSharp) which
provides .NET bindings to TensorFlow. Thus, when a Unity environment that
contains a Learning Brain is built, inference is performed via TensorFlowSharp.
We provide an additional in-depth overview of how to leverage
[TensorFlowSharp within Unity](Using-TensorFlow-Sharp-in-Unity.md)
which will become more
relevant once you install and start training behaviors within the ML-Agents
toolkit. Given the reliance on TensorFlowSharp, the Learning Brain is currently
marked as experimental.

38
docs/Basic-Guide.md


Equivalent or .NET 4.x Equivalent)**
6. Go to **File** > **Save Project**
## Setting up the Inference Engine
## Setting up TensorFlowSharp
We provide pre-trained models for all the agents in all our demo environments.
To be able to run those models, you'll first need to set-up the Inference
Engine. The Inference Engine is a general API to
run neural network models in Unity that leverages existing inference libraries such
as TensorFlowSharp and Apple's Core ML. Since the ML-Agents Toolkit uses TensorFlow
for training neural network models, the output model format is TensorFlow and
the model files include a `.tf` extension. Consequently, you need to install
the TensorFlowSharp backend to be able to run these models within the Unity
Editor. You can find instructions
on how to install the TensorFlowSharp backend [here](Inference-Engine.md).
Once the backend is installed, you will need to reimport the models : Right click
on the `.tf` model and select `Reimport`.
We provide pre-trained models (`.bytes` files) for all the agents
in all our demo environments. To be able to run those models, you'll
first need to set-up TensorFlowSharp support. Consequently, you need to install
the TensorFlowSharp plugin to be able to run these models within the Unity
Editor.
1. Download the [TensorFlowSharp Plugin](https://s3.amazonaws.com/unity-ml-agents/0.5/TFSharpPlugin.unitypackage)
2. Import it into Unity by double clicking the downloaded file. You can check
if it was successfully imported by checking the
TensorFlow files in the Project window under **Assets** > **ML-Agents** >
**Plugins** > **Computer**.
3. Go to **Edit** > **Project Settings** > **Player** and add `ENABLE_TENSORFLOW`
to the `Scripting Define Symbols` for each type of device you want to use
(**`PC, Mac and Linux Standalone`**, **`iOS`** or **`Android`**).
![Project Settings](images/project-settings.png)
**Note**: If you don't see anything under **Assets**, drag the
`UnitySDK/Assets/ML-Agents` folder under **Assets** within Project window.
![Imported TensorFlowsharp](images/imported-tensorflowsharp.png)
## Running a Pre-trained Model

### After training
You can press Ctrl+C to stop the training, and your trained model will be at
`models/<run-identifier>/<brain_name>.tf` where
`models/<run-identifier>/<brain_name>.bytes` where
`<brain_name>` is the name of the Brain corresponding to the model.
(**Note:** There is a known bug on Windows that causes the saving of the model to
fail when you early terminate the training, it's recommended to wait until Step

`UnitySDK/Assets/ML-Agents/Examples/3DBall/TFModels/`.
2. Open the Unity Editor, and select the **3DBall** scene as described above.
3. Select the **3DBallLearning** Learning Brain from the Scene hierarchy.
5. Drag the `<brain_name>.tf` file from the Project window of
5. Drag the `<brain_name>.bytes` file from the Project window of
the Editor to the **Model** placeholder in the **3DBallLearning**
inspector window.
6. Press the :arrow_forward: button at the top of the Editor.

14
docs/FAQ.md


to [Setting Up The ML-Agents Toolkit Within
Unity](Installation.md#setting-up-ml-agent-within-unity) for solution.
## Cannot drag Model into Learning Brain
## TensorFlowSharp flag not turned on
You might not have the appropriate backend required to import the model. Refer to the
[Inference Engine](Inference-Engine.md) for more information on how to import backends
and reimport the asset.
If you have already imported the TensorFlowSharp plugin, but haven't set
ENABLE_TENSORFLOW flag for your scripting define symbols, you will see the
following error message:
```console
UnityAgentsException: The brain 3DBallLearning was set to inference mode but the Tensorflow library is not present in the Unity project.
```
This error message occurs because the TensorFlowSharp plugin won't be used
without the ENABLE_TENSORFLOW flag, refer to [Setting Up The ML-Agents Toolkit
Within Unity](Installation.md#setting-up-ml-agent-within-unity) for solution.
## Environment Permission Error

14
docs/Getting-Started-with-Balance-Ball.md


use it with Agents having a **Learning Brain**.
__Note:__ Do not just close the Unity Window once the `Saved Model` message appears.
Either wait for the training process to close the window or press Ctrl+C at the
command-line prompt. If you simply close the window manually, the `.tf` file
command-line prompt. If you close the window manually, the `.bytes` file
### Setting up Inference Support
### Setting up TensorFlowSharp
In order to run neural network models inside of Unity, you will need to setup the
Inference Engine with an appropriate backend. See [here](Inference-Engine.md) for more
information.
Because TensorFlowSharp support is still experimental, it is disabled by
default. Please note that the `Learning` Brain inference can only be used with
TensorFlowSharp.
To set up the TensorFlowSharp Support, follow [Setting up ML-Agents Toolkit
within Unity](Basic-Guide.md#setting-up-ml-agents-within-unity) section. of the
Basic Guide page.
### Embedding the trained model into Unity

10
docs/Learning-Environment-Design-Learning-Brains.md


To use a graph model:
1. Select the **Learning Brain** asset in the **Project** window of the Unity Editor.
**Note:** In order to use the **Learning** Brain, you have appropriate backend for the
Inference Engine. See [here](Inference-Engine.md).
**Note:** In order to use the **Learning** Brain with inference, you need to have
TensorFlowSharp enabled. Refer to [this section](Basic-Guide.md#setting-up-ml-agents-within-unity) for more information.
2. Import the `model_name` file produced by the PPO training
program. (Where `model_name` is the name of the model file, which is
constructed from the name of your Unity environment executable and the run-id

[import assets into Unity](https://docs.unity3d.com/Manual/ImportingAssets.html)
in various ways. The easiest way is to simply drag the file into the
**Project** window and drop it into an appropriate folder.
3. Once the `model_name.tf` file is imported, drag it from the **Project**
3. Once the `model_name.bytes` file is imported, drag it from the **Project**
window to the **Model** field of the Brain component.
If you are using a model produced by the ML-Agents `mlagents-learn` command, use

The default values of the TensorFlow graph parameters work with the model
produced by the PPO and BC training code in the ML-Agents SDK. To use a default
ML-Agents model, the only parameter that you need to set is the `Model`,
which must be set to the `.tf` file containing the trained model itself.
which must be set to the `.bytes` file containing the trained model itself.
* `Model` : This must be the `.tf` file corresponding to the pre-trained
* `Model` : This must be the `.bytes` file corresponding to the pre-trained
TensorFlow graph. (You must first drag this file into your Project window
and then from the Resources folder into the inspector)

4
docs/Learning-Environment-Executable.md


```
You can press Ctrl+C to stop the training, and your trained model will be at
`models/<run-identifier>/<brain_name>.tf`, which corresponds
`models/<run-identifier>/<brain_name>.bytes`, which corresponds
to your model's latest checkpoint. (**Note:** There is a known bug on Windows
that causes the saving of the model to fail when you early terminate the
training, it's recommended to wait until Step has reached the max_steps

`UnitySDK/Assets/ML-Agents/Examples/3DBall/TFModels/`.
2. Open the Unity Editor, and select the **3DBall** scene as described above.
3. Select the **Ball3DLearning** object from the Project window.
5. Drag the `<brain_name>.tf` file from the Project window of
5. Drag the `<brain_name>.bytes` file from the Project window of
the Editor to the **Model** placeholder in the **Ball3DLearning**
inspector window.
6. Remove the **Ball3DLearning** from the Academy's `Broadcast Hub`

4
docs/ML-Agents-Overview.md


model. This model is then embedded within the Learning Brain during inference to
generate the optimal actions for all Agents linked to that Brain.
**Note that our Learning Brain is currently experimental as it is limited to TensorFlow
models and leverages the third-party
[TensorFlowSharp](https://github.com/migueldeicaza/TensorFlowSharp) library.**
The
[Getting Started with the 3D Balance Ball Example](Getting-Started-with-Balance-Ball.md)
tutorial covers this training mode with the **3D Balance Ball** sample environment.

12
docs/Migrating.md


* When training multiple Brains at the same time, each model is now stored
into a separate model file rather than in the same file under different
graph scopes.
* We have changed the way ML-Agents models perform inference. All previous `.bytes`
files can no longer be used (you will have to retrain them). The models
produced by the training process and the shipped models have now a `.tf`
extension and use TensorflowSharp as a backend for the
[Inference Engine](Inference-Engine.md).
* To use a `.tf` model, drag it inside the `Model` property of the `Learning Brain`
* The **Learning Brain** graph scope, placeholder names, output names and custom
placeholders can no longer be modified.
#### Steps to Migrate

from the `mlagents-learn` Python script, you need to drag the relevant
`LearningBrain` ScriptableObjects used in your scene into entries
into this list.
* You will need to delete the previous TensorFlowSharp package
and install the new one to do inference. To correctly delete the previous
TensorFlowSharp package, Delete all of the files under `ML-Agents/Plugins`
folder except the files under `ML-Agents/Plugins/ProtoBuffer`.
## Migrating from ML-Agents toolkit v0.4 to v0.5

2
docs/Readme.md


* [Using TensorBoard to Observe Training](Using-Tensorboard.md)
## Inference
* [Unity Machine Learning Inference SDK](Inference-Engine.md)
* [TensorFlowSharp in Unity (Experimental)](Using-TensorFlow-Sharp-in-Unity.md)
## Help

2
docs/Training-Imitation-Learning.md


similarly to the demonstrations.
9. Once the Student Agents are exhibiting the desired behavior, end the training
process with `CTL+C` from the command line.
10. Move the resulting `*.tf` file into the `TFModels` subdirectory of the
10. Move the resulting `*.bytes` file into the `TFModels` subdirectory of the
Assets folder (or a subdirectory within Assets of your choosing) , and use
with `Learning` Brain.

2
docs/Training-ML-Agents.md


When training is finished, you can find the saved model in the `models` folder
under the assigned run-id — in the cats example, the path to the model would be
`models/cob_1/CatsOnBicycles_cob_1.tf`.
`models/cob_1/CatsOnBicycles_cob_1.bytes`.
While this example used the default training hyperparameters, you can edit the
[training_config.yaml file](#training-config-file) with a text editor to set

2
ml-agents/mlagents/trainers/policy.py


input_binary=True,
input_checkpoint=ckpt.model_checkpoint_path,
output_node_names=target_nodes,
output_graph=(self.model_path + '.tf'),
output_graph=(self.model_path + '.bytes'),
clear_devices=True, initializer_nodes='', input_saver='',
restore_op_name='save/restore_all',
filename_tensor_name='save/Const:0')

224
UnitySDK/Assets/ML-Agents/Editor/Tests/MultinomialTest.cs


using System;
using NUnit.Framework;
using UnityEngine;
using MLAgents.InferenceBrain;
using MLAgents.InferenceBrain.Utils;
namespace MLAgents.Tests
{
public class MultinomialTest
{
[Test]
public void TestEvalP()
{
Multinomial m = new Multinomial(2018);
Tensor src = new Tensor
{
Data = new float[1, 3] {{0.1f, 0.2f, 0.7f}},
ValueType = Tensor.TensorType.FloatingPoint
};
Tensor dst = new Tensor
{
Data = new float[1, 3],
ValueType = Tensor.TensorType.FloatingPoint
};
m.Eval(src, dst);
float[] reference = {2, 2, 1};
int i = 0;
foreach (var f in dst.Data)
{
Assert.AreEqual(reference[i], f);
++i;
}
}
[Test]
public void TestEvalLogits()
{
Multinomial m = new Multinomial(2018);
Tensor src = new Tensor
{
Data = new float[1, 3] {{Mathf.Log(0.1f) - 50, Mathf.Log(0.2f) - 50, Mathf.Log(0.7f) - 50}},
ValueType = Tensor.TensorType.FloatingPoint
};
Tensor dst = new Tensor
{
Data = new float[1, 3],
ValueType = Tensor.TensorType.FloatingPoint
};
m.Eval(src, dst);
float[] reference = {2, 2, 2};
int i = 0;
foreach (var f in dst.Data)
{
Assert.AreEqual(reference[i], f);
++i;
}
}
[Test]
public void TestEvalBatching()
{
Multinomial m = new Multinomial(2018);
Tensor src = new Tensor
{
Data = new float[2, 3]
{
{Mathf.Log(0.1f) - 50, Mathf.Log(0.2f) - 50, Mathf.Log(0.7f) - 50},
{Mathf.Log(0.3f) - 25, Mathf.Log(0.4f) - 25, Mathf.Log(0.3f) - 25},
},
ValueType = Tensor.TensorType.FloatingPoint
};
Tensor dst = new Tensor
{
Data = new float[2, 3],
ValueType = Tensor.TensorType.FloatingPoint
};
m.Eval(src, dst);
float[] reference = {2, 2, 2, 0, 1, 0};
int i = 0;
foreach (var f in dst.Data)
{
Assert.AreEqual(reference[i], f);
++i;
}
}
[Test]
public void TestSrcInt()
{
Multinomial m = new Multinomial(2018);
Tensor src = new Tensor
{
ValueType = Tensor.TensorType.Integer
};
Assert.Throws<NotImplementedException>(() => m.Eval(src, null));
}
[Test]
public void TestDstInt()
{
Multinomial m = new Multinomial(2018);
Tensor src = new Tensor
{
ValueType = Tensor.TensorType.FloatingPoint
};
Tensor dst = new Tensor
{
ValueType = Tensor.TensorType.Integer
};
Assert.Throws<ArgumentException>(() => m.Eval(src, dst));
}
[Test]
public void TestSrcDataNull()
{
Multinomial m = new Multinomial(2018);
Tensor src = new Tensor
{
ValueType = Tensor.TensorType.FloatingPoint
};
Tensor dst = new Tensor
{
ValueType = Tensor.TensorType.FloatingPoint
};
Assert.Throws<ArgumentNullException>(() => m.Eval(src, dst));
}
[Test]
public void TestDstDataNull()
{
Multinomial m = new Multinomial(2018);
Tensor src = new Tensor
{
ValueType = Tensor.TensorType.FloatingPoint,
Data = new float[1]
};
Tensor dst = new Tensor
{
ValueType = Tensor.TensorType.FloatingPoint
};
Assert.Throws<ArgumentNullException>(() => m.Eval(src, dst));
}
[Test]
public void TestSrcWrongShape()
{
Multinomial m = new Multinomial(2018);
Tensor src = new Tensor
{
ValueType = Tensor.TensorType.FloatingPoint,
Data = new float[1]
};
Tensor dst = new Tensor
{
ValueType = Tensor.TensorType.FloatingPoint,
Data = new float[1]
};
Assert.Throws<ArgumentException>(() => m.Eval(src, dst));
}
[Test]
public void TestDstWrongShape()
{
Multinomial m = new Multinomial(2018);
Tensor src = new Tensor
{
ValueType = Tensor.TensorType.FloatingPoint,
Data = new float[1, 1]
};
Tensor dst = new Tensor
{
ValueType = Tensor.TensorType.FloatingPoint,
Data = new float[1]
};
Assert.Throws<ArgumentException>(() => m.Eval(src, dst));
}
[Test]
public void TestUnequalBatchSize()
{
Multinomial m = new Multinomial(2018);
Tensor src = new Tensor
{
ValueType = Tensor.TensorType.FloatingPoint,
Data = new float[1, 1]
};
Tensor dst = new Tensor
{
ValueType = Tensor.TensorType.FloatingPoint,
Data = new float[2, 1]
};
Assert.Throws<ArgumentException>(() => m.Eval(src, dst));
}
}
}

122
UnitySDK/Assets/ML-Agents/Editor/Tests/RandomNormalTest.cs


using System;
using NUnit.Framework;
using MLAgents.InferenceBrain;
using MLAgents.InferenceBrain.Utils;
namespace MLAgents.Tests
{
public class RandomNormalTest
{
[Test]
public void RandomNormalTestTwoDouble()
{
RandomNormal rn = new RandomNormal(2018);
Assert.AreEqual(-0.46666, rn.NextDouble(), 0.0001);
Assert.AreEqual(-0.37989, rn.NextDouble(), 0.0001);
}
[Test]
public void RandomNormalTestWithMean()
{
RandomNormal rn = new RandomNormal(2018, 5.0f);
Assert.AreEqual(4.53333, rn.NextDouble(), 0.0001);
Assert.AreEqual(4.6201, rn.NextDouble(), 0.0001);
}
[Test]
public void RandomNormalTestWithStddev()
{
RandomNormal rn = new RandomNormal(2018, 1.0f, 4.2f);
Assert.AreEqual(-0.9599, rn.NextDouble(), 0.0001);
Assert.AreEqual(-0.5955, rn.NextDouble(), 0.0001);
}
[Test]
public void RandomNormalTestWithMeanStddev()
{
RandomNormal rn = new RandomNormal(2018, -3.2f, 2.2f);
Assert.AreEqual(-4.2266, rn.NextDouble(), 0.0001);
Assert.AreEqual(-4.0357, rn.NextDouble(), 0.0001);
}
[Test]
public void RandomNormalTestTensorInt()
{
RandomNormal rn = new RandomNormal(1982);
Tensor t = new Tensor
{
ValueType = Tensor.TensorType.Integer
};
Assert.Throws<NotImplementedException>(() => rn.FillTensor(t));
}
[Test]
public void RandomNormalTestDataNull()
{
RandomNormal rn = new RandomNormal(1982);
Tensor t = new Tensor
{
ValueType = Tensor.TensorType.FloatingPoint
};
Assert.Throws<ArgumentNullException>(() => rn.FillTensor(t));
}
[Test]
public void RandomNormalTestTensor()
{
RandomNormal rn = new RandomNormal(1982);
Tensor t = new Tensor
{
ValueType = Tensor.TensorType.FloatingPoint,
Data = Array.CreateInstance(typeof(float), new long[3] {3, 4, 2})
};
rn.FillTensor(t);
float[] reference = new float[]
{
-0.2139822f,
0.5051259f,
-0.5640336f,
-0.3357787f,
-0.2055894f,
-0.09432302f,
-0.01419199f,
0.53621f,
-0.5507085f,
-0.2651141f,
0.09315512f,
-0.04918706f,
-0.179625f,
0.2280539f,
0.1883962f,
0.4047216f,
0.1704049f,
0.5050544f,
-0.3365685f,
0.3542781f,
0.5951571f,
0.03460682f,
-0.5537263f,
-0.4378373f,
};
int i = 0;
foreach (float f in t.Data)
{
Assert.AreEqual(f, reference[i], 0.0001);
++i;
}
}
}
}

7
UnitySDK/Assets/ML-Agents/Examples/3DBall/TFModels/3DBallHardLearning.bytes.meta


fileFormatVersion: 2
guid: 8a2da2218425f46e9921caefda4b7813
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/3DBall/TFModels/3DBallLearning.bytes.meta


fileFormatVersion: 2
guid: 9f58800fa9d54477aa01ee258842f6b3
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/BananaCollectors/TFModels/BananaLearning.bytes.meta


fileFormatVersion: 2
guid: 69bd818d72b944849916d2fda9fe471b
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/Basic/TFModels/BasicLearning.bytes.meta


fileFormatVersion: 2
guid: 503ce1e8257904bd0b5be8f7fb4b5d28
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/Bouncer/TFModels/BouncerLearning.bytes.meta


fileFormatVersion: 2
guid: 760d2b8347b4b46e3a44d9b989e1304e
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/Crawler/TFModels/CrawlerDynamicLearning.bytes.meta


fileFormatVersion: 2
guid: 9482a8782450a4d87b20942c4523176b
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/Crawler/TFModels/CrawlerStaticLearning.bytes.meta


fileFormatVersion: 2
guid: e256bd37f98f246e5be72618766d0a93
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/GridWorld/TFModels/GridWorldLearning.bytes.meta


fileFormatVersion: 2
guid: 0fd168a0ea1d04ef9a68c80cf452ce3d
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/Hallway/TFModels/HallwayLearning.bytes.meta


fileFormatVersion: 2
guid: 84588668e6ea948d3ab55bb813cc769b
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/PushBlock/TFModels/PushBlockLearning.bytes.meta


fileFormatVersion: 2
guid: e22850d2072904a0ab06069cda2599e5
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/Pyramids/TFModels/PyramidsLearning.bytes.meta


fileFormatVersion: 2
guid: 7d1c7f27447234c3a81169de00dcaa8a
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/Reacher/TFModels/ReacherLearning.bytes.meta


fileFormatVersion: 2
guid: 5fb4a3624e9ca4e1c81b51b5117cb31e
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/Soccer/TFModels/GoalieLearning.bytes.meta


fileFormatVersion: 2
guid: 890ab8f03425c4a80a52ba674ddec3f3
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/Soccer/TFModels/StrikerLearning.bytes.meta


fileFormatVersion: 2
guid: 23410257d39d44616bfefdff59c7fbc9
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/Tennis/TFModels/TennisLearning.bytes.meta


fileFormatVersion: 2
guid: 6d4281b70d41f48cb83d663b84f78c9a
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/Walker/TFModels/WalkerLearning.bytes.meta


fileFormatVersion: 2
guid: 48ab33cf9fbee4883948187618027835
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/WallJump/TFModels/BigWallJumpLearning.bytes.meta


fileFormatVersion: 2
guid: c118879bb5db84f269e4da23ba8c4f61
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

7
UnitySDK/Assets/ML-Agents/Examples/WallJump/TFModels/SmallWallJumpLearning.bytes.meta


fileFormatVersion: 2
guid: 92cd96b2c34334db692e93af25b64d2a
TextScriptImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

213
UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/TFSharpInferenceEngine.cs


#if ENABLE_TENSORFLOW
using System.Collections.Generic;
using TensorFlow;
using System.Linq;
using System;
using UnityEngine.Profiling;
using System.Runtime.InteropServices;
using UnityEngine;
namespace MLAgents.InferenceBrain
{
/// <summary>
/// TFSharpInferenceEngine - Inference engine utilizing the TensorFlow Sharp package to run inference
/// on frozen TensorFlow models
/// </summary>
public class TFSharpInferenceEngine
{
private TFGraph m_graph;
private TFSession m_session;
public void PrepareModel(byte[] model)
{
Profiler.BeginSample("TFSharpInferenceComponent.PrepareModel");
#if UNITY_ANDROID && !UNITY_EDITOR
// This needs to ba called only once and will raise an exception if called multiple times
try{
TensorFlowSharp.Android.NativeBinding.Init();
}
catch{
}
#endif
m_graph = new TFGraph();
m_graph.Import(model);
m_session = new TFSession(m_graph);
Profiler.EndSample();
}
public int ExecuteGraph(IEnumerable<Tensor> inputs_it, IEnumerable<Tensor> outputs_it)
{
Profiler.BeginSample("TFSharpInferenceComponent.ExecuteGraph");
Tensor[] inputs = inputs_it.ToArray();
Tensor[] outputs = outputs_it.ToArray();
// TODO: Can/should we pre-allocate that?
TFSession.Runner runner = m_session.GetRunner();
inputs.ToList().ForEach((Tensor input) =>
{
if (input.Shape.Length == 0)
{
var data = input.Data.GetValue(0);
if (input.DataType == typeof(int))
{
runner.AddInput(m_graph[input.Name][0], (int)data);
}
else
{
runner.AddInput(m_graph[input.Name][0], (float)data);
}
}
else
{
runner.AddInput(m_graph[input.Name][0], input.Data);
}
});
// TODO: better way to pre-allocate this?
outputs.ToList().ForEach(s => runner.Fetch(s.Name));
TFStatus status = new TFStatus();
Profiler.BeginSample("TFSharpInferenceComponent.ExecuteGraph.RunnerRun");
var out_tensors = runner.Run(status);
Profiler.EndSample();
if (!status.Ok)
{
Debug.LogError(status.StatusMessage);
return -1;
}
Debug.Assert(outputs.Length == out_tensors.Length);
for (var i = 0; i < outputs.Length; ++i)
{
if (outputs[i].Shape.Length == 0)
{
// Handle scalars
outputs[i].Data = Array.CreateInstance(outputs[i].DataType, new long[1] {1});
outputs[i].Data.SetValue(out_tensors[i].GetValue(), 0);
}
else
{
outputs[i].Data = out_tensors[i].GetValue() as Array;
}
}
Profiler.EndSample();
// TODO: create error codes
return 0;
}
[DllImport("libtensorflow")]
private static extern unsafe void TF_OperationGetAttrType(IntPtr oper, string attr_name,
TFDataType* value, IntPtr status);
[DllImport("libtensorflow")]
private static extern unsafe void TF_OperationGetAttrShape(IntPtr oper, string attr_name, long[] value,
int num_dims, IntPtr status);
private Tensor GetOpMetadata(TFOperation op)
{
TFStatus status = new TFStatus();
// Query the shape
long[] shape = null;
var shape_attr = op.GetAttributeMetadata("shape", status);
if (!status.Ok || shape_attr.TotalSize <= 0)
{
Debug.LogWarning("Operation " + op.Name + " does not contain shape attribute or it" +
" doesn't contain valid shape data!");
}
else
{
if (shape_attr.IsList)
{
throw new NotImplementedException("Querying lists is not implemented yet!");
}
else
{
TFStatus s = new TFStatus();
long[] dims = new long[shape_attr.TotalSize];
TF_OperationGetAttrShape(op.Handle, "shape", dims, (int)shape_attr.TotalSize,
s.Handle);
if (!status.Ok)
{
throw new FormatException("Could not query model for op shape (" + op.Name + ")");
}
else
{
shape = new long[dims.Length];
for (int i = 0; i < shape_attr.TotalSize; ++i)
{
if (dims[i] == -1)
{
// we have to use batchsize 1
shape[i] = 1;
}
else
{
shape[i] = dims[i];
}
}
}
}
}
// Query the data type
TFDataType type_value = new TFDataType();
unsafe
{
TFStatus s = new TFStatus();
TF_OperationGetAttrType(op.Handle, "dtype", &type_value, s.Handle);
if (!s.Ok)
{
Debug.LogWarning("Operation " + op.Name +
": error retrieving dtype, assuming float!");
type_value = TFDataType.Float;
}
}
Tensor.TensorType placeholder_type = Tensor.TensorType.FloatingPoint;
switch (type_value)
{
case TFDataType.Float:
placeholder_type = Tensor.TensorType.FloatingPoint;
break;
case TFDataType.Int32:
placeholder_type = Tensor.TensorType.Integer;
break;
default:
Debug.LogWarning("Operation " + op.Name +
" is not a float/integer. Proceed at your own risk!");
break;
}
Tensor t = new Tensor
{
Data = null,
Name = op.Name,
Shape = shape,
ValueType = placeholder_type
};
return t;
}
public IEnumerable<Tensor> InputFeatures()
{
List<Tensor> inputs = new List<Tensor>();
foreach (var op in m_graph.GetEnumerator())
{
if (op.OpType == "Placeholder")
{
inputs.Add(GetOpMetadata(op));
}
}
return inputs;
}
}
}
#endif

12
UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/TFSharpInferenceEngine.cs.meta


fileFormatVersion: 2
guid: 120cbe3fa702f4e428f57ae1d893a0a7
timeCreated: 1535148728
licenseType: Free
MonoImporter:
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

40
UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/Tensor.cs


using System;
using System.Collections.Generic;
namespace MLAgents.InferenceBrain
{
/// <summary>
/// Tensor - A class to encapsulate a Tensor used for inference.
///
/// This class contains the Array that holds the data array, the shapes, type and the placeholder in the
/// execution graph. All the fields are editable in the inspector, allowing the user to specify everything
/// but the data in a graphical way.
/// </summary>
[System.Serializable]
public class Tensor
{
public enum TensorType
{
Integer,
FloatingPoint
};
private static Dictionary<TensorType, Type> m_typeMap = new Dictionary<TensorType, Type>()
{
{ TensorType.FloatingPoint, typeof(float)},
{TensorType.Integer, typeof(int)}
};
public string Name;
public TensorType ValueType;
// Since Type is not serializable, we use the DisplayType for the Inspector
public Type DataType
{
get { return m_typeMap[ValueType]; }
}
public long[] Shape;
public Array Data;
}
}

11
UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/Tensor.cs.meta


fileFormatVersion: 2
guid: 926149e757bc849689e00e12d8c6fbdb
MonoImporter:
externalObjects: {}
serializedVersion: 2
defaultReferences: []
executionOrder: 0
icon: {instanceID: 0}
userData:
assetBundleName:
assetBundleVariant:

8
UnitySDK/Assets/ML-Agents/Scripts/InferenceBrain/Utils.meta


fileFormatVersion: 2
guid: 7872e22895343467b9fe96d336a7edba
folderAsset: yes
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

55
docs/Using-TensorFlow-Sharp-in-Unity.md


# Using TensorFlowSharp in Unity (Experimental)
The ML-Agents toolkit allows you to use pre-trained
[TensorFlow graphs](https://www.tensorflow.org/programmers_guide/graphs)
inside your Unity
games. This support is possible thanks to the
[TensorFlowSharp project](https://github.com/migueldeicaza/TensorFlowSharp).
The primary purpose for this support is to use the TensorFlow models produced by
the ML-Agents toolkit's own training programs, but a side benefit is that you
can use any TensorFlow model.
_Notice: This feature is still experimental. While it is possible to embed
trained models into Unity games, Unity Technologies does not officially support
this use-case for production games at this time. As such, no guarantees are
provided regarding the quality of experience. If you encounter issues regarding
battery life, or general performance (especially on mobile), please let us
know._
## Supported devices
* Linux 64 bits
* Mac OS X 64 bits
* Windows 64 bits
* iOS (Requires additional steps)
* Android
## Requirements
* Unity 2017.4 or above
* Unity TensorFlow Plugin ([Download here](https://s3.amazonaws.com/unity-ml-agents/0.5/TFSharpPlugin.unitypackage))
## Using TensorFlowSharp with ML-Agents
Go to `Edit` -> `Player Settings` and add `ENABLE_TENSORFLOW` to the `Scripting
Define Symbols` for each type of device you want to use (**`PC, Mac and Linux