浏览代码

Develop side channels: migrate reset parameters (#2990)

* [WIP] Side Channel initial layout

* Working prototype for raw bytes

* fixing format mistake

* Added some errors and some unit tests in C#

* Added the side channel for the Engine Configuration. (#2958)

* Added the side channel for the Engine Configuration.

Note that this change does not require modifying a lot of files :
 - Adding a sender in Python
 - Adding a receiver in C#
 - subscribe the receiver to the communicator (here is a one liner in the Academy)
 - Add the side channel to the Python UnityEnvironment (not represented here)

Adding the side channel to the environment would look like such :

```python
from mlagents.envs.environment import UnityEnvironment
from mlagents.envs.side_channel.raw_bytes_channel import RawBytesChannel
from mlagents.envs.side_channel.engine_configuration_channel import EngineConfigurationChannel

channel0 = RawBytesChannel()
channel1 = EngineConfigurationChanne...
/develop/tanhsquash
GitHub 5 年前
当前提交
8ec5ab62
共有 70 个文件被更改,包括 429 次插入1630 次删除
  1. 1
      UnitySDK/Assets/ML-Agents/Editor/Tests/DemonstrationTests.cs
  2. 1
      UnitySDK/Assets/ML-Agents/Editor/Tests/EditModeTestInternalBrainTensorGenerator.cs
  3. 8
      UnitySDK/Assets/ML-Agents/Editor/Tests/MLAgentsEditModeTest.cs
  4. 7
      UnitySDK/Assets/ML-Agents/Examples/3DBall/Scripts/Ball3DAcademy.cs
  5. 8
      UnitySDK/Assets/ML-Agents/Examples/3DBall/Scripts/Ball3DAgent.cs
  6. 8
      UnitySDK/Assets/ML-Agents/Examples/3DBall/Scripts/Ball3DHardAgent.cs
  7. 2
      UnitySDK/Assets/ML-Agents/Examples/Basic/Scripts/BasicAgent.cs
  8. 6
      UnitySDK/Assets/ML-Agents/Examples/Bouncer/Scripts/BouncerAgent.cs
  9. 5
      UnitySDK/Assets/ML-Agents/Examples/FoodCollector/Scripts/FoodCollectorAgent.cs
  10. 12
      UnitySDK/Assets/ML-Agents/Examples/GridWorld/Scripts/GridAcademy.cs
  11. 4
      UnitySDK/Assets/ML-Agents/Examples/GridWorld/Scripts/GridAgent.cs
  12. 14
      UnitySDK/Assets/ML-Agents/Examples/GridWorld/Scripts/GridArea.cs
  13. 13
      UnitySDK/Assets/ML-Agents/Examples/PushBlock/Scripts/PushAgentBasic.cs
  14. 3
      UnitySDK/Assets/ML-Agents/Examples/Reacher/Scripts/ReacherAcademy.cs
  15. 9
      UnitySDK/Assets/ML-Agents/Examples/Reacher/Scripts/ReacherAgent.cs
  16. 4
      UnitySDK/Assets/ML-Agents/Examples/Soccer/Scripts/SoccerAcademy.cs
  17. 2
      UnitySDK/Assets/ML-Agents/Examples/Soccer/Scripts/SoccerFieldArea.cs
  18. 5
      UnitySDK/Assets/ML-Agents/Examples/Tennis/Scripts/TennisAcademy.cs
  19. 8
      UnitySDK/Assets/ML-Agents/Examples/Tennis/Scripts/TennisAgent.cs
  20. 6
      UnitySDK/Assets/ML-Agents/Examples/Walker/Scripts/WalkerAcademy.cs
  21. 14
      UnitySDK/Assets/ML-Agents/Examples/Walker/Scripts/WalkerAgent.cs
  22. 11
      UnitySDK/Assets/ML-Agents/Examples/WallJump/Scripts/WallJumpAgent.cs
  23. 200
      UnitySDK/Assets/ML-Agents/Scripts/Academy.cs
  24. 52
      UnitySDK/Assets/ML-Agents/Scripts/Grpc/CommunicatorObjects/UnityRlInitializationOutput.cs
  25. 92
      UnitySDK/Assets/ML-Agents/Scripts/Grpc/CommunicatorObjects/UnityRlInput.cs
  26. 23
      UnitySDK/Assets/ML-Agents/Scripts/Grpc/GrpcExtensions.cs
  27. 22
      UnitySDK/Assets/ML-Agents/Scripts/Grpc/RpcCommunicator.cs
  28. 27
      UnitySDK/Assets/ML-Agents/Scripts/ICommunicator.cs
  29. 2
      UnitySDK/Assets/ML-Agents/Scripts/SideChannel/EngineConfigurationChannel.cs
  30. 38
      docs/Basic-Guide.md
  31. 12
      docs/Learning-Environment-Design-Academy.md
  32. 28
      docs/Learning-Environment-Examples.md
  33. 18
      docs/Learning-Environment-Executable.md
  34. 13
      docs/Migrating.md
  35. 81
      docs/Python-API.md
  36. 5
      docs/Training-Curriculum-Learning.md
  37. 4
      docs/Training-Generalized-Reinforcement-Learning-Agents.md
  38. 17
      docs/Training-ML-Agents.md
  39. 14
      ml-agents-envs/mlagents/envs/base_unity_environment.py
  40. 17
      ml-agents-envs/mlagents/envs/communicator_objects/unity_rl_initialization_output_pb2.py
  41. 14
      ml-agents-envs/mlagents/envs/communicator_objects/unity_rl_initialization_output_pb2.pyi
  42. 36
      ml-agents-envs/mlagents/envs/communicator_objects/unity_rl_input_pb2.py
  43. 16
      ml-agents-envs/mlagents/envs/communicator_objects/unity_rl_input_pb2.pyi
  44. 11
      ml-agents-envs/mlagents/envs/env_manager.py
  45. 66
      ml-agents-envs/mlagents/envs/environment.py
  46. 25
      ml-agents-envs/mlagents/envs/side_channel/engine_configuration_channel.py
  47. 29
      ml-agents-envs/mlagents/envs/simple_env_manager.py
  48. 71
      ml-agents-envs/mlagents/envs/subprocess_env_manager.py
  49. 35
      ml-agents-envs/mlagents/envs/tests/test_subprocess_env_manager.py
  50. 9
      ml-agents/mlagents/trainers/curriculum.py
  51. 70
      ml-agents/mlagents/trainers/learn.py
  52. 8
      ml-agents/mlagents/trainers/meta_curriculum.py
  53. 8
      ml-agents/mlagents/trainers/tests/test_curriculum.py
  54. 4
      ml-agents/mlagents/trainers/tests/test_learn.py
  55. 9
      ml-agents/mlagents/trainers/tests/test_meta_curriculum.py
  56. 4
      ml-agents/mlagents/trainers/tests/test_simple_rl.py
  57. 2
      ml-agents/mlagents/trainers/tests/test_trainer_controller.py
  58. 4
      ml-agents/mlagents/trainers/trainer_controller.py
  59. 3
      protobuf-definitions/proto/mlagents/envs/communicator_objects/unity_rl_initialization_output.proto
  60. 5
      protobuf-definitions/proto/mlagents/envs/communicator_objects/unity_rl_input.proto
  61. 179
      UnitySDK/Assets/ML-Agents/Editor/ResetParameterDrawer.cs
  62. 12
      UnitySDK/Assets/ML-Agents/Editor/ResetParameterDrawer.cs.meta
  63. 207
      UnitySDK/Assets/ML-Agents/Scripts/Grpc/CommunicatorObjects/EnvironmentParameters.cs
  64. 11
      UnitySDK/Assets/ML-Agents/Scripts/Grpc/CommunicatorObjects/EnvironmentParameters.cs.meta
  65. 61
      UnitySDK/Assets/ML-Agents/Scripts/ResetParameters.cs
  66. 12
      UnitySDK/Assets/ML-Agents/Scripts/ResetParameters.cs.meta
  67. 116
      docs/images/academy.png
  68. 130
      ml-agents-envs/mlagents/envs/communicator_objects/environment_parameters_pb2.py
  69. 75
      ml-agents-envs/mlagents/envs/communicator_objects/environment_parameters_pb2.pyi
  70. 11
      protobuf-definitions/proto/mlagents/envs/communicator_objects/environment_parameters.proto

1
UnitySDK/Assets/ML-Agents/Editor/Tests/DemonstrationTests.cs


var acaGo = new GameObject("TestAcademy");
acaGo.AddComponent<TestAcademy>();
var aca = acaGo.GetComponent<TestAcademy>();
aca.resetParameters = new ResetParameters();
var academyInitializeMethod = typeof(Academy).GetMethod("InitializeEnvironment",
BindingFlags.Instance | BindingFlags.NonPublic);

1
UnitySDK/Assets/ML-Agents/Editor/Tests/EditModeTestInternalBrainTensorGenerator.cs


var acaGo = new GameObject("TestAcademy");
acaGo.AddComponent<TestAcademy>();
var aca = acaGo.GetComponent<TestAcademy>();
aca.resetParameters = new ResetParameters();
var goA = new GameObject("goA");
var bpA = goA.AddComponent<BehaviorParameters>();

8
UnitySDK/Assets/ML-Agents/Editor/Tests/MLAgentsEditModeTest.cs


var acaGo = new GameObject("TestAcademy");
acaGo.AddComponent<TestAcademy>();
var aca = acaGo.GetComponent<TestAcademy>();
aca.resetParameters = new ResetParameters();
Assert.AreEqual(0, aca.initializeAcademyCalls);
Assert.AreEqual(0, aca.GetStepCount());
Assert.AreEqual(0, aca.GetEpisodeCount());

var acaGo = new GameObject("TestAcademy");
acaGo.AddComponent<TestAcademy>();
var aca = acaGo.GetComponent<TestAcademy>();
aca.resetParameters = new ResetParameters();
Assert.AreEqual(false, agent1.IsDone());
Assert.AreEqual(false, agent2.IsDone());

var acaGo = new GameObject("TestAcademy");
acaGo.AddComponent<TestAcademy>();
var aca = acaGo.GetComponent<TestAcademy>();
aca.resetParameters = new ResetParameters();
var academyInitializeMethod = typeof(Academy).GetMethod("InitializeEnvironment",
BindingFlags.Instance | BindingFlags.NonPublic);
academyInitializeMethod?.Invoke(aca, new object[] { });

var acaGo = new GameObject("TestAcademy");
acaGo.AddComponent<TestAcademy>();
var aca = acaGo.GetComponent<TestAcademy>();
aca.resetParameters = new ResetParameters();
var agentEnableMethod = typeof(Agent).GetMethod(

var acaGo = new GameObject("TestAcademy");
acaGo.AddComponent<TestAcademy>();
var aca = acaGo.GetComponent<TestAcademy>();
aca.resetParameters = new ResetParameters();
var academyInitializeMethod = typeof(Academy).GetMethod(
"InitializeEnvironment", BindingFlags.Instance | BindingFlags.NonPublic);
academyInitializeMethod?.Invoke(aca, new object[] { });

var acaGo = new GameObject("TestAcademy");
acaGo.AddComponent<TestAcademy>();
var aca = acaGo.GetComponent<TestAcademy>();
aca.resetParameters = new ResetParameters();
var agentEnableMethod = typeof(Agent).GetMethod(

var acaGo = new GameObject("TestAcademy");
acaGo.AddComponent<TestAcademy>();
var aca = acaGo.GetComponent<TestAcademy>();
aca.resetParameters = new ResetParameters();
var agentEnableMethod = typeof(Agent).GetMethod(

var acaGo = new GameObject("TestAcademy");
acaGo.AddComponent<TestAcademy>();
var aca = acaGo.GetComponent<TestAcademy>();
aca.resetParameters = new ResetParameters();
var agentEnableMethod = typeof(Agent).GetMethod(

7
UnitySDK/Assets/ML-Agents/Examples/3DBall/Scripts/Ball3DAcademy.cs


public class Ball3DAcademy : Academy
{
public override void AcademyReset()
public override void InitializeAcademy()
Physics.gravity = new Vector3(0, -resetParameters["gravity"], 0);
FloatProperties.RegisterCallback("gravity", f => { Physics.gravity = new Vector3(0, -f, 0); });
public override void AcademyStep()
{
}
}

8
UnitySDK/Assets/ML-Agents/Examples/3DBall/Scripts/Ball3DAgent.cs


[Header("Specific to Ball3D")]
public GameObject ball;
Rigidbody m_BallRb;
ResetParameters m_ResetParams;
IFloatProperties m_ResetParams;
m_ResetParams = academy.resetParameters;
m_ResetParams = academy.FloatProperties;
SetResetParameters();
}

public void SetBall()
{
//Set the attributes of the ball by fetching the information from the academy
m_BallRb.mass = m_ResetParams["mass"];
var scale = m_ResetParams["scale"];
m_BallRb.mass = m_ResetParams.GetPropertyWithDefault("mass", 1.0f);
var scale = m_ResetParams.GetPropertyWithDefault("scale", 1.0f);
ball.transform.localScale = new Vector3(scale, scale, scale);
}

8
UnitySDK/Assets/ML-Agents/Examples/3DBall/Scripts/Ball3DHardAgent.cs


[Header("Specific to Ball3DHard")]
public GameObject ball;
Rigidbody m_BallRb;
ResetParameters m_ResetParams;
IFloatProperties m_ResetParams;
m_ResetParams = academy.resetParameters;
m_ResetParams = academy.FloatProperties;
SetResetParameters();
}

public void SetBall()
{
//Set the attributes of the ball by fetching the information from the academy
m_BallRb.mass = m_ResetParams["mass"];
var scale = m_ResetParams["scale"];
m_BallRb.mass = m_ResetParams.GetPropertyWithDefault("mass", 1.0f);
var scale = m_ResetParams.GetPropertyWithDefault("scale", 1.0f);
ball.transform.localScale = new Vector3(scale, scale, scale);
}

2
UnitySDK/Assets/ML-Agents/Examples/Basic/Scripts/BasicAgent.cs


void WaitTimeInference()
{
if (!m_Academy.GetIsInference())
if (!m_Academy.IsCommunicatorOn)
{
RequestDecision();
}

6
UnitySDK/Assets/ML-Agents/Examples/Bouncer/Scripts/BouncerAgent.cs


int m_NumberJumps = 20;
int m_JumpLeft = 20;
ResetParameters m_ResetParams;
IFloatProperties m_ResetParams;
public override void InitializeAgent()
{

var academy = FindObjectOfType<Academy>();
m_ResetParams = academy.resetParameters;
m_ResetParams = academy.FloatProperties;
SetResetParameters();
}

public void SetTargetScale()
{
var targetScale = m_ResetParams["target_scale"];
var targetScale = m_ResetParams.GetPropertyWithDefault("target_scale", 1.0f);
target.transform.localScale = new Vector3(targetScale, targetScale, targetScale);
}

5
UnitySDK/Assets/ML-Agents/Examples/FoodCollector/Scripts/FoodCollectorAgent.cs


public void SetLaserLengths()
{
m_LaserLength = m_MyAcademy.resetParameters.TryGetValue("laser_length", out m_LaserLength) ? m_LaserLength : 1.0f;
m_LaserLength = m_MyAcademy.FloatProperties.GetPropertyWithDefault("laser_length", 1.0f);
float agentScale;
agentScale = m_MyAcademy.resetParameters.TryGetValue("agent_scale", out agentScale) ? agentScale : 1.0f;
float agentScale = m_MyAcademy.FloatProperties.GetPropertyWithDefault("agent_scale", 1.0f);
gameObject.transform.localScale = new Vector3(agentScale, agentScale, agentScale);
}

12
UnitySDK/Assets/ML-Agents/Examples/GridWorld/Scripts/GridAcademy.cs


{
public Camera MainCamera;
public override void AcademyReset()
public override void InitializeAcademy()
MainCamera.transform.position = new Vector3(-((int)resetParameters["gridSize"] - 1) / 2f,
(int)resetParameters["gridSize"] * 1.25f,
-((int)resetParameters["gridSize"] - 1) / 2f);
MainCamera.orthographicSize = ((int)resetParameters["gridSize"] + 5f) / 2f;
FloatProperties.RegisterCallback("gridSize", f =>
{
MainCamera.transform.position = new Vector3(-(f - 1) / 2f, f * 1.25f, -(f - 1) / 2f);
MainCamera.orthographicSize = (f + 5f) / 2f;
});
}
}

4
UnitySDK/Assets/ML-Agents/Examples/GridWorld/Scripts/GridAgent.cs


// Prevents the agent from picking an action that would make it collide with a wall
var positionX = (int)transform.position.x;
var positionZ = (int)transform.position.z;
var maxPosition = (int)m_Academy.resetParameters["gridSize"] - 1;
var maxPosition = (int)m_Academy.FloatProperties.GetPropertyWithDefault("gridSize", 5f) - 1;
if (positionX == 0)
{

renderCamera.Render();
}
if (!m_Academy.GetIsInference())
if (!m_Academy.IsCommunicatorOn)
{
RequestDecision();
}

14
UnitySDK/Assets/ML-Agents/Examples/GridWorld/Scripts/GridArea.cs


public GameObject trueAgent;
ResetParameters m_ResetParameters;
IFloatProperties m_ResetParameters;
Camera m_AgentCam;

public void Awake()
{
m_ResetParameters = FindObjectOfType<Academy>().resetParameters;
m_ResetParameters = FindObjectOfType<Academy>().FloatProperties;
m_Objects = new[] { goalPref, pitPref };

public void SetEnvironment()
{
transform.position = m_InitialPosition * (m_ResetParameters["gridSize"] + 1);
transform.position = m_InitialPosition * (m_ResetParameters.GetPropertyWithDefault("gridSize", 5f) + 1);
for (var i = 0; i < (int)m_ResetParameters["numObstacles"]; i++)
for (var i = 0; i < (int)m_ResetParameters.GetPropertyWithDefault("numObstacles", 1); i++)
for (var i = 0; i < (int)m_ResetParameters["numGoals"]; i++)
for (var i = 0; i < (int)m_ResetParameters.GetPropertyWithDefault("numGoals", 1f); i++)
var gridSize = (int)m_ResetParameters["gridSize"];
var gridSize = (int)m_ResetParameters.GetPropertyWithDefault("gridSize", 5f);
m_Plane.transform.localScale = new Vector3(gridSize / 10.0f, 1f, gridSize / 10.0f);
m_Plane.transform.localPosition = new Vector3((gridSize - 1) / 2f, -0.5f, (gridSize - 1) / 2f);
m_Sn.transform.localScale = new Vector3(1, 1, gridSize + 2);

public void AreaReset()
{
var gridSize = (int)m_ResetParameters["gridSize"];
var gridSize = (int)m_ResetParameters.GetPropertyWithDefault("gridSize", 5f); ;
foreach (var actor in actorObjs)
{
DestroyImmediate(actor);

13
UnitySDK/Assets/ML-Agents/Examples/PushBlock/Scripts/PushAgentBasic.cs


public void SetGroundMaterialFriction()
{
var resetParams = m_Academy.resetParameters;
var resetParams = m_Academy.FloatProperties;
groundCollider.material.dynamicFriction = resetParams["dynamic_friction"];
groundCollider.material.staticFriction = resetParams["static_friction"];
groundCollider.material.dynamicFriction = resetParams.GetPropertyWithDefault("dynamic_friction", 0);
groundCollider.material.staticFriction = resetParams.GetPropertyWithDefault("static_friction", 0);
var resetParams = m_Academy.resetParameters;
var resetParams = m_Academy.FloatProperties;
var scale = resetParams.GetPropertyWithDefault("block_scale", 2);
m_BlockRb.transform.localScale = new Vector3(resetParams["block_scale"], 0.75f, resetParams["block_scale"]);
m_BlockRb.transform.localScale = new Vector3(scale, 0.75f, scale);
m_BlockRb.drag = resetParams["block_drag"];
m_BlockRb.drag = resetParams.GetPropertyWithDefault("block_drag", 0.5f);
}
public void SetResetParameters()

3
UnitySDK/Assets/ML-Agents/Examples/Reacher/Scripts/ReacherAcademy.cs


{
public override void AcademyReset()
{
Physics.gravity = new Vector3(0, -resetParameters["gravity"], 0);
FloatProperties.RegisterCallback("gravity", f => { Physics.gravity = new Vector3(0, -f, 0); });
}
public override void AcademyStep()

9
UnitySDK/Assets/ML-Agents/Examples/Reacher/Scripts/ReacherAgent.cs


public void SetResetParameters()
{
m_GoalSize = m_MyAcademy.resetParameters["goal_size"];
m_GoalSpeed = Random.Range(-1f, 1f) * m_MyAcademy.resetParameters["goal_speed"];
m_Deviation = m_MyAcademy.resetParameters["deviation"];
m_DeviationFreq = m_MyAcademy.resetParameters["deviation_freq"];
var fp = m_MyAcademy.FloatProperties;
m_GoalSize = fp.GetPropertyWithDefault("goal_size", 5);
m_GoalSpeed = Random.Range(-1f, 1f) * fp.GetPropertyWithDefault("goal_speed", 1);
m_Deviation = fp.GetPropertyWithDefault("deviation", 0);
m_DeviationFreq = fp.GetPropertyWithDefault("deviation_freq", 0);
}
}

4
UnitySDK/Assets/ML-Agents/Examples/Soccer/Scripts/SoccerAcademy.cs


Physics.gravity *= gravityMultiplier; //for soccer a multiplier of 3 looks good
}
public override void AcademyReset()
public override void InitializeAcademy()
Physics.gravity = new Vector3(0, -resetParameters["gravity"], 0);
FloatProperties.RegisterCallback("gravity", f => { Physics.gravity = new Vector3(0, -f, 0); });
}
public override void AcademyStep()

2
UnitySDK/Assets/ML-Agents/Examples/Soccer/Scripts/SoccerFieldArea.cs


ballRb.velocity = Vector3.zero;
ballRb.angularVelocity = Vector3.zero;
var ballScale = m_Academy.resetParameters["ball_scale"];
var ballScale = m_Academy.FloatProperties.GetPropertyWithDefault("ball_scale", 0.015f);
ballRb.transform.localScale = new Vector3(ballScale, ballScale, ballScale);
}
}

5
UnitySDK/Assets/ML-Agents/Examples/Tennis/Scripts/TennisAcademy.cs


public class TennisAcademy : Academy
{
public override void AcademyReset()
public override void InitializeAcademy()
Physics.gravity = new Vector3(0, -resetParameters["gravity"], 0);
FloatProperties.RegisterCallback("gravity", f => { Physics.gravity = new Vector3(0, -f, 0); });
}
public override void AcademyStep()

8
UnitySDK/Assets/ML-Agents/Examples/Tennis/Scripts/TennisAgent.cs


Rigidbody m_AgentRb;
Rigidbody m_BallRb;
float m_InvertMult;
ResetParameters m_ResetParams;
IFloatProperties m_ResetParams;
// Looks for the scoreboard based on the name of the gameObjects.
// Do not modify the names of the Score GameObjects

var canvas = GameObject.Find(k_CanvasName);
GameObject scoreBoard;
var academy = FindObjectOfType<Academy>();
m_ResetParams = academy.resetParameters;
m_ResetParams = academy.FloatProperties;
if (invertX)
{
scoreBoard = canvas.transform.Find(k_ScoreBoardBName).gameObject;

public void SetRacket()
{
angle = m_ResetParams["angle"];
angle = m_ResetParams.GetPropertyWithDefault("angle", 55);
gameObject.transform.eulerAngles = new Vector3(
gameObject.transform.eulerAngles.x,
gameObject.transform.eulerAngles.y,

public void SetBall()
{
scale = m_ResetParams["scale"];
scale = m_ResetParams.GetPropertyWithDefault("scale", 1);
ball.transform.localScale = new Vector3(scale, scale, scale);
}

6
UnitySDK/Assets/ML-Agents/Examples/Walker/Scripts/WalkerAcademy.cs


Physics.defaultSolverVelocityIterations = 12;
Time.fixedDeltaTime = 0.01333f; //(75fps). default is .2 (60fps)
Time.maximumDeltaTime = .15f; // Default is .33
}
public override void AcademyReset()
{
Physics.gravity = new Vector3(0, -resetParameters["gravity"], 0);
FloatProperties.RegisterCallback("gravity", f => { Physics.gravity = new Vector3(0, -f, 0); });
}
public override void AcademyStep()

14
UnitySDK/Assets/ML-Agents/Examples/Walker/Scripts/WalkerAgent.cs


public class WalkerAgent : Agent
{
[Header("Specific to Walker")][Header("Target To Walk Towards")][Space(10)]
[Header("Specific to Walker")]
[Header("Target To Walk Towards")]
[Space(10)]
public Transform target;
Vector3 m_DirToTarget;

Rigidbody m_ChestRb;
Rigidbody m_SpineRb;
ResetParameters m_ResetParams;
IFloatProperties m_ResetParams;
public override void InitializeAgent()
{

m_SpineRb = spine.GetComponent<Rigidbody>();
var academy = FindObjectOfType<WalkerAcademy>();
m_ResetParams = academy.resetParameters;
m_ResetParams = academy.FloatProperties;
SetResetParameters();
}

public void SetTorsoMass()
{
m_ChestRb.mass = m_ResetParams["chest_mass"];
m_SpineRb.mass = m_ResetParams["spine_mass"];
m_HipsRb.mass = m_ResetParams["hip_mass"];
m_ChestRb.mass = m_ResetParams.GetPropertyWithDefault("chest_mass", 8);
m_SpineRb.mass = m_ResetParams.GetPropertyWithDefault("spine_mass", 10);
m_HipsRb.mass = m_ResetParams.GetPropertyWithDefault("hip_mass", 15);
}
public void SetResetParameters()

11
UnitySDK/Assets/ML-Agents/Examples/WallJump/Scripts/WallJumpAgent.cs


{
localScale = new Vector3(
localScale.x,
m_Academy.resetParameters["no_wall_height"],
m_Academy.FloatProperties.GetPropertyWithDefault("no_wall_height", 0),
localScale.z);
wall.transform.localScale = localScale;
GiveModel("SmallWallJump", noWallBrain);

localScale = new Vector3(
localScale.x,
m_Academy.resetParameters["small_wall_height"],
m_Academy.FloatProperties.GetPropertyWithDefault("small_wall_height", 4),
localScale.z);
wall.transform.localScale = localScale;
GiveModel("SmallWallJump", smallWallBrain);

var height =
m_Academy.resetParameters["big_wall_min_height"] +
Random.value * (m_Academy.resetParameters["big_wall_max_height"] -
m_Academy.resetParameters["big_wall_min_height"]);
var min = m_Academy.FloatProperties.GetPropertyWithDefault("big_wall_min_height", 8);
var max = m_Academy.FloatProperties.GetPropertyWithDefault("big_wall_max_height", 8);
var height = min + Random.value * (max - min);
localScale = new Vector3(
localScale.x,
height,

200
UnitySDK/Assets/ML-Agents/Scripts/Academy.cs


namespace MLAgents
{
/// <summary>
/// Wraps the environment-level parameters that are provided within the
/// Editor. These parameters can be provided for training and inference
/// modes separately and represent screen resolution, rendering quality and
/// frame rate.
/// </summary>
[System.Serializable]
public class EnvironmentConfiguration
{
[Tooltip("Width of the environment window in pixels.")]
public int width;
[Tooltip("Height of the environment window in pixels.")]
public int height;
[Tooltip("Rendering quality of environment. (Higher is better quality.)")]
[Range(0, 5)]
public int qualityLevel;
[Tooltip("Speed at which environment is run. (Higher is faster.)")]
[Range(1f, 100f)]
public float timeScale;
[Tooltip("Frames per second (FPS) engine attempts to maintain.")]
public int targetFrameRate;
/// Initializes a new instance of the
/// <see cref="EnvironmentConfiguration"/> class.
/// <param name="width">Width of environment window (pixels).</param>
/// <param name="height">Height of environment window (pixels).</param>
/// <param name="qualityLevel">
/// Rendering quality of environment. Ranges from 0 to 5, with higher.
/// </param>
/// <param name="timeScale">
/// Speed at which environment is run. Ranges from 1 to 100, with higher
/// values representing faster speed.
/// </param>
/// <param name="targetFrameRate">
/// Target frame rate (per second) that the engine tries to maintain.
/// </param>
public EnvironmentConfiguration(
int width, int height, int qualityLevel,
float timeScale, int targetFrameRate)
{
this.width = width;
this.height = height;
this.qualityLevel = qualityLevel;
this.timeScale = timeScale;
this.targetFrameRate = targetFrameRate;
}
}
/// <summary>
/// An Academy is where Agent objects go to train their behaviors.

/// Used to restore original value when deriving Academy modifies it
float m_OriginalMaximumDeltaTime;
// Fields provided in the Inspector
[FormerlySerializedAs("trainingConfiguration")]
[SerializeField]
[Tooltip("The engine-level settings which correspond to rendering " +
"quality and engine speed during Training.")]
EnvironmentConfiguration m_TrainingConfiguration =
new EnvironmentConfiguration(80, 80, 1, 100.0f, -1);
[FormerlySerializedAs("inferenceConfiguration")]
[SerializeField]
[Tooltip("The engine-level settings which correspond to rendering " +
"quality and engine speed during Inference.")]
EnvironmentConfiguration m_InferenceConfiguration =
new EnvironmentConfiguration(1280, 720, 5, 1.0f, 60);
/// <summary>
/// Contains a mapping from parameter names to float values. They are
/// used in <see cref="AcademyReset"/> and <see cref="AcademyStep"/>
/// to modify elements in the environment at reset time.
/// </summary>
/// <remarks>
/// Default reset parameters are specified in the academy Editor, and can
/// be modified when training by passing a config
/// dictionary at reset.
/// </remarks>
[SerializeField]
[Tooltip("List of custom parameters that can be changed in the " +
"environment when it resets.")]
public ResetParameters resetParameters;
public CommunicatorObjects.CustomResetParametersProto customResetParameters;
// Fields not provided in the Inspector.

get { return Communicator != null; }
}
/// If true, the Academy will use inference settings. This field is
/// initialized in <see cref="Awake"/> depending on the presence
/// or absence of a communicator. Furthermore, it can be modified during
/// training via <see cref="SetIsInference"/>.
bool m_IsInference = true;
/// The number of episodes completed by the environment. Incremented
/// each time the environment is reset.
int m_EpisodeCount;

/// The number of total number of steps completed during the whole simulation. Incremented
/// each time a step is taken in the environment.
int m_TotalStepCount;
/// Flag that indicates whether the inference/training mode of the
/// environment was switched by the training process. This impacts the
/// engine settings at the next environment step.
bool m_ModeSwitched;
/// Pointer to the communicator currently in use by the Academy.
public ICommunicator Communicator;

m_OriginalFixedDeltaTime = Time.fixedDeltaTime;
m_OriginalMaximumDeltaTime = Time.maximumDeltaTime;
InitializeAcademy();
InitializeAcademy();
// Try to launch the communicator by using the arguments passed at launch
try

if (Communicator != null)
{
Communicator.RegisterSideChannel(new EngineConfigurationChannel());
Communicator.RegisterSideChannel(floatProperties);
// We try to exchange the first message with Python. If this fails, it means
// no Python Process is ready to train the environment. In this case, the
//environment must use Inference.

{
version = k_ApiVersion,
name = gameObject.name,
environmentResetParameters = new EnvironmentResetParameters
{
resetParameters = resetParameters,
customResetParameters = customResetParameters
}
});
Random.InitState(unityRLInitParameters.seed);
}

{
Communicator.QuitCommandReceived += OnQuitCommandReceived;
Communicator.ResetCommandReceived += OnResetCommand;
Communicator.RLInputReceived += OnRLInputReceived;
Communicator.RegisterSideChannel(new EngineConfigurationChannel());
Communicator.RegisterSideChannel(floatProperties);
}
}

SetIsInference(!IsCommunicatorOn);
DecideAction += () => { };
DestroyAction += () => { };
AgentSetStatus += i => { };

AgentForceReset += () => { };
ConfigureEnvironment();
}
static void OnQuitCommandReceived()

Application.Quit();
}
void OnResetCommand(EnvironmentResetParameters newResetParameters)
void OnResetCommand()
UpdateResetParameters(newResetParameters);
void OnRLInputReceived(UnityRLInputParameters inputParams)
{
m_IsInference = !inputParams.isTraining;
}
void UpdateResetParameters(EnvironmentResetParameters newResetParameters)
{
if (newResetParameters.resetParameters != null)
{
foreach (var kv in newResetParameters.resetParameters)
{
resetParameters[kv.Key] = kv.Value;
}
}
customResetParameters = newResetParameters.customResetParameters;
}
/// <summary>
/// Configures the environment settings depending on the training/inference
/// mode and the corresponding parameters passed in the Editor.
/// </summary>
void ConfigureEnvironment()
{
if (m_IsInference)
{
ConfigureEnvironmentHelper(m_InferenceConfiguration);
Monitor.SetActive(true);
}
else
{
ConfigureEnvironmentHelper(m_TrainingConfiguration);
Monitor.SetActive(false);
}
}
/// <summary>
/// Helper method for initializing the environment based on the provided
/// configuration.
/// </summary>
/// <param name="config">
/// Environment configuration (specified in the Editor).
/// </param>
static void ConfigureEnvironmentHelper(EnvironmentConfiguration config)
{
Screen.SetResolution(config.width, config.height, false);
QualitySettings.SetQualityLevel(config.qualityLevel, true);
Time.timeScale = config.timeScale;
Time.captureFramerate = 60;
Application.targetFrameRate = config.targetFrameRate;
}
/// <summary>
/// Initializes the academy and environment. Called during the waking-up
/// phase of the environment before any of the scene objects/agents have

{
}
/// <summary>
/// Returns the <see cref="m_IsInference"/> flag.
/// </summary>
/// <returns>
/// <c>true</c>, if current mode is inference, <c>false</c> if training.
/// </returns>
public bool GetIsInference()
{
return m_IsInference;
}
/// <summary>
/// Sets the <see cref="m_IsInference"/> flag to the provided value. If
/// the new flag differs from the current flag value, this signals that
/// the environment configuration needs to be updated.
/// </summary>
/// <param name="isInference">
/// Environment mode, if true then inference, otherwise training.
/// </param>
public void SetIsInference(bool isInference)
{
if (m_IsInference != isInference)
{
m_IsInference = isInference;
// This signals to the academy that at the next environment step
// the engine configurations need updating to the respective mode
// (i.e. training vs inference) configuration.
m_ModeSwitched = true;
}
}
/// <summary>
/// Returns the current episode counter.

/// </summary>
void EnvironmentStep()
{
if (m_ModeSwitched)
{
ConfigureEnvironment();
m_ModeSwitched = false;
}
if (!m_FirstAcademyReset)
{
ForcedFullReset();

52
UnitySDK/Assets/ML-Agents/Scripts/Grpc/CommunicatorObjects/UnityRlInitializationOutput.cs


"CkdtbGFnZW50cy9lbnZzL2NvbW11bmljYXRvcl9vYmplY3RzL3VuaXR5X3Js",
"X2luaXRpYWxpemF0aW9uX291dHB1dC5wcm90bxIUY29tbXVuaWNhdG9yX29i",
"amVjdHMaOW1sYWdlbnRzL2VudnMvY29tbXVuaWNhdG9yX29iamVjdHMvYnJh",
"aW5fcGFyYW1ldGVycy5wcm90bxo/bWxhZ2VudHMvZW52cy9jb21tdW5pY2F0",
"b3Jfb2JqZWN0cy9lbnZpcm9ubWVudF9wYXJhbWV0ZXJzLnByb3RvIusBCiBV",
"bml0eVJMSW5pdGlhbGl6YXRpb25PdXRwdXRQcm90bxIMCgRuYW1lGAEgASgJ",
"Eg8KB3ZlcnNpb24YAiABKAkSEAoIbG9nX3BhdGgYAyABKAkSRAoQYnJhaW5f",
"cGFyYW1ldGVycxgFIAMoCzIqLmNvbW11bmljYXRvcl9vYmplY3RzLkJyYWlu",
"UGFyYW1ldGVyc1Byb3RvElAKFmVudmlyb25tZW50X3BhcmFtZXRlcnMYBiAB",
"KAsyMC5jb21tdW5pY2F0b3Jfb2JqZWN0cy5FbnZpcm9ubWVudFBhcmFtZXRl",
"cnNQcm90b0IfqgIcTUxBZ2VudHMuQ29tbXVuaWNhdG9yT2JqZWN0c2IGcHJv",
"dG8z"));
"aW5fcGFyYW1ldGVycy5wcm90byKfAQogVW5pdHlSTEluaXRpYWxpemF0aW9u",
"T3V0cHV0UHJvdG8SDAoEbmFtZRgBIAEoCRIPCgd2ZXJzaW9uGAIgASgJEhAK",
"CGxvZ19wYXRoGAMgASgJEkQKEGJyYWluX3BhcmFtZXRlcnMYBSADKAsyKi5j",
"b21tdW5pY2F0b3Jfb2JqZWN0cy5CcmFpblBhcmFtZXRlcnNQcm90b0oECAYQ",
"B0IfqgIcTUxBZ2VudHMuQ29tbXVuaWNhdG9yT2JqZWN0c2IGcHJvdG8z"));
new pbr::FileDescriptor[] { global::MLAgents.CommunicatorObjects.BrainParametersReflection.Descriptor, global::MLAgents.CommunicatorObjects.EnvironmentParametersReflection.Descriptor, },
new pbr::FileDescriptor[] { global::MLAgents.CommunicatorObjects.BrainParametersReflection.Descriptor, },
new pbr::GeneratedClrTypeInfo(typeof(global::MLAgents.CommunicatorObjects.UnityRLInitializationOutputProto), global::MLAgents.CommunicatorObjects.UnityRLInitializationOutputProto.Parser, new[]{ "Name", "Version", "LogPath", "BrainParameters", "EnvironmentParameters" }, null, null, null)
new pbr::GeneratedClrTypeInfo(typeof(global::MLAgents.CommunicatorObjects.UnityRLInitializationOutputProto), global::MLAgents.CommunicatorObjects.UnityRLInitializationOutputProto.Parser, new[]{ "Name", "Version", "LogPath", "BrainParameters" }, null, null, null)
}));
}
#endregion

version_ = other.version_;
logPath_ = other.logPath_;
brainParameters_ = other.brainParameters_.Clone();
EnvironmentParameters = other.environmentParameters_ != null ? other.EnvironmentParameters.Clone() : null;
_unknownFields = pb::UnknownFieldSet.Clone(other._unknownFields);
}

get { return brainParameters_; }
}
/// <summary>Field number for the "environment_parameters" field.</summary>
public const int EnvironmentParametersFieldNumber = 6;
private global::MLAgents.CommunicatorObjects.EnvironmentParametersProto environmentParameters_;
[global::System.Diagnostics.DebuggerNonUserCodeAttribute]
public global::MLAgents.CommunicatorObjects.EnvironmentParametersProto EnvironmentParameters {
get { return environmentParameters_; }
set {
environmentParameters_ = value;
}
}
[global::System.Diagnostics.DebuggerNonUserCodeAttribute]
public override bool Equals(object other) {
return Equals(other as UnityRLInitializationOutputProto);

if (Version != other.Version) return false;
if (LogPath != other.LogPath) return false;
if(!brainParameters_.Equals(other.brainParameters_)) return false;
if (!object.Equals(EnvironmentParameters, other.EnvironmentParameters)) return false;
return Equals(_unknownFields, other._unknownFields);
}

if (Version.Length != 0) hash ^= Version.GetHashCode();
if (LogPath.Length != 0) hash ^= LogPath.GetHashCode();
hash ^= brainParameters_.GetHashCode();
if (environmentParameters_ != null) hash ^= EnvironmentParameters.GetHashCode();
if (_unknownFields != null) {
hash ^= _unknownFields.GetHashCode();
}

output.WriteString(LogPath);
}
brainParameters_.WriteTo(output, _repeated_brainParameters_codec);
if (environmentParameters_ != null) {
output.WriteRawTag(50);
output.WriteMessage(EnvironmentParameters);
}
if (_unknownFields != null) {
_unknownFields.WriteTo(output);
}

size += 1 + pb::CodedOutputStream.ComputeStringSize(LogPath);
}
size += brainParameters_.CalculateSize(_repeated_brainParameters_codec);
if (environmentParameters_ != null) {
size += 1 + pb::CodedOutputStream.ComputeMessageSize(EnvironmentParameters);
}
if (_unknownFields != null) {
size += _unknownFields.CalculateSize();
}

LogPath = other.LogPath;
}
brainParameters_.Add(other.brainParameters_);
if (other.environmentParameters_ != null) {
if (environmentParameters_ == null) {
environmentParameters_ = new global::MLAgents.CommunicatorObjects.EnvironmentParametersProto();
}
EnvironmentParameters.MergeFrom(other.EnvironmentParameters);
}
_unknownFields = pb::UnknownFieldSet.MergeFrom(_unknownFields, other._unknownFields);
}

}
case 42: {
brainParameters_.AddEntriesFrom(input, _repeated_brainParameters_codec);
break;
}
case 50: {
if (environmentParameters_ == null) {
environmentParameters_ = new global::MLAgents.CommunicatorObjects.EnvironmentParametersProto();
}
input.ReadMessage(environmentParameters_);
break;
}
}

92
UnitySDK/Assets/ML-Agents/Scripts/Grpc/CommunicatorObjects/UnityRlInput.cs


"CjdtbGFnZW50cy9lbnZzL2NvbW11bmljYXRvcl9vYmplY3RzL3VuaXR5X3Js",
"X2lucHV0LnByb3RvEhRjb21tdW5pY2F0b3Jfb2JqZWN0cxo1bWxhZ2VudHMv",
"ZW52cy9jb21tdW5pY2F0b3Jfb2JqZWN0cy9hZ2VudF9hY3Rpb24ucHJvdG8a",
"P21sYWdlbnRzL2VudnMvY29tbXVuaWNhdG9yX29iamVjdHMvZW52aXJvbm1l",
"bnRfcGFyYW1ldGVycy5wcm90bxowbWxhZ2VudHMvZW52cy9jb21tdW5pY2F0",
"b3Jfb2JqZWN0cy9jb21tYW5kLnByb3RvItkDChFVbml0eVJMSW5wdXRQcm90",
"bxJQCg1hZ2VudF9hY3Rpb25zGAEgAygLMjkuY29tbXVuaWNhdG9yX29iamVj",
"dHMuVW5pdHlSTElucHV0UHJvdG8uQWdlbnRBY3Rpb25zRW50cnkSUAoWZW52",
"aXJvbm1lbnRfcGFyYW1ldGVycxgCIAEoCzIwLmNvbW11bmljYXRvcl9vYmpl",
"Y3RzLkVudmlyb25tZW50UGFyYW1ldGVyc1Byb3RvEhMKC2lzX3RyYWluaW5n",
"GAMgASgIEjMKB2NvbW1hbmQYBCABKA4yIi5jb21tdW5pY2F0b3Jfb2JqZWN0",
"cy5Db21tYW5kUHJvdG8SFAoMc2lkZV9jaGFubmVsGAUgASgMGk0KFExpc3RB",
"Z2VudEFjdGlvblByb3RvEjUKBXZhbHVlGAEgAygLMiYuY29tbXVuaWNhdG9y",
"X29iamVjdHMuQWdlbnRBY3Rpb25Qcm90bxpxChFBZ2VudEFjdGlvbnNFbnRy",
"eRILCgNrZXkYASABKAkSSwoFdmFsdWUYAiABKAsyPC5jb21tdW5pY2F0b3Jf",
"b2JqZWN0cy5Vbml0eVJMSW5wdXRQcm90by5MaXN0QWdlbnRBY3Rpb25Qcm90",
"bzoCOAFCH6oCHE1MQWdlbnRzLkNvbW11bmljYXRvck9iamVjdHNiBnByb3Rv",
"Mw=="));
"MG1sYWdlbnRzL2VudnMvY29tbXVuaWNhdG9yX29iamVjdHMvY29tbWFuZC5w",
"cm90byL+AgoRVW5pdHlSTElucHV0UHJvdG8SUAoNYWdlbnRfYWN0aW9ucxgB",
"IAMoCzI5LmNvbW11bmljYXRvcl9vYmplY3RzLlVuaXR5UkxJbnB1dFByb3Rv",
"LkFnZW50QWN0aW9uc0VudHJ5EjMKB2NvbW1hbmQYBCABKA4yIi5jb21tdW5p",
"Y2F0b3Jfb2JqZWN0cy5Db21tYW5kUHJvdG8SFAoMc2lkZV9jaGFubmVsGAUg",
"ASgMGk0KFExpc3RBZ2VudEFjdGlvblByb3RvEjUKBXZhbHVlGAEgAygLMiYu",
"Y29tbXVuaWNhdG9yX29iamVjdHMuQWdlbnRBY3Rpb25Qcm90bxpxChFBZ2Vu",
"dEFjdGlvbnNFbnRyeRILCgNrZXkYASABKAkSSwoFdmFsdWUYAiABKAsyPC5j",
"b21tdW5pY2F0b3Jfb2JqZWN0cy5Vbml0eVJMSW5wdXRQcm90by5MaXN0QWdl",
"bnRBY3Rpb25Qcm90bzoCOAFKBAgCEANKBAgDEARCH6oCHE1MQWdlbnRzLkNv",
"bW11bmljYXRvck9iamVjdHNiBnByb3RvMw=="));
new pbr::FileDescriptor[] { global::MLAgents.CommunicatorObjects.AgentActionReflection.Descriptor, global::MLAgents.CommunicatorObjects.EnvironmentParametersReflection.Descriptor, global::MLAgents.CommunicatorObjects.CommandReflection.Descriptor, },
new pbr::FileDescriptor[] { global::MLAgents.CommunicatorObjects.AgentActionReflection.Descriptor, global::MLAgents.CommunicatorObjects.CommandReflection.Descriptor, },
new pbr::GeneratedClrTypeInfo(typeof(global::MLAgents.CommunicatorObjects.UnityRLInputProto), global::MLAgents.CommunicatorObjects.UnityRLInputProto.Parser, new[]{ "AgentActions", "EnvironmentParameters", "IsTraining", "Command", "SideChannel" }, null, null, new pbr::GeneratedClrTypeInfo[] { new pbr::GeneratedClrTypeInfo(typeof(global::MLAgents.CommunicatorObjects.UnityRLInputProto.Types.ListAgentActionProto), global::MLAgents.CommunicatorObjects.UnityRLInputProto.Types.ListAgentActionProto.Parser, new[]{ "Value" }, null, null, null),
new pbr::GeneratedClrTypeInfo(typeof(global::MLAgents.CommunicatorObjects.UnityRLInputProto), global::MLAgents.CommunicatorObjects.UnityRLInputProto.Parser, new[]{ "AgentActions", "Command", "SideChannel" }, null, null, new pbr::GeneratedClrTypeInfo[] { new pbr::GeneratedClrTypeInfo(typeof(global::MLAgents.CommunicatorObjects.UnityRLInputProto.Types.ListAgentActionProto), global::MLAgents.CommunicatorObjects.UnityRLInputProto.Types.ListAgentActionProto.Parser, new[]{ "Value" }, null, null, null),
null, })
}));
}

[global::System.Diagnostics.DebuggerNonUserCodeAttribute]
public UnityRLInputProto(UnityRLInputProto other) : this() {
agentActions_ = other.agentActions_.Clone();
EnvironmentParameters = other.environmentParameters_ != null ? other.EnvironmentParameters.Clone() : null;
isTraining_ = other.isTraining_;
command_ = other.command_;
sideChannel_ = other.sideChannel_;
_unknownFields = pb::UnknownFieldSet.Clone(other._unknownFields);

get { return agentActions_; }
}
/// <summary>Field number for the "environment_parameters" field.</summary>
public const int EnvironmentParametersFieldNumber = 2;
private global::MLAgents.CommunicatorObjects.EnvironmentParametersProto environmentParameters_;
[global::System.Diagnostics.DebuggerNonUserCodeAttribute]
public global::MLAgents.CommunicatorObjects.EnvironmentParametersProto EnvironmentParameters {
get { return environmentParameters_; }
set {
environmentParameters_ = value;
}
}
/// <summary>Field number for the "is_training" field.</summary>
public const int IsTrainingFieldNumber = 3;
private bool isTraining_;
[global::System.Diagnostics.DebuggerNonUserCodeAttribute]
public bool IsTraining {
get { return isTraining_; }
set {
isTraining_ = value;
}
}
/// <summary>Field number for the "command" field.</summary>
public const int CommandFieldNumber = 4;
private global::MLAgents.CommunicatorObjects.CommandProto command_ = 0;

return true;
}
if (!AgentActions.Equals(other.AgentActions)) return false;
if (!object.Equals(EnvironmentParameters, other.EnvironmentParameters)) return false;
if (IsTraining != other.IsTraining) return false;
if (Command != other.Command) return false;
if (SideChannel != other.SideChannel) return false;
return Equals(_unknownFields, other._unknownFields);

public override int GetHashCode() {
int hash = 1;
hash ^= AgentActions.GetHashCode();
if (environmentParameters_ != null) hash ^= EnvironmentParameters.GetHashCode();
if (IsTraining != false) hash ^= IsTraining.GetHashCode();
if (Command != 0) hash ^= Command.GetHashCode();
if (SideChannel.Length != 0) hash ^= SideChannel.GetHashCode();
if (_unknownFields != null) {

[global::System.Diagnostics.DebuggerNonUserCodeAttribute]
public void WriteTo(pb::CodedOutputStream output) {
agentActions_.WriteTo(output, _map_agentActions_codec);
if (environmentParameters_ != null) {
output.WriteRawTag(18);
output.WriteMessage(EnvironmentParameters);
}
if (IsTraining != false) {
output.WriteRawTag(24);
output.WriteBool(IsTraining);
}
if (Command != 0) {
output.WriteRawTag(32);
output.WriteEnum((int) Command);

public int CalculateSize() {
int size = 0;
size += agentActions_.CalculateSize(_map_agentActions_codec);
if (environmentParameters_ != null) {
size += 1 + pb::CodedOutputStream.ComputeMessageSize(EnvironmentParameters);
}
if (IsTraining != false) {
size += 1 + 1;
}
if (Command != 0) {
size += 1 + pb::CodedOutputStream.ComputeEnumSize((int) Command);
}

return;
}
agentActions_.Add(other.agentActions_);
if (other.environmentParameters_ != null) {
if (environmentParameters_ == null) {
environmentParameters_ = new global::MLAgents.CommunicatorObjects.EnvironmentParametersProto();
}
EnvironmentParameters.MergeFrom(other.EnvironmentParameters);
}
if (other.IsTraining != false) {
IsTraining = other.IsTraining;
}
if (other.Command != 0) {
Command = other.Command;
}

break;
case 10: {
agentActions_.AddEntriesFrom(input, _map_agentActions_codec);
break;
}
case 18: {
if (environmentParameters_ == null) {
environmentParameters_ = new global::MLAgents.CommunicatorObjects.EnvironmentParametersProto();
}
input.ReadMessage(environmentParameters_);
break;
}
case 24: {
IsTraining = input.ReadBool();
break;
}
case 32: {

23
UnitySDK/Assets/ML-Agents/Scripts/Grpc/GrpcExtensions.cs


return bp;
}
/// <summary>
/// Convert a MapField to ResetParameters.
/// </summary>
/// <param name="floatParams">The mapping of strings to floats from a protobuf MapField.</param>
/// <returns></returns>
public static ResetParameters ToResetParameters(this MapField<string, float> floatParams)
{
return new ResetParameters(floatParams);
}
/// <summary>
/// Convert an EnvironmnetParametersProto protobuf object to an EnvironmentResetParameters struct.
/// </summary>
/// <param name="epp">The instance of the EnvironmentParametersProto object.</param>
/// <returns>A new EnvironmentResetParameters struct.</returns>
public static EnvironmentResetParameters ToEnvironmentResetParameters(this EnvironmentParametersProto epp)
{
return new EnvironmentResetParameters
{
resetParameters = epp.FloatParameters?.ToResetParameters(),
customResetParameters = epp.CustomResetParameters
};
}
public static UnityRLInitParameters ToUnityRLInitParameters(this UnityRLInitializationInputProto inputProto)
{

22
UnitySDK/Assets/ML-Agents/Scripts/Grpc/RpcCommunicator.cs


{
public event QuitCommandHandler QuitCommandReceived;
public event ResetCommandHandler ResetCommandReceived;
public event RLInputReceivedHandler RLInputReceived;
/// If true, the communication is active.
bool m_IsOpen;

Version = initParameters.version
};
academyParameters.EnvironmentParameters = new EnvironmentParametersProto();
var resetParameters = initParameters.environmentResetParameters.resetParameters;
foreach (var key in resetParameters.Keys)
{
academyParameters.EnvironmentParameters.FloatParameters.Add(key, resetParameters[key]);
}
UnityInputProto input;
UnityInputProto initializationInput;
try

void UpdateEnvironmentWithInput(UnityRLInputProto rlInput)
{
SendRLInputReceivedEvent(rlInput.IsTraining);
SendCommandEvent(rlInput.Command, rlInput.EnvironmentParameters);
SendCommandEvent(rlInput.Command);
}
UnityInputProto Initialize(UnityOutputProto unityOutput,

#region Sending Events
void SendCommandEvent(CommandProto command, EnvironmentParametersProto environmentParametersProto)
void SendCommandEvent(CommandProto command)
{
switch (command)
{

}
case CommandProto.Reset:
{
ResetCommandReceived?.Invoke(environmentParametersProto.ToEnvironmentResetParameters());
ResetCommandReceived?.Invoke();
return;
}
default:

}
}
void SendRLInputReceivedEvent(bool isTraining)
{
RLInputReceived?.Invoke(new UnityRLInputParameters { isTraining = isTraining });
}
#endregion

27
UnitySDK/Assets/ML-Agents/Scripts/ICommunicator.cs


namespace MLAgents
{
public struct EnvironmentResetParameters
{
/// <summary>
/// Mapping of string : float which defines which parameters can be
/// reset from python.
/// </summary>
public ResetParameters resetParameters;
/// <summary>
/// The protobuf for custom reset parameters.
/// NOTE: This is the last remaining relic of gRPC protocol
/// that is left in our code. We need to decide how to handle this
/// moving forward.
/// </summary>
public CustomResetParametersProto customResetParameters;
}
public struct CommunicatorInitParameters
{
/// <summary>

/// The version of the Unity SDK.
/// </summary>
public string version;
/// <summary>
/// The set of environment parameters defined by the user that will be sent to the communicator.
/// </summary>
public EnvironmentResetParameters environmentResetParameters;
}
public struct UnityRLInitParameters
{

/// Delegate for handling reset parameter updates sent from the communicator.
/// </summary>
/// <param name="resetParams"></param>
public delegate void ResetCommandHandler(EnvironmentResetParameters resetParams);
public delegate void ResetCommandHandler();
/// <summary>
/// Delegate to handle UnityRLInputParameters updates from the communicator.

/// Reset command sent back from the communicator.
/// </summary>
event ResetCommandHandler ResetCommandReceived;
/// <summary>
/// Unity RL Input was received by the communicator.
/// </summary>
event RLInputReceivedHandler RLInputReceived;
/// <summary>
/// Sends the academy parameters through the Communicator.

2
UnitySDK/Assets/ML-Agents/Scripts/SideChannel/EngineConfigurationChannel.cs


var timeScale = binaryReader.ReadSingle();
var targetFrameRate = binaryReader.ReadInt32();
timeScale = Mathf.Clamp(timeScale, 1, 100);
Screen.SetResolution(width, height, false);
QualitySettings.SetQualityLevel(qualityLevel, true);
Time.timeScale = timeScale;

38
docs/Basic-Guide.md


page](Learning-Environment-Executable.md) for instructions on how to build and
use an executable.
```console
ml-agents$ mlagents-learn config/trainer_config.yaml --run-id=first-run --train
▄▄▄▓▓▓▓
╓▓▓▓▓▓▓█▓▓▓▓▓
,▄▄▄m▀▀▀' ,▓▓▓▀▓▓▄ ▓▓▓ ▓▓▌
▄▓▓▓▀' ▄▓▓▀ ▓▓▓ ▄▄ ▄▄ ,▄▄ ▄▄▄▄ ,▄▄ ▄▓▓▌▄ ▄▄▄ ,▄▄
▄▓▓▓▀ ▄▓▓▀ ▐▓▓▌ ▓▓▌ ▐▓▓ ▐▓▓▓▀▀▀▓▓▌ ▓▓▓ ▀▓▓▌▀ ^▓▓▌ ╒▓▓▌
▄▓▓▓▓▓▄▄▄▄▄▄▄▄▓▓▓ ▓▀ ▓▓▌ ▐▓▓ ▐▓▓ ▓▓▓ ▓▓▓ ▓▓▌ ▐▓▓▄ ▓▓▌
▀▓▓▓▓▀▀▀▀▀▀▀▀▀▀▓▓▄ ▓▓ ▓▓▌ ▐▓▓ ▐▓▓ ▓▓▓ ▓▓▓ ▓▓▌ ▐▓▓▐▓▓
^█▓▓▓ ▀▓▓▄ ▐▓▓▌ ▓▓▓▓▄▓▓▓▓ ▐▓▓ ▓▓▓ ▓▓▓ ▓▓▓▄ ▓▓▓▓`
'▀▓▓▓▄ ^▓▓▓ ▓▓▓ └▀▀▀▀ ▀▀ ^▀▀ `▀▀ `▀▀ '▀▀ ▐▓▓▌
▀▀▀▀▓▄▄▄ ▓▓▓▓▓▓, ▓▓▓▓▀
`▀█▓▓▓▓▓▓▓▓▓▌
¬`▀▀▀█▓
INFO:mlagents.learn:{'--curriculum': 'None',
'--docker-target-name': 'Empty',
'--env': 'None',
'--help': False,
'--keep-checkpoints': '5',
'--lesson': '0',
'--load': False,
'--no-graphics': False,
'--num-runs': '1',
'--run-id': 'first-run',
'--save-freq': '50000',
'--seed': '-1',
'--slow': False,
'--train': True,
'--worker-id': '0',
'<trainer-config-path>': 'config/trainer_config.yaml'}
INFO:mlagents.envs:Start training by pressing the Play button in the Unity Editor.
```
**Note**: If you're using Anaconda, don't forget to activate the ml-agents
environment first.

INFO:mlagents.envs:
'Ball3DAcademy' started successfully!
Unity Academy name: Ball3DAcademy
Reset Parameters : {}
INFO:mlagents.envs:Connected new brain:
Unity brain name: 3DBallLearning

12
docs/Learning-Environment-Design-Academy.md