Unity 机器学习代理工具包 (ML-Agents) 是一个开源项目,它使游戏和模拟能够作为训练智能代理的环境。
您最多选择25个主题 主题必须以中文或者字母或数字开头,可以包含连字符 (-),并且长度不得超过35个字符
 
 
 
 
 

293 行
10 KiB

{
"nbformat": 4,
"nbformat_minor": 0,
"metadata": {
"colab": {
"name": "Colab-UnityEnvironment-3-SideChannel.ipynb",
"private_outputs": true,
"provenance": [],
"collapsed_sections": [],
"toc_visible": true
},
"kernelspec": {
"name": "python3",
"display_name": "Python 3"
}
},
"cells": [
{
"cell_type": "markdown",
"metadata": {
"id": "pbVXrmEsLXDt"
},
"source": [
"# ML-Agents Use SideChannels\n",
"<img src=\"https://raw.githubusercontent.com/Unity-Technologies/ml-agents/release_1/docs/images/3dball_big.png\" align=\"middle\" width=\"435\"/>"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "WNKTwHU3d2-l"
},
"source": [
"## Setup"
]
},
{
"cell_type": "code",
"metadata": {
"id": "htb-p1hSNX7D"
},
"source": [
"#@title Install Rendering Dependencies { display-mode: \"form\" }\n",
"#@markdown (You only need to run this code when using Colab's hosted runtime)\n",
"\n",
"from IPython.display import HTML, display\n",
"\n",
"def progress(value, max=100):\n",
" return HTML(\"\"\"\n",
" <progress\n",
" value='{value}'\n",
" max='{max}',\n",
" style='width: 100%'\n",
" >\n",
" {value}\n",
" </progress>\n",
" \"\"\".format(value=value, max=max))\n",
"\n",
"pro_bar = display(progress(0, 100), display_id=True)\n",
"\n",
"try:\n",
" import google.colab\n",
" IN_COLAB = True\n",
"except ImportError:\n",
" IN_COLAB = False\n",
"\n",
"if IN_COLAB:\n",
" with open('frame-buffer', 'w') as writefile:\n",
" writefile.write(\"\"\"#taken from https://gist.github.com/jterrace/2911875\n",
"XVFB=/usr/bin/Xvfb\n",
"XVFBARGS=\":1 -screen 0 1024x768x24 -ac +extension GLX +render -noreset\"\n",
"PIDFILE=./frame-buffer.pid\n",
"case \"$1\" in\n",
" start)\n",
" echo -n \"Starting virtual X frame buffer: Xvfb\"\n",
" /sbin/start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile --background --exec $XVFB -- $XVFBARGS\n",
" echo \".\"\n",
" ;;\n",
" stop)\n",
" echo -n \"Stopping virtual X frame buffer: Xvfb\"\n",
" /sbin/start-stop-daemon --stop --quiet --pidfile $PIDFILE\n",
" rm $PIDFILE\n",
" echo \".\"\n",
" ;;\n",
" restart)\n",
" $0 stop\n",
" $0 start\n",
" ;;\n",
" *)\n",
" echo \"Usage: /etc/init.d/xvfb {start|stop|restart}\"\n",
" exit 1\n",
"esac\n",
"exit 0\n",
" \"\"\")\n",
" pro_bar.update(progress(5, 100))\n",
" !apt-get install daemon >/dev/null 2>&1\n",
" pro_bar.update(progress(10, 100))\n",
" !apt-get install wget >/dev/null 2>&1\n",
" pro_bar.update(progress(20, 100))\n",
" !wget http://security.ubuntu.com/ubuntu/pool/main/libx/libxfont/libxfont1_1.5.1-1ubuntu0.16.04.4_amd64.deb >/dev/null 2>&1\n",
" pro_bar.update(progress(30, 100))\n",
" !wget --output-document xvfb.deb http://security.ubuntu.com/ubuntu/pool/universe/x/xorg-server/xvfb_1.18.4-0ubuntu0.12_amd64.deb >/dev/null 2>&1\n",
" pro_bar.update(progress(40, 100))\n",
" !dpkg -i libxfont1_1.5.1-1ubuntu0.16.04.4_amd64.deb >/dev/null 2>&1\n",
" pro_bar.update(progress(50, 100))\n",
" !dpkg -i xvfb.deb >/dev/null 2>&1\n",
" pro_bar.update(progress(70, 100))\n",
" !rm libxfont1_1.5.1-1ubuntu0.16.04.4_amd64.deb\n",
" pro_bar.update(progress(80, 100))\n",
" !rm xvfb.deb\n",
" pro_bar.update(progress(90, 100))\n",
" !bash frame-buffer start\n",
" import os\n",
" os.environ[\"DISPLAY\"] = \":1\"\n",
"pro_bar.update(progress(100, 100))"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "Pzj7wgapAcDs"
},
"source": [
"### Installing ml-agents"
]
},
{
"cell_type": "code",
"metadata": {
"id": "N8yfQqkbebQ5"
},
"source": [
"try:\n",
" import mlagents\n",
" print(\"ml-agents already installed\")\n",
"except ImportError:\n",
" !pip install -q mlagents==0.25.1\n",
" print(\"Installed ml-agents\")"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "_u74YhSmW6gD"
},
"source": [
"## Side Channel\n",
"\n",
"SideChannels are objects that can be passed to the constructor of a UnityEnvironment or the `make()` method of a registry entry to send non Reinforcement Learning related data. \n",
"More information available [here](https://github.com/Unity-Technologies/ml-agents/blob/release_1/docs/Python-API.md#communicating-additional-information-with-the-environment)\n",
"\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {
"id": "U4RXnhLRk7Uc"
},
"source": [
"### Engine Configuration SideChannel\n",
"The [Engine Configuration Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_1/docs/Python-API.md#engineconfigurationchannel) is used to configure how the Unity Engine should run. \n",
"We will use the GridWorld environment to demonstrate how to use the EngineConfigurationChannel. "
]
},
{
"cell_type": "code",
"metadata": {
"id": "YSf-WhxbqtLw"
},
"source": [
"# -----------------\n",
"# This code is used to close an env that might not have been closed before\n",
"try:\n",
" env.close()\n",
"except:\n",
" pass\n",
"# -----------------\n",
"\n",
"from mlagents_envs.registry import default_registry\n",
"env_id = \"GridWorld\"\n",
"\n",
"# Import the EngineConfigurationChannel class\n",
"from mlagents_envs.side_channel.engine_configuration_channel import EngineConfigurationChannel\n",
"\n",
"# Create the side channel\n",
"engine_config_channel = EngineConfigurationChannel()\n",
"\n",
"# Pass the side channel to the make method\n",
"# Note, the make method takes a LIST of SideChannel as input\n",
"env = default_registry[env_id].make(side_channels = [engine_config_channel])\n",
"\n",
"# Configure the Unity Engine\n",
"engine_config_channel.set_configuration_parameters(target_frame_rate = 30)\n",
"\n",
"env.reset()\n",
"\n",
"# ... \n",
"# Perform experiment on environment\n",
"# ...\n",
"\n",
"env.close()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "h1lIx3_l24OP"
},
"source": [
"### Environment Parameters Channel\n",
"The [Environment Parameters Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_1/docs/Python-API.md#environmentparameters) is used to modify environment parameters during the simulation. \n",
"We will use the GridWorld environment to demonstrate how to use the EngineConfigurationChannel. "
]
},
{
"cell_type": "code",
"metadata": {
"id": "dhtl0mpeqxYi"
},
"source": [
"import matplotlib.pyplot as plt\n",
"%matplotlib inline\n",
"\n",
"# -----------------\n",
"# This code is used to close an env that might not have been closed before\n",
"try:\n",
" env.close()\n",
"except:\n",
" pass\n",
"# -----------------\n",
"\n",
"from mlagents_envs.registry import default_registry\n",
"env_id = \"GridWorld\"\n",
"\n",
"# Import the EngineConfigurationChannel class\n",
"from mlagents_envs.side_channel.environment_parameters_channel import EnvironmentParametersChannel\n",
"\n",
"# Create the side channel\n",
"env_parameters = EnvironmentParametersChannel()\n",
"\n",
"# Pass the side channel to the make method\n",
"# Note, the make method takes a LIST of SideChannel as input\n",
"env = default_registry[env_id].make(side_channels = [env_parameters])\n",
"\n",
"env.reset()\n",
"behavior_name = list(env.behavior_specs)[0] \n",
"\n",
"print(\"Observation without changing the environment parameters\")\n",
"decision_steps, terminal_steps = env.get_steps(behavior_name)\n",
"plt.imshow(decision_steps.obs[0][0,:,:,:])\n",
"plt.show()\n",
"\n",
"print(\"Increasing the dimensions of the grid from 5 to 7\")\n",
"env_parameters.set_float_parameter(\"gridSize\", 7)\n",
"print(\"Increasing the number of X from 1 to 5\")\n",
"env_parameters.set_float_parameter(\"numObstacles\", 5)\n",
"\n",
"# Any change to a SideChannel will only be effective after a step or reset\n",
"# In the GridWorld Environment, the grid's dimensions can only change at reset\n",
"env.reset()\n",
"\n",
"\n",
"decision_steps, terminal_steps = env.get_steps(behavior_name)\n",
"plt.imshow(decision_steps.obs[0][0,:,:,:])\n",
"plt.show()\n",
"\n",
"\n",
"\n",
"env.close()"
],
"execution_count": null,
"outputs": []
},
{
"cell_type": "markdown",
"metadata": {
"id": "k1rwnVq2qyoO"
},
"source": [
"### Creating your own Side Channels\n",
"You can send various kinds of data between a Unity Environment and Python but you will need to [create your own implementation of a Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_1/docs/Custom-SideChannels.md#custom-side-channels) for advanced use cases.\n"
]
}
]
}