{ "nbformat": 4, "nbformat_minor": 0, "metadata": { "colab": { "name": "Colab-UnityEnvironment-3-SideChannel.ipynb", "private_outputs": true, "provenance": [], "collapsed_sections": [], "toc_visible": true }, "kernelspec": { "name": "python3", "language": "python", "display_name": "Python 3" }, "pycharm": { "stem_cell": { "cell_type": "raw", "source": [], "metadata": { "collapsed": false } } } }, "cells": [ { "cell_type": "markdown", "metadata": { "id": "pbVXrmEsLXDt" }, "source": [ "# ML-Agents Use SideChannels\n", "" ] }, { "cell_type": "markdown", "metadata": { "id": "WNKTwHU3d2-l" }, "source": [ "## Setup" ] }, { "cell_type": "code", "metadata": { "id": "htb-p1hSNX7D" }, "source": [ "#@title Install Rendering Dependencies { display-mode: \"form\" }\n", "#@markdown (You only need to run this code when using Colab's hosted runtime)\n", "\n", "import os\n", "from IPython.display import HTML, display\n", "\n", "def progress(value, max=100):\n", " return HTML(\"\"\"\n", " \n", " {value}\n", " \n", " \"\"\".format(value=value, max=max))\n", "\n", "pro_bar = display(progress(0, 100), display_id=True)\n", "\n", "try:\n", " import google.colab\n", " INSTALL_XVFB = True\n", "except ImportError:\n", " INSTALL_XVFB = 'COLAB_ALWAYS_INSTALL_XVFB' in os.environ\n", "\n", "if INSTALL_XVFB:\n", " with open('frame-buffer', 'w') as writefile:\n", " writefile.write(\"\"\"#taken from https://gist.github.com/jterrace/2911875\n", "XVFB=/usr/bin/Xvfb\n", "XVFBARGS=\":1 -screen 0 1024x768x24 -ac +extension GLX +render -noreset\"\n", "PIDFILE=./frame-buffer.pid\n", "case \"$1\" in\n", " start)\n", " echo -n \"Starting virtual X frame buffer: Xvfb\"\n", " /sbin/start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile --background --exec $XVFB -- $XVFBARGS\n", " echo \".\"\n", " ;;\n", " stop)\n", " echo -n \"Stopping virtual X frame buffer: Xvfb\"\n", " /sbin/start-stop-daemon --stop --quiet --pidfile $PIDFILE\n", " rm $PIDFILE\n", " echo \".\"\n", " ;;\n", " restart)\n", " $0 stop\n", " $0 start\n", " ;;\n", " *)\n", " echo \"Usage: /etc/init.d/xvfb {start|stop|restart}\"\n", " exit 1\n", "esac\n", "exit 0\n", " \"\"\")\n", " pro_bar.update(progress(5, 100))\n", " !apt-get install daemon >/dev/null 2>&1\n", " pro_bar.update(progress(10, 100))\n", " !apt-get install wget >/dev/null 2>&1\n", " pro_bar.update(progress(20, 100))\n", " !wget http://security.ubuntu.com/ubuntu/pool/main/libx/libxfont/libxfont1_1.5.1-1ubuntu0.16.04.4_amd64.deb >/dev/null 2>&1\n", " pro_bar.update(progress(30, 100))\n", " !wget --output-document xvfb.deb http://security.ubuntu.com/ubuntu/pool/universe/x/xorg-server/xvfb_1.18.4-0ubuntu0.12_amd64.deb >/dev/null 2>&1\n", " pro_bar.update(progress(40, 100))\n", " !dpkg -i libxfont1_1.5.1-1ubuntu0.16.04.4_amd64.deb >/dev/null 2>&1\n", " pro_bar.update(progress(50, 100))\n", " !dpkg -i xvfb.deb >/dev/null 2>&1\n", " pro_bar.update(progress(70, 100))\n", " !rm libxfont1_1.5.1-1ubuntu0.16.04.4_amd64.deb\n", " pro_bar.update(progress(80, 100))\n", " !rm xvfb.deb\n", " pro_bar.update(progress(90, 100))\n", " !bash frame-buffer start\n", " os.environ[\"DISPLAY\"] = \":1\"\n", "pro_bar.update(progress(100, 100))" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "Pzj7wgapAcDs" }, "source": [ "### Installing ml-agents" ] }, { "cell_type": "code", "metadata": { "id": "N8yfQqkbebQ5" }, "source": [ "try:\n", " import mlagents\n", " print(\"ml-agents already installed\")\n", "except ImportError:\n", " !python -m pip install -q mlagents==0.27.0\n", " print(\"Installed ml-agents\")" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "_u74YhSmW6gD" }, "source": [ "## Side Channel\n", "\n", "SideChannels are objects that can be passed to the constructor of a UnityEnvironment or the `make()` method of a registry entry to send non Reinforcement Learning related data.\n", "More information available [here](https://github.com/Unity-Technologies/ml-agents/blob/release_18_docs/docs/Python-API.md#communicating-additional-information-with-the-environment)\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "id": "U4RXnhLRk7Uc" }, "source": [ "### Engine Configuration SideChannel\n", "The [Engine Configuration Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_18_docs/docs/Python-API.md#engineconfigurationchannel) is used to configure how the Unity Engine should run.\n", "We will use the GridWorld environment to demonstrate how to use the EngineConfigurationChannel." ] }, { "cell_type": "code", "metadata": { "id": "YSf-WhxbqtLw" }, "source": [ "# -----------------\n", "# This code is used to close an env that might not have been closed before\n", "try:\n", " env.close()\n", "except:\n", " pass\n", "# -----------------\n", "\n", "from mlagents_envs.registry import default_registry\n", "env_id = \"GridWorld\"\n", "\n", "# Import the EngineConfigurationChannel class\n", "from mlagents_envs.side_channel.engine_configuration_channel import EngineConfigurationChannel\n", "\n", "# Create the side channel\n", "engine_config_channel = EngineConfigurationChannel()\n", "\n", "# Pass the side channel to the make method\n", "# Note, the make method takes a LIST of SideChannel as input\n", "env = default_registry[env_id].make(side_channels = [engine_config_channel])\n", "\n", "# Configure the Unity Engine\n", "engine_config_channel.set_configuration_parameters(target_frame_rate = 30)\n", "\n", "env.reset()\n", "\n", "# ...\n", "# Perform experiment on environment\n", "# ...\n", "\n", "env.close()" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "h1lIx3_l24OP" }, "source": [ "### Environment Parameters Channel\n", "The [Environment Parameters Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_18_docs/docs/Python-API.md#environmentparameters) is used to modify environment parameters during the simulation.\n", "We will use the GridWorld environment to demonstrate how to use the EngineConfigurationChannel." ] }, { "cell_type": "code", "metadata": { "id": "dhtl0mpeqxYi" }, "source": [ "import matplotlib.pyplot as plt\n", "%matplotlib inline\n", "\n", "# -----------------\n", "# This code is used to close an env that might not have been closed before\n", "try:\n", " env.close()\n", "except:\n", " pass\n", "# -----------------\n", "\n", "from mlagents_envs.registry import default_registry\n", "env_id = \"GridWorld\"\n", "\n", "# Import the EngineConfigurationChannel class\n", "from mlagents_envs.side_channel.environment_parameters_channel import EnvironmentParametersChannel\n", "\n", "# Create the side channel\n", "env_parameters = EnvironmentParametersChannel()\n", "\n", "# Pass the side channel to the make method\n", "# Note, the make method takes a LIST of SideChannel as input\n", "env = default_registry[env_id].make(side_channels = [env_parameters])\n", "\n", "env.reset()\n", "behavior_name = list(env.behavior_specs)[0]\n", "\n", "print(\"Observation without changing the environment parameters\")\n", "decision_steps, terminal_steps = env.get_steps(behavior_name)\n", "plt.imshow(decision_steps.obs[0][0,:,:,:])\n", "plt.show()\n", "\n", "print(\"Increasing the dimensions of the grid from 5 to 7\")\n", "env_parameters.set_float_parameter(\"gridSize\", 7)\n", "print(\"Increasing the number of X from 1 to 5\")\n", "env_parameters.set_float_parameter(\"numObstacles\", 5)\n", "\n", "# Any change to a SideChannel will only be effective after a step or reset\n", "# In the GridWorld Environment, the grid's dimensions can only change at reset\n", "env.reset()\n", "\n", "\n", "decision_steps, terminal_steps = env.get_steps(behavior_name)\n", "plt.imshow(decision_steps.obs[0][0,:,:,:])\n", "plt.show()\n", "\n", "\n", "\n", "env.close()" ], "execution_count": null, "outputs": [] }, { "cell_type": "markdown", "metadata": { "id": "k1rwnVq2qyoO" }, "source": [ "### Creating your own Side Channels\n", "You can send various kinds of data between a Unity Environment and Python but you will need to [create your own implementation of a Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_18_docs/docs/Custom-SideChannels.md#custom-side-channels) for advanced use cases.\n" ] } ] }