ml-agents/colab/Colab_UnityEnvironment_3_SideChannel.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Colab-UnityEnvironment-3-SideChannel.ipynb",
      "private_outputs": true,
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "pbVXrmEsLXDt"
      },
      "source": [
        "# ML-Agents Use SideChannels\n",
        "<img src=\"https://raw.githubusercontent.com/Unity-Technologies/ml-agents/release_1/docs/images/3dball_big.png\" align=\"middle\" width=\"435\"/>"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "WNKTwHU3d2-l"
      },
      "source": [
        "## Setup"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "htb-p1hSNX7D"
      },
      "source": [
        "#@title Install Rendering Dependencies { display-mode: \"form\" }\n",
        "#@markdown (You only need to run this code when using Colab's hosted runtime)\n",
        "\n",
        "from IPython.display import HTML, display\n",
        "\n",
        "def progress(value, max=100):\n",
        "    return HTML(\"\"\"\n",
        "        <progress\n",
        "            value='{value}'\n",
        "            max='{max}',\n",
        "            style='width: 100%'\n",
        "        >\n",
        "            {value}\n",
        "        </progress>\n",
        "    \"\"\".format(value=value, max=max))\n",
        "\n",
        "pro_bar = display(progress(0, 100), display_id=True)\n",
        "\n",
        "try:\n",
        "  import google.colab\n",
        "  IN_COLAB = True\n",
        "except ImportError:\n",
        "  IN_COLAB = False\n",
        "\n",
        "if IN_COLAB:\n",
        "  with open('frame-buffer', 'w') as writefile:\n",
        "    writefile.write(\"\"\"#taken from https://gist.github.com/jterrace/2911875\n",
        "XVFB=/usr/bin/Xvfb\n",
        "XVFBARGS=\":1 -screen 0 1024x768x24 -ac +extension GLX +render -noreset\"\n",
        "PIDFILE=./frame-buffer.pid\n",
        "case \"$1\" in\n",
        "  start)\n",
        "    echo -n \"Starting virtual X frame buffer: Xvfb\"\n",
        "    /sbin/start-stop-daemon --start --quiet --pidfile $PIDFILE --make-pidfile --background --exec $XVFB -- $XVFBARGS\n",
        "    echo \".\"\n",
        "    ;;\n",
        "  stop)\n",
        "    echo -n \"Stopping virtual X frame buffer: Xvfb\"\n",
        "    /sbin/start-stop-daemon --stop --quiet --pidfile $PIDFILE\n",
        "    rm $PIDFILE\n",
        "    echo \".\"\n",
        "    ;;\n",
        "  restart)\n",
        "    $0 stop\n",
        "    $0 start\n",
        "    ;;\n",
        "  *)\n",
        "        echo \"Usage: /etc/init.d/xvfb {start|stop|restart}\"\n",
        "        exit 1\n",
        "esac\n",
        "exit 0\n",
        "    \"\"\")\n",
        "  pro_bar.update(progress(5, 100))\n",
        "  !apt-get install daemon >/dev/null 2>&1\n",
        "  pro_bar.update(progress(10, 100))\n",
        "  !apt-get install wget >/dev/null 2>&1\n",
        "  pro_bar.update(progress(20, 100))\n",
        "  !wget http://security.ubuntu.com/ubuntu/pool/main/libx/libxfont/libxfont1_1.5.1-1ubuntu0.16.04.4_amd64.deb >/dev/null 2>&1\n",
        "  pro_bar.update(progress(30, 100))\n",
        "  !wget --output-document xvfb.deb http://security.ubuntu.com/ubuntu/pool/universe/x/xorg-server/xvfb_1.18.4-0ubuntu0.12_amd64.deb >/dev/null 2>&1\n",
        "  pro_bar.update(progress(40, 100))\n",
        "  !dpkg -i libxfont1_1.5.1-1ubuntu0.16.04.4_amd64.deb >/dev/null 2>&1\n",
        "  pro_bar.update(progress(50, 100))\n",
        "  !dpkg -i xvfb.deb >/dev/null 2>&1\n",
        "  pro_bar.update(progress(70, 100))\n",
        "  !rm libxfont1_1.5.1-1ubuntu0.16.04.4_amd64.deb\n",
        "  pro_bar.update(progress(80, 100))\n",
        "  !rm xvfb.deb\n",
        "  pro_bar.update(progress(90, 100))\n",
        "  !bash frame-buffer start\n",
        "  import os\n",
        "  os.environ[\"DISPLAY\"] = \":1\"\n",
        "pro_bar.update(progress(100, 100))"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "Pzj7wgapAcDs"
      },
      "source": [
        "### Installing ml-agents"
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "N8yfQqkbebQ5"
      },
      "source": [
        "try:\n",
        "  import mlagents\n",
        "  print(\"ml-agents already installed\")\n",
        "except ImportError:\n",
        "  !pip install -q mlagents==0.25.1\n",
        "  print(\"Installed ml-agents\")"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "_u74YhSmW6gD"
      },
      "source": [
        "## Side Channel\n",
        "\n",
        "SideChannels are objects that can be passed to the constructor of a UnityEnvironment or the `make()` method of a registry entry to send non Reinforcement Learning related data. \n",
        "More information available [here](https://github.com/Unity-Technologies/ml-agents/blob/release_1/docs/Python-API.md#communicating-additional-information-with-the-environment)\n",
        "\n",
        "\n",
        "\n"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "U4RXnhLRk7Uc"
      },
      "source": [
        "### Engine Configuration SideChannel\n",
        "The [Engine Configuration Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_1/docs/Python-API.md#engineconfigurationchannel) is used to configure how the Unity Engine should run. \n",
        "We will use the GridWorld environment to demonstrate how to use the EngineConfigurationChannel. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "YSf-WhxbqtLw"
      },
      "source": [
        "# -----------------\n",
        "# This code is used to close an env that might not have been closed before\n",
        "try:\n",
        "  env.close()\n",
        "except:\n",
        "  pass\n",
        "# -----------------\n",
        "\n",
        "from mlagents_envs.registry import default_registry\n",
        "env_id = \"GridWorld\"\n",
        "\n",
        "# Import the EngineConfigurationChannel class\n",
        "from mlagents_envs.side_channel.engine_configuration_channel import EngineConfigurationChannel\n",
        "\n",
        "# Create the side channel\n",
        "engine_config_channel = EngineConfigurationChannel()\n",
        "\n",
        "# Pass the side channel to the make method\n",
        "# Note, the make method takes a LIST of SideChannel as input\n",
        "env = default_registry[env_id].make(side_channels = [engine_config_channel])\n",
        "\n",
        "# Configure the Unity Engine\n",
        "engine_config_channel.set_configuration_parameters(target_frame_rate = 30)\n",
        "\n",
        "env.reset()\n",
        "\n",
        "# ... \n",
        "# Perform experiment on environment\n",
        "# ...\n",
        "\n",
        "env.close()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "h1lIx3_l24OP"
      },
      "source": [
        "### Environment Parameters Channel\n",
        "The [Environment Parameters Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_1/docs/Python-API.md#environmentparameters) is used to modify environment parameters during the simulation. \n",
        "We will use the GridWorld environment to demonstrate how to use the EngineConfigurationChannel. "
      ]
    },
    {
      "cell_type": "code",
      "metadata": {
        "id": "dhtl0mpeqxYi"
      },
      "source": [
        "import matplotlib.pyplot as plt\n",
        "%matplotlib inline\n",
        "\n",
        "# -----------------\n",
        "# This code is used to close an env that might not have been closed before\n",
        "try:\n",
        "  env.close()\n",
        "except:\n",
        "  pass\n",
        "# -----------------\n",
        "\n",
        "from mlagents_envs.registry import default_registry\n",
        "env_id = \"GridWorld\"\n",
        "\n",
        "# Import the EngineConfigurationChannel class\n",
        "from mlagents_envs.side_channel.environment_parameters_channel import EnvironmentParametersChannel\n",
        "\n",
        "# Create the side channel\n",
        "env_parameters = EnvironmentParametersChannel()\n",
        "\n",
        "# Pass the side channel to the make method\n",
        "# Note, the make method takes a LIST of SideChannel as input\n",
        "env = default_registry[env_id].make(side_channels = [env_parameters])\n",
        "\n",
        "env.reset()\n",
        "behavior_name = list(env.behavior_specs)[0] \n",
        "\n",
        "print(\"Observation without changing the environment parameters\")\n",
        "decision_steps, terminal_steps = env.get_steps(behavior_name)\n",
        "plt.imshow(decision_steps.obs[0][0,:,:,:])\n",
        "plt.show()\n",
        "\n",
        "print(\"Increasing the dimensions of the grid from 5 to 7\")\n",
        "env_parameters.set_float_parameter(\"gridSize\", 7)\n",
        "print(\"Increasing the number of X from 1 to 5\")\n",
        "env_parameters.set_float_parameter(\"numObstacles\", 5)\n",
        "\n",
        "# Any change to a SideChannel will only be effective after a step or reset\n",
        "# In the GridWorld Environment, the grid's dimensions can only change at reset\n",
        "env.reset()\n",
        "\n",
        "\n",
        "decision_steps, terminal_steps = env.get_steps(behavior_name)\n",
        "plt.imshow(decision_steps.obs[0][0,:,:,:])\n",
        "plt.show()\n",
        "\n",
        "\n",
        "\n",
        "env.close()"
      ],
      "execution_count": null,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "k1rwnVq2qyoO"
      },
      "source": [
        "### Creating your own Side Channels\n",
        "You can send various kinds of data between a Unity Environment and Python but you will need to [create your own implementation of a Side Channel](https://github.com/Unity-Technologies/ml-agents/blob/release_1/docs/Custom-SideChannels.md#custom-side-channels) for advanced use cases.\n"
      ]
    }
  ]
}