浏览代码

Add note on curriculum learning documentation (#3723)

* Add note on environment

* Remove potentially misleading graph

* Add note on resuming training w/ curriculum
/develop/dockerfile
GitHub 4 年前
当前提交
ca43af24
共有 2 个文件被更改,包括 11 次插入277 次删除
  1. 28
      docs/Training-Curriculum-Learning.md
  2. 260
      docs/images/curriculum_progress.png

28
docs/Training-Curriculum-Learning.md


# Training with Curriculum Learning
## Sample Environment
Curriculum learning is a feature of ML-Agents which allows for the properties of environments to be changed during the training process to aid in learning.
## An Instructional Example
*[**Note**: The example provided below is for instructional purposes, and was based on an early version of the [Wall Jump example environment](Example-Environments.md). As such, it is not possible to directly replicate the results here using that environment.]*
Imagine a task in which an agent needs to scale a wall to arrive at a goal. The
starting point when training an agent to accomplish this task will be a random

then the agent can easily learn to accomplish the task. From there, we can
slowly add to the difficulty of the task by increasing the size of the wall
until the agent can complete the initially near-impossible task of scaling the
wall. We have included an environment to demonstrate this with ML-Agents,
called __Wall Jump__.
wall.
_Demonstration of a curriculum training scenario in which a progressively taller
_Demonstration of a hypothetical curriculum training scenario in which a progressively taller
To see curriculum learning in action, observe the two learning curves below. Each
displays the reward over time for an agent trained using PPO with the same set of
training hyperparameters. The difference is that one agent was trained using the
full-height wall version of the task, and the other agent was trained using the
curriculum version of the task. As you can see, without using curriculum
learning the agent has a lot of difficulty. We think that by using well-crafted
curricula, agents trained using reinforcement learning will be able to
accomplish tasks otherwise much more difficult.
![Log](images/curriculum_progress.png)
## How-To

Once we have specified our metacurriculum and curricula, we can launch
`mlagents-learn` using the `–curriculum` flag to point to the config file
for our curricula and PPO will train using Curriculum Learning. For example,
to train agents in the Wall Jump environment with curriculum learning, we can run:
to train agents in the Wall Jump environment with curriculum learning, you can run:
We can then keep track of the current lessons and progresses via TensorBoard.
You can then keep track of the current lessons and progresses via TensorBoard.
__Note__: If you are resuming a training session that uses curriculum, please pass the number of the last-reached lesson using the `--lesson` flag when running `mlagents-learn`.

260
docs/images/curriculum_progress.png

之前 之后
正在加载...
取消
保存