|
|
|
|
|
|
|
|
|
|
Regardless of the training method deployed, there are a few model types that |
|
|
|
users can train using the ML-Agents Toolkit. This is due to the flexibility in |
|
|
|
defining agent observations, which can include vector, ray cast and visual |
|
|
|
defining agent observations, which include vector, ray cast and visual |
|
|
|
observations. You can learn more about how to instrument an agent's observation |
|
|
|
in the [Designing Agents](Learning-Environment-Design-Agents.md) guide. |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
The choice of the architecture depends on the visual complexity of the scene and |
|
|
|
the available computational resources. |
|
|
|
|
|
|
|
### Learning from Variable Length Observations using Attention |
|
|
|
|
|
|
|
Using the ML-Agents toolkit, it is possible to have agents learn from a |
|
|
|
varying number of inputs. To do so, each agent can keep track of a buffer |
|
|
|
of vector observations. At each step, the agent will go through all the |
|
|
|
elements in the buffer and extract information but the elements |
|
|
|
in the buffer can change at every step. |
|
|
|
This can be useful in scenarios in which the agents must keep track of |
|
|
|
a varying number of elements throughout the episode. You can learn more |
|
|
|
about variable length observations and the BufferSensor |
|
|
|
[here](Learning-Environment-Design-Agents.md#variable-length-observations) |
|
|
|
When variable length observations are utilized, the ML-Agents Toolkit |
|
|
|
leverages attention networks to learn from a varying number of entities. |
|
|
|
Agents using attention will ignore entities that are deemed not relevant |
|
|
|
and pay special attention to entities relevant to the current situation |
|
|
|
based on context. |
|
|
|
|
|
|
|
### Memory-enhanced Agents using Recurrent Neural Networks |
|
|
|
|
|
|
|