浏览代码

Updated documentation

/aisv647_visualizations
Steven Borkman 4 年前
当前提交
0062f3db
共有 11 个文件被更改,包括 1721 次插入35 次删除
  1. 4
      TestProjects/PerceptionURP/Assets/Scenes/SampleScene.unity
  2. 17
      com.unity.perception/CHANGELOG.md
  3. 78
      com.unity.perception/Documentation~/GettingStarted.md
  4. 2
      com.unity.perception/Runtime/GroundTruth/Labelers/CameraLabeler.cs
  5. 21
      com.unity.perception/Runtime/GroundTruth/Labelers/Visualization/ControlPanel.cs
  6. 14
      com.unity.perception/Runtime/GroundTruth/Labelers/Visualization/HUDPanel.cs
  7. 4
      com.unity.perception/Runtime/GroundTruth/Labelers/Visualization/KeyValuePanel.cs
  8. 13
      com.unity.perception/Runtime/GroundTruth/Labelers/Visualization/VisualizationCanvas.cs
  9. 3
      com.unity.perception/Runtime/GroundTruth/PerceptionCamera.cs
  10. 1001
      com.unity.perception/Documentation~/images/controls.gif
  11. 599
      com.unity.perception/Documentation~/images/visualized.png

4
TestProjects/PerceptionURP/Assets/Scenes/SampleScene.unity


- id: 1
- id: 2
- id: 3
m_VisualizationEnabled: 1
showVisualizations: 1
references:
version: 1
00000000:

type: {class: SemanticSegmentationLabeler, ns: UnityEngine.Perception.GroundTruth,
asm: Unity.Perception.Runtime}
data:
enabled: 0
enabled: 1
annotationId: 12F94D8D-5425-4DEB-9B21-5E53AD957D66
labelConfig: {fileID: 11400000, guid: c140c5aa05dd09e4fadaa26de31b1f39, type: 2}
m_TargetTextureOverride: {fileID: 0}

17
com.unity.perception/CHANGELOG.md


# Changelog
All notable changes to this package will be documented in this file.
The format is based on [Keep a Changelog](http://keepachangelog.com/en/1.0.0/)

### Fixed
### Security
## [0.2.0] - 2020-07-31
### Added
Added realtime visualization capability to the perception package.
Added visualizers for built-in labelers: Semantic Segmentation, 2D Bounding Boxes, Object Count, and Render Object Info.
### This is the first release of the _Perception_ package

78
com.unity.perception/Documentation~/GettingStarted.md


# Getting Started with Perception
This step can be skipped for HDRP projects.
1. Select your project's `ScriptableRenderer` asset and open the inspector window. In most projects it is located at `Assets/Settings/ForwardRenderer.asset`.

## Step 3: Create a new scene and camera
3. Select the Main Camera and reset the Position transform to 0
3. Select the Main Camera and reset the Position transform to 0
1. In the inspector panel of the main camera select Add Component
2. Add a **Perception Camera** component
1. In the inspector panel of the main camera select Add Component
2. Add a **Perception Camera** component
1. Create a cube by right-clicking in the Hierarchy window, select 3D Object -> Cube
1. Create 2 more cubes this way
1. Change the names of the cubes to Cube, Box, and Crate
1. Position the Cubes in front of the main Camera
1. Create a cube by right-clicking in the Hierarchy window, select 3D Object -> Cube
2. Create 2 more cubes this way
3. Change the names of the cubes to Cube, Box, and Crate
4. Position the Cubes in front of the main Camera
1. On each cube, from the inspector panel add a **Labeling** component
1. Click the **+**
2. In the text field add the name of the object i.e Crate. This will be the label used in the semantic segmentation images
<img src="images/LabeledObject.PNG" align="middle"/>
1. On each cube, from the inspector panel add a **Labeling** component
1. Click the **+**
2. In the text field add the name of the object i.e Crate. This will be the label used in the semantic segmentation images
<img src="images/LabeledObject.PNG" align="middle"/>
2. In the Project panel right click -> Perception -> Labeling Configuration
3. Select the new **Labeling Configuration**
1. Click the **+**
2. In the label text field add the same text that the Label script contains on the objects created in the scene (i.e Cube, Box, Crate)
3. Add a numerical value to the value field
1. In the Project panel right click -> Perception -> Labeling Configuration
1. Select the new **Labeling Configuration**
1. Click the **+**
2. In the label text field add the same text that the Label script contains on the objects created in the scene (i.e Cube, Box, Crate)
3. Add a numerical value to the value field
Make sure the labels all have different values, for this example use increments of 10,000 to ensure they show up as very distinct colors in the segmentation images
Make sure the labels all have different values, for this example use increments of 10,000 to ensure they show up as very distinct colors in the segmentation images
1. Select the Main Camera in the Hierarchy panel
1. In the Perception Camera attach the Labeling Configuration created in previous step
1. Select the Main Camera in the Hierarchy panel
2. In the Perception Camera attach the Labeling Configuration created in previous step
>Example file path on a Windows PC : `C:/Users/<User Name>/AppData/LocalLow/DefaultCompany/UnityTestFramework\2e10ec21-9d97-4cee-b5a2-7e95e299afa4\RGB18f61842-ef8d-4b31-acb5-cb1da36fb7b1`
> Example file path on a Windows PC : `C:/Users/<User Name>/AppData/LocalLow/DefaultCompany/UnityTestFramework\2e10ec21-9d97-4cee-b5a2-7e95e299afa4\RGB18f61842-ef8d-4b31-acb5-cb1da36fb7b1`
- RGB captures
- Semantic segmentation images
- Logs
- JSON dataset
- RGB captures
- Semantic segmentation images
- Logs
- JSON dataset
<img src="images/rgb_2.png" align="middle"/>
_RGB image_

_Example semantic segmentation image_
_Example semantic segmentation image_
## Optional Step: Realtime visualization of labelers
The perception package now comes with the ability to show realtime results of the labeler in the scene. To enable this capability:
<img src="images/visualized.png" align="middle"/>
_Example of perception running with show visualizations on_
1. To use the visualizer, verify that *Show Visualizations* is checked on in the Inspector pane. This turns on the built in labelers which includes segmentation data, 2D bounding boxes, pixel and object counts.
2. Turning on the visualizer creates new UI controls in the editor's game view. These controls allow you to atomically control each of the individual visualizers. Each individual can be turned on/off on their own. Some visualizers also include controls to change their output.
<img src="images/controls.gif" align="middle"/>
_Visualization controls in action_
***Important Note:*** The perception package takes advantage of asynchronous processing to ensure reasonable frame rates of a scene. A side effect of realtime visualization is that the labelers have to be applied to the capture in its actual frame, which will potentially adversely affect the scene's framerate.

2
com.unity.perception/Runtime/GroundTruth/Labelers/CameraLabeler.cs


/// <summary>
/// Called immediately after <see cref="setup"/>. Implement this to initialize labeler's visualization
/// capability if one exists <see cref="supportVisualization"/>.
/// <param name="panel">The target control panel for the labeler's visualization component</param>
/// <param name="enabled">The current enabled state of the visualizer</param>
/// </summary>
protected virtual void OnVisualizerEnabledChanged(bool enabled) {}
/// <summary>

21
com.unity.perception/Runtime/GroundTruth/Labelers/Visualization/ControlPanel.cs


/// <summary>
/// Adds a new UI control to the control panel. If the control cannot be added and the method will
/// return false. Also, all UI elements must have a LayoutElement component and if they do not,
/// return false. Also, all UI elements must have a LayoutElement component and if they do not,
/// <param name="uiControl">The control that is to be added to the control panel</param>
/// <returns>True if the control could be added, false if it there was a problem adding it</returns>
/// </summary>
public bool AddNewControl(GameObject uiControl)
{

/// Removes the passed in component from the control panel. Returns
/// false if the element does not exist. Returns true on a successful removal.
/// The caller is responsible for destroying the control.
/// <param name="uiControl">The control that needs to be removed from the panel</param>
/// <returns>True if the control could be removed, false if there was an issue removing the control</returns>
/// </summary>
public bool RemoveControl(GameObject uiControl)
{

}
/// <summary>
/// Creates a new toogle control with passed in name. The passed in listener will be
/// Creates a new toggle control with passed in name. The passed in listener will be
/// Returns the control panel elemet upon a succssful add.
/// Returns the control panel element upon a successful add.
/// <param name="name">The name of the toggle</param>
/// <param name="listener">The callback action that will be triggered when the toggle's state changes</param>
/// <returns>The created toggle</returns>
/// </summary>
public GameObject AddToggleControl(string name, UnityAction<bool> listener)
{

}
/// <summary>
/// Creates a new slider control with the passed in name and default value. The passed in listener will be
/// called on slider changes. If anything goes wrong this method will return null.
/// Creates a new slider control with the passed in name and default value. The slider's value runs from 0 to 1.
/// The passed in listener will be called on slider changes. If anything goes wrong this method will return null.
/// <param name="name">The name of the slider control</param>
/// <param name="defaultValue">The default value of the slider, between 0 and 1</param>
/// <param name="listener">The callback action that will be triggered when the slider's value changes</param>
/// <returns>The created slider</returns>
/// </summary>
public GameObject AddSliderControl(string name, float defaultValue, UnityAction<float> listener)
{

14
com.unity.perception/Runtime/GroundTruth/Labelers/Visualization/HUDPanel.cs


Dictionary<string, (bool, KeyValuePanel)> entries = new Dictionary<string, (bool, KeyValuePanel)>();
Stack<KeyValuePanel> orphans = new Stack<KeyValuePanel>();
/// <summary>
/// The panel that will hold all of the key value panel elements, this reference is needed to be able to hide the panel
/// </summary>
/// <summary>
/// The scroll rect of the HUD panel, this reference is needed to be able to hide the panel
/// </summary>
/// <summary>
/// The background image of the HUD panel, this reference is needed to be able to hide the panel
/// </summary>
public Image img = null;
void Update()

/// <summary>
/// Updates (or creates) an entry with the passed in key value pair
/// <param name="key">The key of the HUD entry</param>
/// <param name="value">The value of the entry</param>
/// </summary>
public void UpdateEntry(string key, string value)
{

/// <summary>
/// Removes the key value pair from the HUD
/// <param name="key">The key of the entry to remove</param>
/// </summary>
public void RemoveEntry(string key)
{

/// <summary>
/// Removes all of the passed in entries from the HUD
/// <param name="keys">List of keys to remove from the panel</param>
/// </summary>
public void RemoveEntries(List<string> keys)
{

/// <summary>
/// Removes all of the passed in entries from the HUD
/// <param name="keys">List of keys t remove from the panel</param>
/// </summary>
public void RemoveEntries(string[] keys)
{

4
com.unity.perception/Runtime/GroundTruth/Labelers/Visualization/KeyValuePanel.cs


namespace UnityEngine.Perception.GroundTruth
{
/// <summary>
///
/// Key value pair panel UI object
/// </summary>
public class KeyValuePanel : MonoBehaviour
{

/// <summary>
/// Sets the key of this key value pair
/// <param name="k">The key of the key/value pair</param>
/// </summary>
public void SetKey(string k)
{

/// <summary>
/// Sets the value of this key value pair
/// <param name="v">The value of the key/value pair</param>
/// </summary>
public void SetValue(string v)
{

13
com.unity.perception/Runtime/GroundTruth/Labelers/Visualization/VisualizationCanvas.cs


/// </summary>
public class VisualizationCanvas : MonoBehaviour
{
/// <summary>
/// The control panel contains the UI control elements used to interact with the labelers.
/// </summary>
/// <summary>
/// The HUD panel displays realtime key/value pair data on a UI panel.
/// </summary>
/// <summary>
/// Game object which acts as the scene container for all of the dynamic labeler visuals
/// </summary>
public GameObject dynaicContentHolder;
// Start is called before the first frame update

/// Adds a new UI components to the visualization canvas. Pass in fullscreen if the component should take up
/// the entire screen. Pass in setAsLowestElement if the component should be rendered behind all other components.
/// This method will return false if the element could not be added, true if everything works properly.
/// <param name="component">UI component that should be added to this UI canvas</param>
/// <param name="fullScreen">Should this component's rect transform be set to fill the entire dimensions of the parent, defaults to true</param>
/// <param name="setAsLowestElement">Should this UI component be rendered as the lowest UI component in the scene, defaults to false</param>
/// <returns>True if the component was added properly, false if an error occurred.</returns>
/// </summary>
public bool AddComponent(GameObject component, bool fullScreen = true, bool setAsLowestElement = false)
{

3
com.unity.perception/Runtime/GroundTruth/PerceptionCamera.cs


static GameObject s_VisualizationCamera;
static GameObject s_VisualizationCanvas;
/// <summary>
/// Turns on/off the realtime visualization capability.
/// </summary>
[SerializeField]
public bool showVisualizations = true;

1001
com.unity.perception/Documentation~/images/controls.gif
文件差异内容过多而无法显示
查看文件

599
com.unity.perception/Documentation~/images/visualized.png
文件差异内容过多而无法显示
查看文件

正在加载...
取消
保存