Using [Time.CaptureDeltaTime](https://docs.unity3d.com/ScriptReference/Time-captureDeltaTime.html), it also decouples wall clock time from simulation time, allowing the simulation to run as fast as possible.
## Custom sensors
You can register custom sensors using `DatasetCapture.RegisterSensor()`. The `period` you pass in at registration time determines how often in simulation time frames should be scheduled for the sensor to run. The sensor implementation then checks `ShouldCaptureThisFrame` on the returned `SensorHandle` each frame to determine whether it is time for the sensor to perform a capture. `SensorHandle.ReportCapture` should then be called in each of these frames to report the state of the sensor to populate the dataset.
You can register custom sensors using `DatasetCapture.RegisterSensor()`. The `period` you pass in at registration time determines how often (in simulation time) frames should be scheduled for the sensor to run. The sensor implementation then checks `ShouldCaptureThisFrame` on the returned `SensorHandle` each frame to determine whether it is time for the sensor to perform a capture. `SensorHandle.ReportCapture` should then be called in each of these frames to report the state of the sensor to populate the dataset.
## Custom annotations and metrics
In addition to the common annotations and metrics produced by [PerceptionCamera](PerceptionCamera.md), scripts can produce their own via `DatasetCapture`. You must first register annotation and metric definitions using `DatasetCapture.RegisterAnnotationDefinition()` or `DatasetCapture.RegisterMetricDefinition()`. These return `AnnotationDefinition` and `MetricDefinition` instances which you can then use to report values during runtime.
1. Create a new scene using **File** > **New Scene**
2. Ctrl+S to save the scene and give it a name
2. Ctrl+S to save the Scene and give it a name
1. In the inspector panel of the Main Camera select **Add Component**
1. In the Main Camera's Inspector window select **Add Component**
2. Add a Perception Camera component
<br/>_Position of the cubes in front of the Main Camera_
5. On each cube, in the Inspector panel add a Labeling component
5. On each cube, in the Inspector window, add a Labeling component
1. Select **Add (+)**
2. In the text field add the name of the object, for example Crate. Unity uses this label in the semantic segmentation images.
6. Create and set up an IdLabelConfig
1. In the Project panel select **Add (+)**, then **Perception** > **ID Label Config**
1. In the Project window select **Add (+)**, then **Perception** > **ID Label Config**
4. In the three label text fields, add the text from the Labeling script on the objects you created in the Scene (that is Crate, Cube and Box)
4. In the three label text fields, add the text (Crate, Cube and Box) from the Labeling script on the objects you created in the Scene
<br/>_IdLabelConfig with three labels_
1. In the Project panel select **Add (+)**, then **Perception** > **Semantic Segmentation Label Config**
2. In the Assets folder, select the new SemanticSegmentationLabelingConfiguration
3. In the Inspector, select **Add to list (+)** three times
4. In the three label text fields, add the text from the Labeling script on the objects you created in the Scene (that is Crate, Cube and Box)
4. In the three label text fields, add the text (Cube, Crate and Box) from the Labeling script on the objects you created in the Scene
8. In the Hierarchy panel select the Main Camera
8. In the Hierarchy window select the Main Camera
9. Add the IdLabelConfig to the Perception Camera script
10. Add the SemanticSegmentationLabelingConfiguration to the Perception Camera script
1. In the Perception Camera script, find the SemanticSegmentationLabeler Camera Labeler. In its Label Config field, click the circle button.
1. In the Perception Camera script, find the SemanticSegmentationLabeler Camera Labeler. In its Label Config field, select the circle button.
2. In the Select SemanticSegmentationLabelConfig window, select the **SemanticSegmentationLabelConfig** you created.
## Step 5: Run simulation and generate dataset
1. In the Editor, press the play button, allow the scene to run for a few seconds, then exit Play mode.
1. In the Editor, press the play button, allow the scene to run for 10 seconds, then exit Play mode.
2. In the console log you see a Shutdown in Progress message that shows a file path to the location of the generated dataset.
> Example file path on a Windows PC : `C:/Users/<User Name>/AppData/LocalLow/DefaultCompany/UnityTestFramework\2e10ec21-9d97-4cee-b5a2-7e95e299afa4\RGB18f61842-ef8d-4b31-acb5-cb1da36fb7b1`
<br/>_Example of Perception running with show visualizations enabled_
1. To use the visualizer, select the Main Camera, and in the Inspector pane, in the Perception Camera component, enable **Show Visualizations**. This enables the built-in labelers which includes segmentation data, 2D bounding boxes, pixel and object counts.
1. To use the visualizer, select the Main Camera, and in the Inspector window, in the Perception Camera component, enable **Show Visualizations**. This enables the built-in labelers which includes segmentation data, 2D bounding boxes, pixel and object counts.
2. Enabling the visualizer creates new UI controls in the Editor's Game view. These controls allow you to control each of the individual visualizers. You can enable or disable individual visualizers. Some visualizers also include controls that let you change their output.
Many labelers require require a Label Config asset. This asset specifies a list of all labels to be captured in the dataset along with extra information used by the various labelers.
## Best practices
Generally algorithm testing and training requires a single label on an asset for proper identification such as “chair”, “table”, or “door". To maximize asset reuse, however, it is useful to give each object multiple labels in a hierarchy.
Generally algorithm testing and training requires a single label on an asset for proper identification such as "chair", "table" or "door". To maximize asset reuse, however, it is useful to give each object multiple labels in a hierarchy.
* “food”: type
* “cereal”: subtype
* “kellogs”: main descriptor
* “ricekrispies”: sub descriptor
* "food": type
* "cereal": subtype
* "kellogs": main descriptor
* "ricekrispies": sub descriptor
If the goal of the algorithm is to identify all objects in a Scene that are "food", that label is available and can be used. Conversely if the goal is to identify only Rice Krispies cereal within a Scene that label is also available. Depending on the goal of the algorithm, you can use any mix of labels in the hierarchy.
If the goal of the algorithm is to identify all objects in a Scene that are “food”, that label is available and can be used. Conversely if the goal is to identify only Rice Krispies cereal within a Scene that label is also available. Depending on the goal of the algorithm, you can use any mix of labels in the hierarchy.
|[Labeling](GroundTruth-Labeling.md)|Component that marks a GameObject and its descendants with a set of labels|
|[LabelConfig](GroundTruth-Labeling.md#LabelConfig)|Asset that defines a taxonomy of labels for ground truth generation|
|[Perception Camera](PerceptionCamera.md)|Captures RGB images and ground truth from a [Camera](https://docs.unity3d.com/Manual/class-Camera.html)|
|[DatasetCapture](DatasetCapture.md)|Ensures sensors are triggered at proper rates and accepts data for the JSON dataset|
|[Randomization (Experimental)](Randomization/Index.md)|Integrate domain randomization principles into your simulation|
|[Labeling](GroundTruth-Labeling.md)|A component that marks a GameObject and its descendants with a set of labels|
|[LabelConfig](GroundTruth-Labeling.md#LabelConfig)|An asset that defines a taxonomy of labels for ground truth generation|
|[Perception Camera](PerceptionCamera.md)|Captures RGB images and ground truth from a [Camera](https://docs.unity3d.com/Manual/class-Camera.html).|
|[DatasetCapture](DatasetCapture.md)|Ensures sensors are triggered at proper rates and accepts data for the JSON dataset.|
|[Randomization (Experimental)](Randomization/Index.md)|The Randomization tool set lets you integrate domain randomization principles into your simulation.|
christopher.kerr
4 年前