浏览代码

Merge remote-tracking branch 'origin/aisv647_visualizations' into aisv647_visualizations

/aisv647_visualizations
Steven Borkman 4 年前
当前提交
aaa3011c
共有 32 个文件被更改,包括 2247 次插入350 次删除
  1. 21
      .yamato/publish.yml
  2. 4
      .yamato/promote.yml
  3. 4
      TestProjects/PerceptionHDRP/Assets/IdLabelConfig.asset
  4. 11
      TestProjects/PerceptionHDRP/Assets/Scenes/SampleScene.unity
  5. 4
      TestProjects/PerceptionURP/Assets/ExampleScripts/CustomAnnotationAndMetricReporter.cs
  6. 1
      com.unity.perception/CHANGELOG.md
  7. 13
      com.unity.perception/Documentation~/index.md
  8. 31
      com.unity.perception/Documentation~/GettingStarted.md
  9. 204
      com.unity.perception/Documentation~/images/PerceptionCameraFinished.png
  10. 366
      com.unity.perception/Documentation~/images/MainCameraLabelConfig.PNG
  11. 125
      com.unity.perception/Documentation~/images/IDLabelingConfigurationFinished.PNG
  12. 4
      com.unity.perception/Documentation~/DatasetCapture.md
  13. 3
      com.unity.perception/Runtime/GroundTruth/Labelers/SemanticSegmentationLabeler.cs
  14. 3
      com.unity.perception/Runtime/GroundTruth/PerceptionCamera.cs
  15. 6
      com.unity.perception/Runtime/GroundTruth/SimulationState_Json.cs
  16. 2
      com.unity.perception/Tests/Editor/DatasetCaptureEditorTests.cs
  17. 5
      com.unity.perception/Tests/Runtime/GroundTruthTests/SegmentationGroundTruthTests.cs
  18. 2
      com.unity.perception/package.json
  19. 1
      TestProjects/PerceptionHDRP/Assets/csc.rsp
  20. 7
      TestProjects/PerceptionHDRP/Assets/csc.rsp.meta
  21. 1
      TestProjects/PerceptionURP/Assets/csc.rsp
  22. 7
      TestProjects/PerceptionURP/Assets/csc.rsp.meta
  23. 759
      com.unity.perception/Documentation~/images/smartcamera.png
  24. 1001
      com.unity.perception/Documentation~/images/synthdet.png
  25. 5
      com.unity.perception/Documentation~/TableOfContents.md
  26. 7
      com.unity.perception/Documentation~/filter.yml
  27. 0
      /.yamato/promote.yml
  28. 0
      /com.unity.perception/Documentation~/images/PerceptionCameraFinished.png
  29. 0
      /com.unity.perception/Documentation~/images/MainCameraLabelConfig.PNG
  30. 0
      /com.unity.perception/Documentation~/images/IDLabelingConfigurationFinished.PNG
  31. 0
      /com.unity.perception/Tests/Editor/DatasetCaptureEditorTests.cs
  32. 0
      /com.unity.perception/Tests/Editor/DatasetCaptureEditorTests.cs.meta

21
.yamato/publish.yml


{% metadata_file .yamato/environments.yml %}
---
{% for variant in package_variants %}
name: Publish {{ variant.name }} package to internal registry
name: Publish Perception package to internal registry
agent:
type: Unity::VM
image: package-ci/win10:stable

triggers:
tags:
only:
- "/^(r|R)(c|C)-+(-preview(+)?)?$/"
- /^(r|R)(c|C)-\d+\.\d+\.\d+(-preview(\.\d+)?)?$/
- .yamato/upm-ci-full.yml#pack_{{ variant.name }}
{% for editor in test_editors %}
{% for platform in test_platforms %}
- .yamato/upm-ci-full.yml#pkg_test_{{variant.name}}_{{platform.name}}_{{editor.version}}
- .yamato/upm-ci-full.yml#pack_Perception
{% for editor in publish_editors %}
{% for platform in publish_platforms %}
{% for suite in suites %}
{% for project in projects %}
- .yamato/upm-ci-full.yml#pkg_test_Perception_{{platform.name}}_{{editor.version}}
- .yamato/upm-ci-testprojects.yml#{{project.name}}_windows_{{suite.name}}_{{editor.version}}
{% endfor %}
{% endfor %}
{% endfor %}

4
.yamato/promote.yml


{% for platform in publish_platforms %}
{% for suite in suites %}
{% for project in projects %}
{% for variant in package_variants %}
- .yamato/upm-ci-full.yml#pkg_test_{{variant.name}}_{{platform.name}}_{{editor.version}}
- .yamato/upm-ci-full.yml#pkg_test_Perception_{{platform.name}}_{{editor.version}}
{% endfor %}
{% endfor %}
{% endfor %}
{% endfor %}

4
TestProjects/PerceptionHDRP/Assets/IdLabelConfig.asset


m_Name: IdLabelConfig
m_EditorClassIdentifier: Unity.Perception.Runtime:UnityEngine.Perception.GroundTruth:IdLabelConfig
m_LabelEntries:
- label: Box
- label: Crate
- label: Crate
- label: Box
id: 3
autoAssignIds: 1
startingLabelId: 1

11
TestProjects/PerceptionHDRP/Assets/Scenes/SampleScene.unity


asm: Unity.Perception.Runtime}
data:
enabled: 1
foldout: 0
labelingConfiguration: {fileID: 11400000, guid: be3971a848968144e8d07d9136a5bf49,
idLabelConfig: {fileID: 11400000, guid: 034ffd4e09ffb7348b2fe6634259058a,
type: 2}
00000001:
type: {class: SemanticSegmentationLabeler, ns: UnityEngine.Perception.GroundTruth,

foldout: 0
m_TargetTextureOverride: {fileID: 0}
foldout: 0
m_LabelingConfiguration: {fileID: 11400000, guid: be3971a848968144e8d07d9136a5bf49,
m_LabelConfig: {fileID: 11400000, guid: 034ffd4e09ffb7348b2fe6634259058a,
type: 2}
00000003:
type: {class: RenderedObjectInfoLabeler, ns: UnityEngine.Perception.GroundTruth,

foldout: 0
labelingConfiguration: {fileID: 11400000, guid: be3971a848968144e8d07d9136a5bf49,
idLabelConfig: {fileID: 11400000, guid: 034ffd4e09ffb7348b2fe6634259058a,
type: 2}
--- !u!1 &1640252278
GameObject:

4
TestProjects/PerceptionURP/Assets/ExampleScripts/CustomAnnotationAndMetricReporter.cs


[RequireComponent(typeof(PerceptionCamera))]
public class CustomAnnotationAndMetricReporter : MonoBehaviour
{
public GameObject light;
public GameObject targetLight;
public GameObject target;
MetricDefinition lightMetricDefinition;

public void Update()
{
//Report the light's position by manually creating the json array string.
var lightPos = light.transform.position;
var lightPos = targetLight.transform.position;
DatasetCapture.ReportMetric(lightMetricDefinition,
$@"[{{ ""x"": {lightPos.x}, ""y"": {lightPos.y}, ""z"": {lightPos.z} }}]");
//compute the location of the object in the camera's local space

1
com.unity.perception/CHANGELOG.md


### Removed
### Fixed
Fixed compilation warnings with latest com.unity.simulation.core package.
### Security

13
com.unity.perception/Documentation~/index.md


[Setting up your first perception scene](GettingStarted.md)
## Example projects using Perception
### SynthDet
<img src="images/synthdet.png"/>
[SynthDet](https://github.com/Unity-Technologies/SynthDet) is an end-to-end solution for training a 2d object detection model using synthetic data.
### Unity Simulation Smart Camera Example
<img src="images/smartcamera.png"/>
The [Unity Simulation Smart Camera Example](https://github.com/Unity-Technologies/Unity-Simulation-Smart-Camera-Outdoor) illustrates how Perception could be used in a smart city or autonomous vehicle simulation. Datasets can be generated locally or at scale in [Unity Simulation](https://unity.com/products/unity-simulation).
## Package contents
|Feature|Description

31
com.unity.perception/Documentation~/GettingStarted.md


1. Select your project's `ScriptableRenderer` asset and open the inspector window. In most projects it is located at `Assets/Settings/ForwardRenderer.asset`.
2. Click `Add Renderer Feature` and select `Ground Truth Renderer Feature`
<img src="images/ScriptableRendererStep.PNG" align="middle"/>
<img src="images/ScriptableRendererStep.png" align="middle"/>
## Step 3: Create a new scene and camera

1. In the inspector panel of the main camera select Add Component
2. Add a **Perception Camera** component
<img src="images/MainCameraConfig.PNG" align="middle"/>
<img src="images/PerceptionCameraFinished.png" align="middle"/>
## Step 4: Create labeled objects

<img src="images/CompletedScene.PNG" align="middle"/>
1. On each cube, from the inspector panel add a **Labeling** component
1. Click the **+**
2. In the text field add the name of the object i.e Crate. This will be the label used in the semantic segmentation images
<img src="images/LabeledObject.PNG" align="middle"/>
2. In the Project panel right click -> Perception -> Labeling Configuration
3. Select the new **Labeling Configuration**
1. Click the **+**
2. In the label text field add the same text that the Label script contains on the objects created in the scene (i.e Cube, Box, Crate)
3. Add a numerical value to the value field
5. On each cube, from the inspector panel add a **Labeling** component
1. Click the **+**
2. In the text field add the name of the object i.e Crate. This will be the label used in the semantic segmentation images
Make sure the labels all have different values, for this example use increments of 10,000 to ensure they show up as very distinct colors in the segmentation images
<img src="images/LabeledObject.PNG" align="middle"/>
<img src="images/LabelingConfigurationFinished.PNG" align="middle"/>
6. In the Project panel right click -> Perception -> Labeling Configuration
7. Select the new **ID Label Config**
1. Click the **+**
2. In the label text field add the same text that the Label script contains on the objects created in the scene (i.e Cube, Box, Crate)
<img src="images/IDLabelingConfigurationFinished.PNG" align="middle"/>
1. Select the Main Camera in the Hierarchy panel
2. In the Perception Camera attach the Labeling Configuration created in previous step
8. Select the Main Camera in the Hierarchy panel
9. In the Perception Camera attach the ID Label Config created in previous step for each ID Label config
<img src="images/MainCameraConfig.PNG" align="middle"/>
<img src="images/MainCameraLabelConfig.PNG" align="middle"/>
## Step 5: Run simulation and generate dataset

204
com.unity.perception/Documentation~/images/PerceptionCameraFinished.png

之前 之后
宽度: 701  |  高度: 415  |  大小: 45 KiB

366
com.unity.perception/Documentation~/images/MainCameraLabelConfig.PNG

之前 之后
宽度: 701  |  高度: 830  |  大小: 76 KiB

125
com.unity.perception/Documentation~/images/IDLabelingConfigurationFinished.PNG

之前 之后
宽度: 704  |  高度: 280  |  大小: 12 KiB

4
com.unity.perception/Documentation~/DatasetCapture.md


[RequireComponent(typeof(PerceptionCamera))]
public class CustomAnnotationAndMetricReporter : MonoBehaviour
{
public GameObject light;
public GameObject targetLight;
public GameObject target;
MetricDefinition lightMetricDefinition;

public void Update()
{
//Report the light's position by manually creating the json array string.
var lightPos = light.transform.position;
var lightPos = targetLight.transform.position;
DatasetCapture.ReportMetric(lightMetricDefinition,
$@"[{{ ""x"": {lightPos.x}, ""y"": {lightPos.y}, ""z"": {lightPos.z} }}]");
//compute the location of the object in the camera's local space

3
com.unity.perception/Runtime/GroundTruth/Labelers/SemanticSegmentationLabeler.cs


height = targetTexture.height,
path = localPath
};
asyncRequest.Start((r) =>
asyncRequest.Enqueue((r) =>
{
Profiler.BeginSample("Encode");
var pngBytes = ImageConversion.EncodeArrayToPNG(r.data.data.ToArray(), GraphicsFormat.R8G8B8A8_UNorm, (uint)r.data.width, (uint)r.data.height);

r.data.data.Dispose();
return AsyncRequest.Result.Completed;
});
asyncRequest.Execute();
}
/// <inheritdoc/>

3
com.unity.perception/Runtime/GroundTruth/PerceptionCamera.cs


bool m_CapturedLastFrame;
Ego m_EgoMarker;
#pragma warning disable 414
#pragma warning restore 414
static PerceptionCamera s_VisualizedPerceptionCamera;
static GameObject s_VisualizationCamera;

6
com.unity.perception/Runtime/GroundTruth/SimulationState_Json.cs


PendingCaptures = pendingCapturesToWrite,
SimulationState = this
};
req.Start(r =>
req.Enqueue(r =>
req.Execute(AsyncRequest.ExecutionContext.JobSystem);
}
m_SerializeCapturesSampler.End();

MetricFileIndex = m_MetricsFileIndex,
PendingMetrics = pendingMetricsToWrite
};
req.Start(r =>
req.Enqueue(r =>
req.Execute();
}
m_MetricsFileIndex++;

2
com.unity.perception/Tests/Editor/DatasetCaptureEditorTests.cs


using NUnit.Framework;
using UnityEngine.Perception.GroundTruth;
namespace GroundTruth
namespace GroundTruthTests
{
public class DatasetCaptureEditorTests
{

5
com.unity.perception/Tests/Runtime/GroundTruthTests/SegmentationGroundTruthTests.cs


using NUnit.Framework;
using Unity.Collections;
using UnityEngine;
using UnityEngine.Experimental.Rendering;
using UnityEngine.Rendering.HighDefinition;
using UnityEngine.Experimental.Rendering;
using UnityEngine.UIElements;
using Object = UnityEngine.Object;
namespace GroundTruthTests

[UnityTest]
public IEnumerator SemanticSegmentationPass_WithTextureOverride_RendersToOverride()
{
int timesSegmentationImageReceived = 0;
var expectedPixelValue = new Color32(0, 0, 255, 255);
var targetTextureOverride = new RenderTexture(2, 2, 1, RenderTextureFormat.R8);

2
com.unity.perception/package.json


"displayName": "Perception",
"name": "com.unity.perception",
"unity": "2019.3",
"version": "0.1.0-preview.4"
"version": "0.3.0-preview.1"
}

1
TestProjects/PerceptionHDRP/Assets/csc.rsp


-warnaserror+

7
TestProjects/PerceptionHDRP/Assets/csc.rsp.meta


fileFormatVersion: 2
guid: b590f0b421136b44098ad7e92fc45f3e
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

1
TestProjects/PerceptionURP/Assets/csc.rsp


-warnaserror+

7
TestProjects/PerceptionURP/Assets/csc.rsp.meta


fileFormatVersion: 2
guid: 1800f44b598c0b647bf3fad2f5c9b957
DefaultImporter:
externalObjects: {}
userData:
assetBundleName:
assetBundleVariant:

759
com.unity.perception/Documentation~/images/smartcamera.png

之前 之后
宽度: 701  |  高度: 209  |  大小: 201 KiB

1001
com.unity.perception/Documentation~/images/synthdet.png
文件差异内容过多而无法显示
查看文件

5
com.unity.perception/Documentation~/TableOfContents.md


* [Installation Instructions](SetupSteps.md)
* [Getting Started](GettingStarted.md)
* [Labeling](GroundTruth-Labeling.md)
* [Perception Camera](PerceptionCamera.md)
* [Dataset Capture](DatasetCapture.md)

7
com.unity.perception/Documentation~/filter.yml


apiRules:
- exclude:
uidRegex: Tests$
type: Namespace
- exclude:
uidRegex: Tests
type: Type

/.yamato/promotion.yml → /.yamato/promote.yml

/com.unity.perception/Documentation~/images/PerceptionCamera.png → /com.unity.perception/Documentation~/images/PerceptionCameraFinished.png

/com.unity.perception/Documentation~/images/MainCameraConfig.PNG → /com.unity.perception/Documentation~/images/MainCameraLabelConfig.PNG

/com.unity.perception/Documentation~/images/LabelingConfigurationFinished.PNG → /com.unity.perception/Documentation~/images/IDLabelingConfigurationFinished.PNG

/com.unity.perception/Tests/Editor/SimulationManagerEditorTests.cs → /com.unity.perception/Tests/Editor/DatasetCaptureEditorTests.cs

/com.unity.perception/Tests/Editor/SimulationManagerEditorTests.cs.meta → /com.unity.perception/Tests/Editor/DatasetCaptureEditorTests.cs.meta

正在加载...
取消
保存