浏览代码

some progress

/main
Mohsen Kamalzadeh 4 年前
当前提交
cca8e251
共有 4 个文件被更改,包括 595 次插入6 次删除
  1. 176
      com.unity.perception/Documentation~/HPETutorial/TUTORIAL.md
  2. 241
      com.unity.perception/Documentation~/HPETutorial/Images/add_label_from_config.png
  3. 184
      com.unity.perception/Documentation~/HPETutorial/Images/keypoint_labeler.png

176
com.unity.perception/Documentation~/HPETutorial/TUTORIAL.md


Through-out the tutorial, lines starting with bullet points followed by **":green_circle: Action:"** denote the individual actions you will need to perform in order to progress through the tutorial. This is while non-bulleted lines will provide additional context and explanation around the actions. If in a hurry, you can just follow the actions!
Steps included in this tutorial:
- [<a name="step-1">Step 2: Set Up a Humanoid Character in a Scene</a>](#step-2-set-up-a-humanoid-character-in-a-scene)
- [<a name="step-2">Step 2: Set Up a Humanoid Character in a Scene</a>](#step-2-set-up-a-humanoid-character-in-a-scene)
-
- [Step 3: Set Up the Perception Camera for Key Point Annotation](#step-3)
- [Step 4: Configure Human Pose Estimation](#step-4)
- [Step 5: Randomize the Humanoid Character's Animations](#step-5)
### <a name="step-1">Step 1: Import `.fbx` Models and Animations</a>

<img src="Images/scenario_empty.png" width="400"/>
</p>
* **:green_circle: Action**: Select `Main Camera` and in the _**Inspector**_ view of the `Perception Camera` component, disable all previously added labelers using the check-mark in front of each.
* **:green_circle: Action**: Select `Main Camera` and in the _**Inspector**_ view of the `Perception Camera` component, **disable** all previously added labelers using the check-mark in front of each. We will be using a new labeler in this tutorial.
We now need to import the sample files required for this tutorial.
* **:green_circle: Action**: Open _**Package Manager**_ and select the Perception package, which should already be present in the navigation pane to the left side.
* **:green_circle: Action**: From the list of ***Samples*** for the Perception package, click on the ***Import into Project*** button for the sample bundle named _**Human Pose Estimation**_.

This will make sure all animations included in the sample `.fbx` files are ready to be used on a rigged humanoid model.
### <a name="step-1">Step 2: Set Up a Humanoid Character in a Scene</a>
### <a name="step-2">Step 2: Set Up a Humanoid Character in a Scene</a>
* **:green_circle: Action**: Drag and drop the file named `Player` into your Scene _**Hierarchy**_.
* **:green_circle: Action**: Select the new `Player` object in the Scene and in the _**Inspector**_ tab set its transform's position and rotation according to the image below to make the character face the camera.

<img src="Images/assign_controller.png" width="400"/>
</p>
s
**[Continue to Phase 1: Setup and Basic Randomizations](Phase1.md)**
If you run the simulation now you will see the character going through a series of three animations for picking up a hypothetical object as seen in the GIF below. This is one complete run of the `TakeObjects` animation clip we selected earlier.
<p align="center">
<img src="Images/take_objects.gif" width="600"/>
</p>
### <a name="step-3">Step 3: Set Up the Perception Camera for Key Point Annotation</a>
Now that we have our character performing animations, let's modify our `Perception Camera` to report the character's key points in the output dataset, updating frame by frame as they animate.
* **:green_circle: Action**: Add a `KeyPointLabeler` to the list of labelers in `Perception Camera`.
Similar to the labelers we used in the Perception Tutorial, we will need a label configuration for this new labeler.
* **:green_circle: Action**: In the _**Project**_ tab, right-click the `Assets` folder, then click _**Create -> Perception -> Id Label Config**_. Name the new asset `HPE_IdLabelConfig`.
* **:green_circle: Action**: Add the label `MyCharacter` to the newly created config.
> :information_source: You can use any label string, as long as you assign the same label to the `Player` object in the next step.
* **:green_circle: Action**: Add a `Labeling` component to the `Player` object in the Scene.
* **:green_circle: Action**: In the _**Inspector**_ UI for this new `Labeling` component, expand `HPE_IdLabelConfig` and click _**Add to Labels** on `MyCharacter`.
<p align="center">
<img src="Images/add_label_from_config.png" width="400"/>
</p>
* **:green_circle: Action**: Return to `Perception Camera` and assign `HPE_IdLabelConfig` to the `KeyPointLabeler`'s label configuration property.
The labelers should now look like the image below:
<p align="center">
<img src="Images/keypoint_labeler.png" width="500"/>
</p>
Note the `CocoKeyPointTemplate` asset that is already assigned as the `Active Template`. This template will tell the labeler how to map default Unity rig joints to human joint labels in the popular COCO dataset, so that the output of the labeler can be easily converted to COCO format. Later in this tutorial, we will learn how to add more joints to our character and how to customize joint mapping templates.
<p align="center">
<img src="Images/take_objects_keypoints.gif" width="600"/>
</p>
You can now check out the output dataset to see what the annotations look like. To do this, click the _**Show Folder**_ button in the `Perception Camera` UI, then navigate inside to the dataset folder to find the `captures_000.json` file. Here is an example annotation for the first frame of our test-case here:
```json
"pose": "unset",
"keypoints": [
{
"index": 0,
"x": 0.0,
"y": 0.0,
"state": 0
},
{
"index": 1,
"x": 649.05615234375,
"y": 300.65264892578125,
"state": 2
},
{
"index": 2,
"x": 594.4522705078125,
"y": 335.8978271484375,
"state": 2
},
{
"index": 3,
"x": 492.46444702148438,
"y": 335.72491455078125,
"state": 2
},
{
"index": 4,
"x": 404.89456176757813,
"y": 335.57647705078125,
"state": 2
},
{
"index": 5,
"x": 705.89404296875,
"y": 335.897705078125,
"state": 2
},
{
"index": 6,
"x": 807.74688720703125,
"y": 335.7244873046875,
"state": 2
},
{
"index": 7,
"x": 895.1993408203125,
"y": 335.57574462890625,
"state": 2
},
{
"index": 8,
"x": 612.51654052734375,
"y": 509.065185546875,
"state": 2
},
{
"index": 9,
"x": 608.50006103515625,
"y": 647.0631103515625,
"state": 2
},
{
"index": 10,
"x": 611.7791748046875,
"y": 797.7828369140625,
"state": 2
},
{
"index": 11,
"x": 682.175048828125,
"y": 509.06524658203125,
"state": 2
},
{
"index": 12,
"x": 683.1016845703125,
"y": 649.64434814453125,
"state": 2
},
{
"index": 13,
"x": 686.3271484375,
"y": 804.203857421875,
"state": 2
},
{
"index": 14,
"x": 628.012939453125,
"y": 237.50531005859375,
"state": 2
},
{
"index": 15,
"x": 660.023193359375,
"y": 237.50543212890625,
"state": 2
},
{
"index": 16,
"x": 0.0,
"y": 0.0,
"state": 0
},
{
"index": 17,
"x": 0.0,
"y": 0.0,
"state": 0
}
]
}
```

241
com.unity.perception/Documentation~/HPETutorial/Images/add_label_from_config.png

之前 之后
宽度: 912  |  高度: 888  |  大小: 75 KiB

184
com.unity.perception/Documentation~/HPETutorial/Images/keypoint_labeler.png

之前 之后
宽度: 1078  |  高度: 192  |  大小: 54 KiB

部分文件因为文件数量过多而无法显示

正在加载...
取消
保存