You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Copy file name to clipboardExpand all lines: com.unity.perception/Documentation~/HPETutorial/TUTORIAL.md
+170-6Lines changed: 170 additions & 6 deletions
Display the source diff
Display the rich diff
Original file line number
Diff line number
Diff line change
@@ -7,9 +7,17 @@ We strongly recommend you finish [Phase 1 of the Perception Tutorial](../Tutoria
7
7
Through-out the tutorial, lines starting with bullet points followed by **":green_circle: Action:"** denote the individual actions you will need to perform in order to progress through the tutorial. This is while non-bulleted lines will provide additional context and explanation around the actions. If in a hurry, you can just follow the actions!
-[<aname="step-1">Step 1: Import `.fbx` Models and Animations</a>](#step-1-import-fbx-models-and-animations)
12
-
-[<aname="step-1">Step 2: Set Up a Humanoid Character in a Scene</a>](#step-2-set-up-a-humanoid-character-in-a-scene)
13
+
-[<aname="step-2">Step 2: Set Up a Humanoid Character in a Scene</a>](#step-2-set-up-a-humanoid-character-in-a-scene)
14
+
15
+
16
+
17
+
-
18
+
-[Step 3: Set Up the Perception Camera for Key Point Annotation](#step-3)
19
+
-[Step 4: Configure Human Pose Estimation](#step-4)
20
+
-[Step 5: Randomize the Humanoid Character's Animations](#step-5)
13
21
14
22
### <aname="step-1">Step 1: Import `.fbx` Models and Animations</a>
15
23
@@ -28,9 +36,9 @@ Your Scenario should now look like this:
28
36
<imgsrc="Images/scenario_empty.png"width="400"/>
29
37
</p>
30
38
31
-
***:green_circle: Action**: Select `Main Camera` and in the _**Inspector**_ view of the `Perception Camera` component, disable all previously added labelers using the check-mark in front of each.
32
-
39
+
***:green_circle: Action**: Select `Main Camera` and in the _**Inspector**_ view of the `Perception Camera` component, **disable** all previously added labelers using the check-mark in front of each. We will be using a new labeler in this tutorial.
33
40
41
+
We now need to import the sample files required for this tutorial.
34
42
35
43
***:green_circle: Action**: Open _**Package Manager**_ and select the Perception package, which should already be present in the navigation pane to the left side.
36
44
***:green_circle: Action**: From the list of ***Samples*** for the Perception package, click on the ***Import into Project*** button for the sample bundle named _**Human Pose Estimation**_.
@@ -46,7 +54,7 @@ Once the sample files are imported, they will be placed inside the `Assets/Sampl
46
54
47
55
This will make sure all animations included in the sample `.fbx` files are ready to be used on a rigged humanoid model.
48
56
49
-
### <aname="step-1">Step 2: Set Up a Humanoid Character in a Scene</a>
57
+
### <aname="step-2">Step 2: Set Up a Humanoid Character in a Scene</a>
50
58
51
59
***:green_circle: Action**: Drag and drop the file named `Player` into your Scene _**Hierarchy**_.
52
60
***:green_circle: Action**: Select the new `Player` object in the Scene and in the _**Inspector**_ tab set its transform's position and rotation according to the image below to make the character face the camera.
@@ -83,5 +91,161 @@ In the selector window that pops up, you will see several clips named `Take 001`
**[Continue to Phase 1: Setup and Basic Randomizations](Phase1.md)**
94
+
If you run the simulation now you will see the character going through a series of three animations for picking up a hypothetical object as seen in the GIF below. This is one complete run of the `TakeObjects` animation clip we selected earlier.
95
+
96
+
<palign="center">
97
+
<imgsrc="Images/take_objects.gif"width="600"/>
98
+
</p>
99
+
100
+
101
+
### <aname="step-3">Step 3: Set Up the Perception Camera for Key Point Annotation</a>
102
+
103
+
Now that we have our character performing animations, let's modify our `Perception Camera` to report the character's key points in the output dataset, updating frame by frame as they animate.
104
+
105
+
***:green_circle: Action**: Add a `KeyPointLabeler` to the list of labelers in `Perception Camera`.
106
+
107
+
Similar to the labelers we used in the Perception Tutorial, we will need a label configuration for this new labeler.
108
+
109
+
***:green_circle: Action**: In the _**Project**_ tab, right-click the `Assets` folder, then click _**Create -> Perception -> Id Label Config**_. Name the new asset `HPE_IdLabelConfig`.
110
+
***:green_circle: Action**: Add the label `MyCharacter` to the newly created config.
111
+
112
+
> :information_source: You can use any label string, as long as you assign the same label to the `Player` object in the next step.
113
+
114
+
***:green_circle: Action**: Add a `Labeling` component to the `Player` object in the Scene.
115
+
***:green_circle: Action**: In the _**Inspector**_ UI for this new `Labeling` component, expand `HPE_IdLabelConfig` and click _**Add to Labels** on `MyCharacter`.
Note the `CocoKeyPointTemplate` asset that is already assigned as the `Active Template`. This template will tell the labeler how to map default Unity rig joints to human joint labels in the popular COCO dataset, so that the output of the labeler can be easily converted to COCO format. Later in this tutorial, we will learn how to add more joints to our character and how to customize joint mapping templates.
You can now check out the output dataset to see what the annotations look like. To do this, click the _**Show Folder**_ button in the `Perception Camera` UI, then navigate inside to the dataset folder to find the `captures_000.json` file. Here is an example annotation for the first frame of our test-case here:
0 commit comments