0.8.0 changes -> master (#273)

* Fix HDRP warnings on 2020.X (#260)

* Fix HDRP warnings on 2020.2

* Updating changelog

* Fixing HDRP version expression on 2019.4

* Fix keypoint issues (#259)

* Fixing two bugs in KeypointLabeler:
AISV-1254 - Some keypoints fall outside of the image bounds
AISV-1257 - Partially visible characters with no visible keypoints are not reported

* Adding JointLabelEditor  with message about Labeling requirement

* Reorganizing keypoint docs and elaborating on setup requirements.

* Updating PerceptionCamera.md

* Fixing issues on 2020.2 and failing test

* Add object filtering modes to keypoint labeler

* Updating docs for object filtering

* Adding link to tutorial

* Moving KeypointLabeler.md to proper directory

* Update TUTORIAL.md

* Updating capitalization

Co-authored-by: Mohsen K <mohsen.kamalzadeh@unity3d.com>

* Apply doc suggestions

Co-authored-by: Mohsen K <mohsen.kamalzadeh@unity3d.com>

* Update P...

TestProjects/PerceptionHDRP/Packages/packages-lock.json

         "com.unity.collections": "0.9.0-preview.6",
         "com.unity.nuget.newtonsoft-json": "1.1.2",
         "com.unity.render-pipelines.core": "7.1.6",
-        "com.unity.simulation.capture": "0.0.10-preview.19",
+        "com.unity.simulation.capture": "0.0.10-preview.20",
         "com.unity.simulation.client": "0.0.10-preview.10",
         "com.unity.simulation.core": "0.0.10-preview.22"
       }
       "url": "https://packages.unity.com"
     },
     "com.unity.simulation.capture": {
-      "version": "0.0.10-preview.19",
+      "version": "0.0.10-preview.20",
       "depth": 1,
       "source": "registry",
       "dependencies": {

com.unity.perception/CHANGELOG.md

 
 ### Fixed
 
-A bug in the Normal Sampler where it would return values less than the passed in min, or greater than the passed in max, for random values very close to 0 or 1 respectively.
+## [0.8.0-preview.3] - 2021-03-24
+### Changed
+
+Expanded documentation on the Keypoint Labeler
+Updated Keypoint Labeler logic to only report keypoints for visible objects by default
+Increased color variety in instance segmentation images
+
+### Fixed
+
+Fixed compiler warnings in projects with HDRP on 2020.1 and later
+
+Fixed a bug in the Normal Sampler where it would return values less than the passed in minimum value, or greater than the passed in maximum value, for random values very close to 0 or 1 respectively.
+
 
 ## [0.8.0-preview.2] - 2021-03-15
 

com.unity.perception/Documentation~/HPTutorial/TUTORIAL.md

 }
 ```
 
-In the above annotation, all of the 18 joints defined in the COCO template we used are listed. For each joint that is present in our character, you can see the X and Y coordinates within the captured frame. However, you may notice three of the joints are listed with (0,0) coordinates. These joints are not present in our character. A fact that is also denoted by the `state` field. A state of **0** means the joint was not present, **1** denotes a joint that is present but not visible (to be implemented in a later version of the package), and **2** means the joint was present and visible.
+In the above annotation, all of the 18 joints defined in the COCO template we used are listed. For each joint that is present in our character, you can see the X and Y coordinates within the captured frame. However, you may notice three of the joints are listed with (0,0) coordinates. These joints are not present in our character. A fact that is also denoted by the `state` field. A state of **0** means the joint either does not exist or is outside of the image's bounds, **1** denotes a joint that is inside of the image but cannot be seen because the part of the object it belongs to is not visible in the image, and **2** means the joint was present and visible.
 
 You may also note that the `pose` field has a value of `unset`. This is because we have not defined poses for our animation clip and `Perception Camera` yet. We will do this next.
 

com.unity.perception/Documentation~/PerceptionCamera.md

 ```
 _Example rendered object info for a single object_
 
-The RenderedObjectInfoLabeler records a list of all objects visible in the Camera image, including its instance ID, resolved label ID and visible pixels. If Unity cannot resolve objects to a label in the IdLabelConfig, it does not record these objects.
+The `RenderedObjectInfoLabeler` records a list of all objects visible in the camera image, including their instance IDs, resolved label IDs, and visible pixel counts. If Unity cannot resolve objects to a label in the `IdLabelConfig`, it does not record these objects.
-The keypoint labeler captures keypoints of a labeled gameobject. The typical use of this labeler is capturing human pose estimation data. The labeler uses a [keypoint template](#KeypointTemplate) which defines the keypoints to capture for the model and the skeletal connections between those keypoints. The positions of the keypoints are recorded in pixel coordinates. Each keypoint has a state value: 0 - the keypoint either does not exist or is outside of the image's bounds, 1 - the keypoint exists and is inside of the image's bounds but cannot be seen because it is occluded by another object, and 2 - the keypoint exists and is visible.
-
-```
-keypoints {
-  label_id:      <int>   -- Integer identifier of the label
-  instance_id:   <str>   -- UUID of the instance.
-  template_guid: <str>   -- UUID of the keypoint template
-  pose:          <str>   -- Pose ground truth information
-  keypoints [            -- Array of keypoint data, one entry for each keypoint defined in associated template file.
-    {
-      index:     <int>   -- Index of keypoint in template
-      x:         <float> -- X pixel coordinate of keypoint
-      y:         <float> -- Y pixel coordinate of keypoint
-      state:     <int>   -- 0: keypoint does not exist, 1: keypoint exists but is not visible, 2: keypoint exists and is visible
-    }, ...
-  ]
-}
-```
-
-#### Keypoint Template
-
-keypoint templates are used to define the keypoints and skeletal connections captured by the KeypointLabeler. The keypoint
-template takes advantage of Unity's humanoid animation rig, and allows the user to automatically associate template keypoints
-to animation rig joints. Additionally, the user can choose to ignore the rigged points, or add points not defined in the rig.
-A Coco keypoint template is included in the perception package.
-
-##### Editor
-
-The keypoint template editor allows the user to create/modify a keypoint template. The editor consists of the header information,
-the keypoint array, and the skeleton array.
-
-![Header section of the keypoint template](images/keypoint_template_header.png)
-<br/>_Header section of the keypoint template_
-
-In the header section, a user can change the name of the template and supply textures that they would like to use for the keypoint
-visualization.
-
-![The keypoint section of the keypoint template](images/keypoint_template_keypoints.png)
-<br/>_Keypoint section of the keypoint template_
-
-The keypoint section allows the user to create/edit keypoints and associate them with Unity animation rig points. Each keypoint record
-has 4 fields: label (the name of the keypoint), Associate to Rig (a boolean value which, if true, automatically maps the keypoint to
-the gameobject defined by the rig), Rig Label (only needed if Associate To Rig is true, defines which rig component to associate with
-the keypoint), and Color (RGB color value of the keypoint in the visualization).
-
-![Skeleton section of the keypoint template](images/keypoint_template_skeleton.png)
-<br/>_Skeleton section of the keypoint template_
-
-The skeleton section allows the user to create connections between joints, basically defining the skeleton of a labeled object.
-
-##### Format
-```
-annotation_definition.spec {
-  template_id:       <str>           -- The UUID of the template
-  template_name:     <str>           -- Human readable name of the template
-  key_points [                       -- Array of joints defined in this template
-    {
-      label:         <str>           -- The label of the joint
-      index:         <int>           -- The index of the joint
-    }, ...
-  ]
-  skeleton [                         -- Array of skeletal connections (which joints have connections between one another) defined in this template
-    {
-      joint1:        <int>           -- The first joint of the connection
-      joint2:        <int>           -- The second joint of the connection
-    }, ...
-  ]
-}
-```
+The keypoint labeler captures the screen locations of specific points on labeled GameObjects. The typical use of this Labeler is capturing human pose estimation data, but it can be used to capture points on any kind of object. The Labeler uses a [Keypoint Template](#KeypointTemplate) which defines the keypoints to capture for the model and the skeletal connections between those keypoints. The positions of the keypoints are recorded in pixel coordinates.
-#### Animation Pose Label
-
-This file is used to define timestamps in an animation to a pose label.
+For more information, see [Keypoint Labeler](GroundTruth/KeypointLabeler.md) or the [Human Pose Labeling and Randomization Tutorial](HPTutorial/TUTORIAL.md)
 
 ## Limitations
 

com.unity.perception/Runtime/GroundTruth/Labelers/BoundingBox3DLabeler.cs

                     combinedBounds.center = labelTransform.TransformPoint(combinedBounds.center);
                     combinedBounds.extents = Vector3.Scale(combinedBounds.extents,  labelTransform.lossyScale);
 
-                    // Now convert all points into camera's space
-                    var cameraCenter = cameraTransform.InverseTransformPoint(combinedBounds.center);
-                    cameraCenter = Vector3.Scale(cameraTransform.localScale, cameraCenter);
+                    // Now adjust the center and rotation to camera space. Camera space transforms never rescale objects
+                    combinedBounds.center = combinedBounds.center - cameraTransform.position;
+                    combinedBounds.center = Quaternion.Inverse(cameraTransform.rotation) * combinedBounds.center;
-                    // Rotation to go from label space to camera space
-                    var converted = ConvertToBoxData(labelEntry, labeledEntity.instanceId, cameraCenter, combinedBounds.extents, cameraRotation);
+                    var converted = ConvertToBoxData(labelEntry, labeledEntity.instanceId, combinedBounds.center, combinedBounds.extents, cameraRotation);
 
                     m_BoundingBoxValues[m_CurrentFrame][labeledEntity.instanceId] = converted;
                 }
         static Vector3 CalculateRotatedPoint(Camera cam, Vector3 start, Vector3 xDirection, Vector3 yDirection, Vector3 zDirection, float xScalar, float yScalar, float zScalar)
         {
             var rotatedPoint = start + xDirection * xScalar + yDirection * yScalar + zDirection * zScalar;
-            var worldPoint = cam.transform.TransformPoint(rotatedPoint);
+            var worldPoint = cam.transform.position + cam.transform.rotation * rotatedPoint;
             return VisualizationHelper.ConvertToScreenSpace(cam, worldPoint);
         }
 

com.unity.perception/Runtime/GroundTruth/Labelers/KeypointLabeler.cs

 using System.Collections.Generic;
 using System.Linq;
 using Unity.Collections;
+using Unity.Mathematics;
 using UnityEngine.Rendering;
 
 namespace UnityEngine.Perception.GroundTruth
         /// The <see cref="IdLabelConfig"/> which associates objects with labels.
         /// </summary>
         public IdLabelConfig idLabelConfig;
+
+        /// <summary>
+        /// Controls which objects will have keypoints recorded in the dataset.
+        /// <see cref="KeypointObjectFilter"/>
+        /// </summary>
+        public KeypointObjectFilter objectFilter;
         // ReSharper restore MemberCanBePrivate.Global
 
         AnnotationDefinition m_AnnotationDefinition;
-        List<KeypointEntry> m_ToReport;
+        List<KeypointEntry> m_KeypointEntriesToReport;
 
         int m_CurrentFrame;
 
         /// </summary>
         public List<AnimationPoseConfig> animationPoseConfigs;
 
+
+
         /// <inheritdoc/>
         protected override void Setup()
         {
             m_KnownStatus = new Dictionary<uint, CachedData>();
 
             m_AsyncAnnotations = new Dictionary<int, (AsyncAnnotation, Dictionary<uint, KeypointEntry>)>();
-            m_ToReport = new List<KeypointEntry>();
+            m_KeypointEntriesToReport = new List<KeypointEntry>();
+            perceptionCamera.RenderedObjectInfosCalculated += OnRenderedObjectInfoReadback;
         }
 
         bool AreEqual(Color32 lhs, Color32 rhs)
 
         bool PixelsMatch(int x, int y, Color32 idColor, (int x, int y) dimensions, NativeArray<Color32> data)
         {
-            var h = dimensions.y - y;
+            var h = dimensions.y - 1 - y;
             var pixelColor = data[h * dimensions.x + x];
             return AreEqual(pixelColor, idColor);
         }
         {
             if (keypoint.state == 0) return 0;
 
-            var centerX = Mathf.RoundToInt(keypoint.x);
-            var centerY = Mathf.RoundToInt(keypoint.y);
+            var centerX = Mathf.FloorToInt(keypoint.x);
+            var centerY = Mathf.FloorToInt(keypoint.y);
-            var pixelOnScreen = false;
+            if (!PixelOnScreen(centerX, centerY, dimensions))
+                return 0;
+
+            var pixelMatched = false;
 
             for (var y = centerY - s_PixelTolerance; y <= centerY + s_PixelTolerance; y++)
             {
 
-                    pixelOnScreen = true;
+                    pixelMatched = true;
                     if (PixelsMatch(x, y, instanceIdColor, dimensions, data))
                     {
                         return 2;
 
-            return pixelOnScreen ? 1 : 0;
+            return pixelMatched ? 1 : 0;
         }
 
         void OnInstanceSegmentationImageReadback(int frameCount, NativeArray<Color32> data, RenderTexture renderTexture)
-
-            m_AsyncAnnotations.Remove(frameCount);
-            m_ToReport.Clear();
-
-                    var shouldReport = false;
-
                     foreach (var keypoint in keypointSet.Value.keypoints)
                     {
                         keypoint.state = DetermineKeypointState(keypoint, idColor, dimensions, data);
                         }
                         else
                         {
-                            shouldReport = true;
+                            keypoint.x = math.clamp(keypoint.x, 0, dimensions.width - .001f);
+                            keypoint.y = math.clamp(keypoint.y, 0, dimensions.height - .001f);
+                }
+            }
+        }
+
+        private void OnRenderedObjectInfoReadback(int frameCount, NativeArray<RenderedObjectInfo> objectInfos)
+        {
+            if (!m_AsyncAnnotations.TryGetValue(frameCount, out var asyncAnnotation))
+                return;
-                    if (shouldReport)
-                        m_ToReport.Add(keypointSet.Value);
+            m_AsyncAnnotations.Remove(frameCount);
+
+            m_KeypointEntriesToReport.Clear();
+
+            //filter out objects that are not visible
+            foreach (var keypointSet in asyncAnnotation.keypoints)
+            {
+                var entry = keypointSet.Value;
+
+                var include = false;
+                if (objectFilter == KeypointObjectFilter.All)
+                    include = true;
+                else
+                {
+                    foreach (var objectInfo in objectInfos)
+                    {
+                        if (entry.instance_id == objectInfo.instanceId)
+                        {
+                            include = true;
+                            break;
-            KeypointsComputed?.Invoke(frameCount, m_ToReport);
-            asyncAnnotation.annotation.ReportValues(m_ToReport);
+                    if (!include && objectFilter == KeypointObjectFilter.VisibleAndOccluded)
+                        include = keypointSet.Value.keypoints.Any(k => k.state == 1);
+        }
+                if (include)
+                    m_KeypointEntriesToReport.Add(entry);
+            }
+
+            //This code assumes that OnRenderedObjectInfoReadback will be called immediately after OnInstanceSegmentationImageReadback
+            KeypointsComputed?.Invoke(frameCount, m_KeypointEntriesToReport);
+            asyncAnnotation.annotation.ReportValues(m_KeypointEntriesToReport);
         }
 
         /// <param name="scriptableRenderContext"></param>
             return targetTexture != null ?
                 targetTexture.height : Screen.height;
         }
-
-        // Converts a coordinate from world space into pixel space
+            if (Mathf.Approximately(pt.y, perceptionCamera.attachedCamera.pixelHeight))
+                pt.y -= .0001f;
+            if (Mathf.Approximately(pt.x, perceptionCamera.attachedCamera.pixelWidth))
+                pt.x -= .0001f;
+
             return pt;
         }
 
                         var bone = animator.GetBoneTransform(pt.rigLabel);
                         if (bone != null)
                         {
-                            var loc = ConvertToScreenSpace(bone.position);
-                            keypoints[i].index = i;
-                            keypoints[i].x = loc.x;
-                            keypoints[i].y = loc.y;
-                            keypoints[i].state = 2;
+                                InitKeypoint(bone.position, keypoints, i);
                         }
                     }
                 }
                 foreach (var (joint, idx) in cachedData.overrides)
                 {
-                    var loc = ConvertToScreenSpace(joint.transform.position);
-                    keypoints[idx].index = idx;
-                    keypoints[idx].x = loc.x;
-                    keypoints[idx].y = loc.y;
-                    keypoints[idx].state = 2;
+                        InitKeypoint(joint.transform.position, keypoints, idx);
                 }
 
                 cachedData.keypoints.pose = "unset";
             }
         }
 
+        private void InitKeypoint(Vector3 position, Keypoint[] keypoints, int idx)
+        {
+            var loc = ConvertToScreenSpace(position);
+            keypoints[idx].index = idx;
+            if (loc.z < 0)
+            {
+                keypoints[idx].x = 0;
+                keypoints[idx].y = 0;
+                keypoints[idx].state = 0;
+            }
+            else
+            {
+                keypoints[idx].x = loc.x;
+                keypoints[idx].y = loc.y;
+                keypoints[idx].state = 2;
+            }
+        }
+
         string GetPose(Animator animator)
         {
             var info = animator.GetCurrentAnimatorClipInfo(0);
         /// <inheritdoc/>
         protected override void OnVisualize()
         {
-            if (m_ToReport == null) return;
+            if (m_KeypointEntriesToReport == null) return;
 
             var jointTexture = activeTemplate.jointTexture;
             if (jointTexture == null) jointTexture = m_MissingTexture;
 
-            foreach (var entry in m_ToReport)
+            foreach (var entry in m_KeypointEntriesToReport)
             {
                 foreach (var bone in activeTemplate.skeleton)
                 {

com.unity.perception/Runtime/GroundTruth/RenderPasses/HdrpPasses/GroundTruthPass.cs

             targetDepthBuffer = TargetBuffer.Custom;
         }
 
-        protected sealed override void Execute(
-            ScriptableRenderContext renderContext, CommandBuffer cmd, HDCamera hdCamera, CullingResults cullingResult)
+        //overrides obsolete member in HDRP on 2020.1+. Re-address when removing 2019.4 support or the API is dropped
+#if HDRP_9_OR_NEWER
+        protected override void Execute(CustomPassContext ctx)
+        {
+            ScriptableRenderContext renderContext = ctx.renderContext;
+            var cmd = ctx.cmd;
+            var hdCamera = ctx.hdCamera;
+            var cullingResult = ctx.cullingResults;
+#else
+        protected override void Execute(ScriptableRenderContext renderContext, CommandBuffer cmd, HDCamera hdCamera, CullingResults cullingResult)
+#endif
             // CustomPasses are executed for each camera. We only want to run for the target camera
             if (hdCamera.camera != targetCamera)
                 return;

com.unity.perception/Runtime/GroundTruth/RenderPasses/HdrpPasses/InstanceSegmentationPass.cs

         [UsedImplicitly]
         public InstanceSegmentationPass() {}
 
+        //overrides obsolete member in HDRP on 2020.1+. Re-address when removing 2019.4 support or the API is dropped
+#if HDRP_9_OR_NEWER
+        protected override void Execute(CustomPassContext ctx)
+        {
+            ScriptableRenderContext renderContext = ctx.renderContext;
+            var cmd = ctx.cmd;
+            var hdCamera = ctx.hdCamera;
+            var cullingResult = ctx.cullingResults;
+#else
+#endif
             CoreUtils.SetRenderTarget(cmd, targetTexture, ClearFlag.All);
             m_InstanceSegmentationCrossPipelinePass.Execute(renderContext, cmd, hdCamera.camera, cullingResult);
         }

com.unity.perception/Runtime/GroundTruth/RenderPasses/HdrpPasses/LensDistortionPass.cs

             lensDistortionCrossPipelinePass.Setup();
         }
 
+        //overrides obsolete member in HDRP on 2020.1+. Re-address when removing 2019.4 support or the API is dropped
+#if HDRP_9_OR_NEWER
+        protected override void Execute(CustomPassContext ctx)
+        {
+            ScriptableRenderContext renderContext = ctx.renderContext;
+            var cmd = ctx.cmd;
+            var hdCamera = ctx.hdCamera;
+            var cullingResult = ctx.cullingResults;
+#else
+#endif
             CoreUtils.SetRenderTarget(cmd, targetTexture);
             lensDistortionCrossPipelinePass.Execute(renderContext, cmd, hdCamera.camera, cullingResult);
         }

com.unity.perception/Runtime/GroundTruth/RenderPasses/HdrpPasses/SemanticSegmentationPass.cs

             m_SemanticSegmentationCrossPipelinePass.Setup();
         }
 
+        //overrides obsolete member in HDRP on 2020.1+. Re-address when removing 2019.4 support or the API is dropped
+#if HDRP_9_OR_NEWER
+        protected override void Execute(CustomPassContext ctx)
+        {
+            ScriptableRenderContext renderContext = ctx.renderContext;
+            var cmd = ctx.cmd;
+            var hdCamera = ctx.hdCamera;
+            var cullingResult = ctx.cullingResults;
+#else
+#endif
             CoreUtils.SetRenderTarget(cmd, targetTexture);
             m_SemanticSegmentationCrossPipelinePass.Execute(renderContext, cmd, hdCamera.camera, cullingResult);
         }

com.unity.perception/Runtime/Randomization/Randomizers/RandomizerExamples/Randomizers/AnimationRandomizer.cs

 
         void RandomizeAnimation(AnimationRandomizerTag tag)
         {
+            if (!tag.gameObject.activeInHierarchy)
+                return;
+
             var animator = tag.gameObject.GetComponent<Animator>();
             animator.applyRootMotion = tag.applyRootMotion;
 

com.unity.perception/Runtime/Randomization/Randomizers/RandomizerExamples/Randomizers/BackgroundObjectPlacementRandomizer.cs

         /// <summary>
         /// The Z offset component applied to all generated background layers
         /// </summary>
+        [Tooltip("The Z offset applied to positions of all placed objects.")]
+        [Tooltip("The number of background layers to generate.")]
+        [Tooltip("The minimum distance between the centers of the placed objects.")]
+        [Tooltip("The width and height of the area in which objects will be placed. These should be positive numbers and sufficiently large in relation with the Separation Distance specified.")]
+        [Tooltip("The list of Prefabs to be placed by this Randomizer.")]
         public GameObjectParameter prefabs;
 
         GameObject m_Container;

com.unity.perception/Runtime/Randomization/Randomizers/RandomizerExamples/Randomizers/ColorRandomizer.cs

         static readonly int k_BaseColor = Shader.PropertyToID("_BaseColor");
 
         /// <summary>
-        /// Describes the range of random colors to assign to tagged objects
+        /// The range of random colors to assign to target objects
+        [Tooltip("The range of random colors to assign to target objects.")]
         public ColorHsvaParameter colorParameter;
 
         /// <summary>

com.unity.perception/Runtime/Randomization/Randomizers/RandomizerExamples/Randomizers/ForegroundObjectPlacementRandomizer.cs

         /// <summary>
         /// The Z offset component applied to the generated layer of GameObjects
         /// </summary>
+        [Tooltip("The Z offset applied to positions of all placed objects.")]
+        [Tooltip("The minimum distance between the centers of the placed objects.")]
+        [Tooltip("The width and height of the area in which objects will be placed. These should be positive numbers and sufficiently large in relation with the Separation Distance specified.")]
+        [Tooltip("The list of Prefabs to be placed by this Randomizer.")]
         public GameObjectParameter prefabs;
 
         GameObject m_Container;

com.unity.perception/Runtime/Randomization/Randomizers/RandomizerExamples/Randomizers/HueOffsetRandomizer.cs

         static readonly int k_HueOffsetShaderProperty = Shader.PropertyToID("_HueOffset");
 
         /// <summary>
-        /// The range of hue offsets to assign to tagged objects
+        /// The range of random hue offsets to assign to target objects
+        [Tooltip("The range of random hue offsets to assign to target objects.")]
         public FloatParameter hueOffset = new FloatParameter { value = new UniformSampler(-180f, 180f) };
 
         /// <summary>

com.unity.perception/Runtime/Randomization/Randomizers/RandomizerExamples/Randomizers/RotationRandomizer.cs

     public class RotationRandomizer : Randomizer
     {
         /// <summary>
-        /// Defines the range of random rotations that can be assigned to tagged objects
+        /// The range of random rotations to assign to target objects
+        [Tooltip("The range of random rotations to assign to target objects.")]
         public Vector3Parameter rotation = new Vector3Parameter
         {
             x = new UniformSampler(0, 360),

com.unity.perception/Runtime/Randomization/Randomizers/RandomizerExamples/Randomizers/SunAngleRandomizer.cs

     public class SunAngleRandomizer : Randomizer
     {
         /// <summary>
-        /// The hour of the day (0 to 24)
+        /// The range of hours in a day (default is 0 to 24)
+        [Tooltip("The range of hours in a day (default is 0 to 24).")]
-        /// The day of the year (0 being Jan 1st and 364 being December 31st)
+        /// The range of days in a year with 0 being Jan 1st and 364 being December 31st (default is 0 to 364)
-        public FloatParameter dayOfTheYear = new FloatParameter { value = new UniformSampler(0, 365)};
+        [Tooltip("The range of days in a year with 0 being Jan 1st and 364 being December 31st (default is 0 to 364).")]
+        public FloatParameter dayOfTheYear = new FloatParameter { value = new UniformSampler(0, 364)};
-        /// The earth's latitude (-90 is the south pole, 0 is the equator, and +90 is the north pole)
+        /// The range of latitudes. A latitude of -90 is the south pole, 0 is the equator, and +90 is the north pole (default is -90 to 90).
+        [Tooltip("The range of latitudes. A latitude of -90 is the south pole, 0 is the equator, and +90 is the north pole (default is -90 to 90).")]
         public FloatParameter latitude = new FloatParameter { value = new UniformSampler(-90, 90)};
 
         /// <summary>

com.unity.perception/Runtime/Randomization/Randomizers/RandomizerExamples/Randomizers/TextureRandomizer.cs

 #endif
 
         /// <summary>
-        /// The list of textures to sample and apply to tagged objects
+        /// The list of textures to sample and apply to target objects
+        [Tooltip("The list of textures to sample and apply to target objects.")]
         public Texture2DParameter texture;
 
         /// <summary>

com.unity.perception/Runtime/Unity.Perception.Runtime.asmdef

             "name": "com.unity.simulation.capture",
             "expression": "0.0.10-preview.16",
             "define": "SIMULATION_CAPTURE_0_0_10_PREVIEW_16_OR_NEWER"
+        },
+        {
+            "name": "com.unity.render-pipelines.high-definition",
+            "expression": "9.0",
+            "define": "HDRP_9_OR_NEWER"
-}
+}

com.unity.perception/Tests/Runtime/GroundTruthTests/BoundingBox3dTests.cs

                     labelId = 1,
                     labelName = "label",
                     position = new Vector3(0, 0, 10),
-                    scale = new Vector3(5, 5, 5),
+                    scale = new Vector3(10, 10, 10),
                     rotation = Quaternion.identity
                 }
             };
                     labelId = 1,
                     labelName = "label",
                     position = new Vector3(0, 0, Mathf.Sqrt(200)),
-                    scale = new Vector3(5, 5, 5),
+                    scale = new Vector3(10, 10, 10),
                     rotation = Quaternion.identity
                 }
             };
                     labelId = 1,
                     labelName = "label",
                     position = new Vector3(0, 0, 10),
-                    scale = new Vector3(6.5f, 2.5f, 2.5f),
+                    scale = new Vector3(13f, 5f, 5f),
                     rotation = Quaternion.identity
                 }
             };
             return ExecuteTest(target, cameraPosition, cameraRotation, expected);
         }
 
+        public class ParentedTestData
+        {
+            public string name;
+
+            public Vector3 expectedScale = Vector3.one;
+            public Vector3 expectedPosition = new Vector3(0, 0, 10);
+            public Quaternion expectedRotation = Quaternion.identity;
+
+            public Vector3 childScale = Vector3.one;
+            public Vector3 childPosition = Vector3.zero;
+            public Quaternion childRotation = Quaternion.identity;
+
+            public Vector3 grandchildScale = Vector3.one;
+            public Vector3 grandchildPosition = Vector3.zero;
+            public Quaternion grandchildRotation = Quaternion.identity;
+
+            public Vector3 parentScale = Vector3.one;
+            public Vector3 parentPosition = Vector3.zero;
+            public Quaternion parentRotation = Quaternion.identity;
+
+            public Vector3 grandparentScale = Vector3.one;
+            public Vector3 grandparentPosition = Vector3.zero;
+            public Quaternion grandparentRotation = Quaternion.identity;
+
+            public Vector3 cameraParentScale = Vector3.one;
+            public Vector3 cameraParentPosition = Vector3.zero;
+            public Quaternion cameraParentRotation = Quaternion.identity;
+
+            public Vector3 cameraScale = Vector3.one;
+            public Vector3 cameraPosition = new Vector3(0, 0, -10);
+            public Quaternion cameraRotation = Quaternion.identity;
+
+            public override string ToString()
+            {
+                return name;
+            }
+        }
+
+        static IEnumerable<ParentedTestData> ParentedObject_ProduceProperResults_Values()
+        {
+            yield return new ParentedTestData()
+            {
+                name = "ParentScale",
+                expectedScale = new Vector3(1/5f, 1/5f, 1/5f),
+                parentScale = new Vector3(1/5f, 1/5f, 1/5f),
+            };
+            yield return new ParentedTestData()
+            {
+                name = "GrandparentScale",
+                expectedScale = new Vector3(1/5f, 1/5f, 1/5f),
+                grandparentScale = new Vector3(1/5f, 1/5f, 1/5f),
+            };
+            yield return new ParentedTestData()
+            {
+                name = "ChildScale",
+                expectedScale = new Vector3(1/5f, 1/5f, 1/5f),
+                childScale = new Vector3(1/5f, 1/5f, 1/5f),
+            };
+            yield return new ParentedTestData()
+            {
+                name = "ChildAndParentScale",
+                expectedScale = new Vector3(1f, 1f, 1f),
+                childScale = new Vector3(5, 5, 5),
+                parentScale = new Vector3(1/5f, 1/5f, 1/5f),
+            };
+            yield return new ParentedTestData()
+            {
+                name = "GrandchildScale",
+                expectedScale = new Vector3(2, 2, 2),
+                childScale = new Vector3(2, 2, 2),
+                grandchildScale = new Vector3(5, 5, 5),
+                parentScale = new Vector3(1/5f, 1/5f, 1/5f),
+            };
+            yield return new ParentedTestData()
+            {
+                name = "ParentRotation",
+                expectedRotation = Quaternion.Euler(1f, 2f, 3f),
+                parentRotation = Quaternion.Euler(1f, 2f, 3f),
+            };
+            yield return new ParentedTestData()
+            {
+                name = "ChildRotation",
+                expectedRotation = Quaternion.Euler(1f, 2f, 3f),
+                childRotation = Quaternion.Euler(1f, 2f, 3f),
+            };
+            yield return new ParentedTestData()
+            {
+                name = "ParentAndChildRotation",
+                expectedRotation = Quaternion.identity,
+                childRotation = Quaternion.Euler(20f, 0, 0),
+                parentRotation = Quaternion.Euler(-20f, 0, 0),
+            };
+            var diagonalSize = Mathf.Sqrt(2 * 2 + 2 * 2); //A^2 + B^2 = C^2
+            yield return new ParentedTestData()
+            {
+                name = "GrandchildRotation",
+                expectedRotation = Quaternion.identity,
+                expectedScale = new Vector3(diagonalSize / 2f, 1, diagonalSize / 2f),
+                grandchildRotation = Quaternion.Euler(0, 45, 0),
+            };
+            yield return new ParentedTestData()
+            {
+                name = "GrandparentRotation",
+                expectedRotation = Quaternion.Euler(-20f, 0, 0),
+                grandparentRotation = Quaternion.Euler(-20f, 0, 0),
+            };
+            yield return new ParentedTestData()
+            {
+                name = "GrandchildTRS",
+                expectedRotation = Quaternion.identity,
+                expectedPosition = new Vector3(-5, 0, 10),
+                expectedScale = new Vector3(.5f * diagonalSize / 2f, .5f, .5f * diagonalSize / 2f),
+                grandchildRotation = Quaternion.Euler(0, -45, 0),
+                grandchildPosition = new Vector3(-5, 0, 0),
+                grandchildScale = new Vector3(.5f, .5f, .5f),
+            };
+            yield return new ParentedTestData()
+            {
+                name = "CamParentPositionAndScale",
+                expectedRotation = Quaternion.identity,
+                expectedPosition = new Vector3(2, 0, 6.5f),
+                expectedScale = new Vector3(1, 1, 1),
+                childPosition = new Vector3(0, 0, 4),
+                cameraParentPosition = new Vector3(-2, 0, 0),
+                cameraParentScale = new Vector3(1/2f, 1/3f, 1/4f),
+            };
+            //point at the left side of the box
+            yield return new ParentedTestData()
+            {
+                name = "CamParentRotate",
+                expectedRotation = Quaternion.Euler(0, -90, 0),
+                expectedPosition = new Vector3(0, 0, 10),
+                cameraParentRotation = Quaternion.Euler(0, 90, 0),
+            };
+            //point at the left side of the box
+            yield return new ParentedTestData()
+            {
+                name = "CamParentScale",
+                expectedPosition = new Vector3(0, 0, 2.5f),
+                //Scale on the camera's hierarchy only affects the position of the camera. It does not affect the camera frustum
+                cameraParentScale = new Vector3(1/2f, 1/3f, 1/4f),
+            };
+            yield return new ParentedTestData()
+            {
+                name = "CamRotationParentScale",
+                expectedRotation = Quaternion.Euler(0, -90, 0),
+                expectedPosition = new Vector3(0, 0, 5),
+                cameraParentPosition = new Vector3(-5, 0, 0),
+                cameraParentScale = new Vector3(.5f, 1, 1),
+                cameraPosition = Vector3.zero,
+                cameraRotation = Quaternion.Euler(0, 90, 0),
+            };
+        }
+        [UnityTest]
+        public IEnumerator ParentedObject_ProduceProperResults([ValueSource(nameof(ParentedObject_ProduceProperResults_Values))] ParentedTestData parentedTestData)
+        {
+            var expected = new[]
+            {
+                new ExpectedResult
+                {
+                    instanceId = 1,
+                    labelId = 1,
+                    labelName = "label",
+                    position = parentedTestData.expectedPosition,
+                    scale = parentedTestData.expectedScale,
+                    rotation = parentedTestData.expectedRotation
+                }
+            };
+
+            var goGrandparent = new GameObject();
+            goGrandparent.transform.localPosition = parentedTestData.grandparentPosition;
+            goGrandparent.transform.localScale = parentedTestData.grandparentScale;
+            goGrandparent.transform.localRotation = parentedTestData.grandparentRotation;
+
+            var goParent = new GameObject();
+            goParent.transform.SetParent(goGrandparent.transform, false);
+            goParent.transform.localPosition = parentedTestData.parentPosition;
+            goParent.transform.localScale = parentedTestData.parentScale;
+            goParent.transform.localRotation = parentedTestData.parentRotation;
+
+            var goChild = new GameObject();
+            goChild.transform.SetParent(goParent.transform, false);
+
+            goChild.transform.localPosition = parentedTestData.childPosition;
+            goChild.transform.localScale = parentedTestData.childScale;
+            goChild.transform.localRotation = parentedTestData.childRotation;
+
+            var labeling = goChild.AddComponent<Labeling>();
+            labeling.labels.Add("label");
+
+            var goGrandchild = GameObject.CreatePrimitive(PrimitiveType.Cube);
+            goGrandchild.transform.SetParent(goChild.transform, false);
+
+            goGrandchild.transform.localPosition = parentedTestData.grandchildPosition;
+            goGrandchild.transform.localScale = parentedTestData.grandchildScale;
+            goGrandchild.transform.localRotation = parentedTestData.grandchildRotation;
+
+            var goCameraParent = new GameObject();
+            goCameraParent.transform.localPosition = parentedTestData.cameraParentPosition;
+            goCameraParent.transform.localScale = parentedTestData.cameraParentScale;
+            goCameraParent.transform.localRotation = parentedTestData.cameraParentRotation;
+
+            var receivedResults = new List<(int, List<BoundingBox3DLabeler.BoxData>)>();
+            var cameraObject = SetupCamera(SetupLabelConfig(), (frame, data) =>
+            {
+                receivedResults.Add((frame, data));
+            });
+
+            cameraObject.transform.SetParent(goCameraParent.transform, false);
+            cameraObject.transform.localPosition = parentedTestData.cameraPosition;
+            cameraObject.transform.localScale = parentedTestData.cameraScale;
+            cameraObject.transform.localRotation = parentedTestData.cameraRotation;
+            cameraObject.SetActive(true);
+
+            return ExecuteTestOnCamera(goGrandparent, expected, goCameraParent, receivedResults);
+        }
+
         [UnityTest]
         public IEnumerator MultiInheritedMesh_ProduceProperTranslationTest()
         {
                     labelId = 2,
                     labelName = "car",
                     position = new Vector3(0, 0.525f, 20),
-                    scale = new Vector3(2f, 0.875f, 2.4f),
+                    scale = new Vector3(4f, 1.75f, 4.8f),
                     rotation = Quaternion.identity
                 },
             };
         [UnityTest]
         public IEnumerator MultiInheritedMeshDifferentLabels_ProduceProperTranslationTest()
         {
-            var wheelScale = new Vector3(0.35f, 1.0f, 0.35f);
+            var wheelScale = new Vector3(0.7f, 2.0f, 0.7f);
             var wheelRot = Quaternion.Euler(0, 0, 90);
 
             var expected = new[]
                     labelId = 2,
                     labelName = "car",
                     position = new Vector3(0, 1.05f, 20),
-                    scale = new Vector3(1f, 0.7f, 2.4f),
+                    scale = new Vector3(2f, 1.4f, 4.8f),
                     rotation = Quaternion.identity
                 },
                 new ExpectedResult
             var target = TestHelper.CreateLabeledCube(scale: 15f, z: -50f);
             return ExecuteSeenUnseenTest(target, Vector3.zero, quaternion.identity, 0);
         }
-        
-        
+
+
         struct ExpectedResult
         {
             public int labelId;
 
             yield return null;
             yield return null;
-            
+
-        
+
         IEnumerator ExecuteTest(GameObject target, Vector3 cameraPos, Quaternion cameraRotation, IList<ExpectedResult> expectations)
         {
             var receivedResults = new List<(int, List<BoundingBox3DLabeler.BoxData>)>();
             gameObject.transform.position = cameraPos;
             gameObject.transform.rotation = cameraRotation;
 
-            AddTestObjectForCleanup(gameObject);
+            return ExecuteTestOnCamera(target, expectations, gameObject, receivedResults);
+        }
-            gameObject.SetActive(false);
+        private IEnumerator ExecuteTestOnCamera(GameObject target, IList<ExpectedResult> expectations, GameObject cameraObject,
+            List<(int, List<BoundingBox3DLabeler.BoxData>)> receivedResults)
+        {
+            AddTestObjectForCleanup(cameraObject);
+            AddTestObjectForCleanup(target);
+
+            cameraObject.SetActive(false);
-            gameObject.SetActive(true);
+            cameraObject.SetActive(true);
-            
+
             Assert.AreEqual(expectations.Count, receivedResults[0].Item2.Count);
 
             for (var i = 0; i < receivedResults[0].Item2.Count; i++)
                 TestResults(b, expectations[i]);
             }
 
-            DestroyTestObject(gameObject);
-            UnityEngine.Object.DestroyImmediate(target);
+            DestroyTestObject(cameraObject);
         }
 
         static IdLabelConfig SetupLabelConfig()
 
         static void TestResults(BoundingBox3DLabeler.BoxData data, ExpectedResult e)
         {
-            var scale = e.scale * 2;
-
-            Assert.AreEqual(scale[0], data.size[0], k_Delta);
-            Assert.AreEqual(scale[1], data.size[1], k_Delta);
-            Assert.AreEqual(scale[2], data.size[2], k_Delta);
+            Assert.AreEqual(e.scale[0], data.size[0], k_Delta);
+            Assert.AreEqual(e.scale[1], data.size[1], k_Delta);
+            Assert.AreEqual(e.scale[2], data.size[2], k_Delta);
             Assert.AreEqual(e.rotation[0], data.rotation[0], k_Delta);
             Assert.AreEqual(e.rotation[1], data.rotation[1], k_Delta);
             Assert.AreEqual(e.rotation[2], data.rotation[2], k_Delta);

com.unity.perception/Tests/Runtime/GroundTruthTests/KeypointGroundTruthTests.cs

     public class KeyPointGroundTruthTests : GroundTruthTestBase, IPrebuildSetup, IPostBuildCleanup
     {
         private const string kAnimatedCubeScenePath = "Packages/com.unity.perception/Tests/Runtime/TestAssets/AnimatedCubeScene.unity";
+        private const double k_Delta = .01;
 
         public void Setup()
         {
 #endif
         }
 
-        static GameObject SetupCamera(IdLabelConfig config, KeypointTemplate template, Action<int, List<KeypointLabeler.KeypointEntry>> computeListener, RenderTexture renderTexture = null)
+        static GameObject SetupCamera(IdLabelConfig config, KeypointTemplate template, Action<int, List<KeypointLabeler.KeypointEntry>> computeListener, RenderTexture renderTexture = null, KeypointObjectFilter keypointObjectFilter = KeypointObjectFilter.Visible)
         {
             var cameraObject = new GameObject();
             cameraObject.SetActive(false);
             var perceptionCamera = cameraObject.AddComponent<PerceptionCamera>();
             perceptionCamera.captureRgbImages = false;
             var keyPointLabeler = new KeypointLabeler(config, template);
+            keyPointLabeler.objectFilter = keypointObjectFilter;
             if (computeListener != null)
                 keyPointLabeler.KeypointsComputed += computeListener;
 
         }
 
         [UnityTest]
-        public IEnumerator Keypoint_TestAllBlockedByOther()
+        public IEnumerator Keypoint_FullyOccluded_DoesNotReport()
         {
             var incoming = new List<List<KeypointLabeler.KeypointEntry>>();
             var template = CreateTestTemplate(Guid.NewGuid(), "TestTemplate");
                 incoming.Add(new List<KeypointLabeler.KeypointEntry>(data));
             }, texture);
 
+            CreateFullyOccludedScene(template, cam);
+
+            yield return null;
+
+            //force all async readbacks to complete
+            DestroyTestObject(cam);
+
+            if (texture != null) texture.Release();
+
+            var testCase = incoming.Last();
+            Assert.AreEqual(0, testCase.Count);
+        }
+        [UnityTest]
+        public IEnumerator Keypoint_FullyOccluded_WithIncludeOccluded_ReportsProperly(
+            [Values(KeypointObjectFilter.VisibleAndOccluded, KeypointObjectFilter.All)] KeypointObjectFilter keypointObjectFilter)
+        {
+            var incoming = new List<List<KeypointLabeler.KeypointEntry>>();
+            var template = CreateTestTemplate(Guid.NewGuid(), "TestTemplate");
+            var texture = new RenderTexture(1024, 768, 16);
+            texture.Create();
+            var cam = SetupCamera(SetUpLabelConfig(), template, (frame, data) =>
+            {
+                incoming.Add(data);
+            }, texture, keypointObjectFilter);
+
+            CreateFullyOccludedScene(template, cam);
+
+            yield return null;
+
+            //force all async readbacks to complete
+            DestroyTestObject(cam);
+
+            if (texture != null) texture.Release();
+
+            var testCase = incoming.Last();
+            Assert.AreEqual(1, testCase.Count);
+
+            var t = testCase.First();
+            Assert.NotNull(t);
+            Assert.AreEqual(1, t.instance_id);
+            Assert.AreEqual(1, t.label_id);
+            Assert.AreEqual(template.templateID.ToString(), t.template_guid);
+            Assert.AreEqual(9, t.keypoints.Length);
+
+            for (var i = 0; i < 8; i++)
+                Assert.AreEqual(1, t.keypoints[i].state);
+
+            for (var i = 0; i < 9; i++) Assert.AreEqual(i, t.keypoints[i].index);
+            Assert.Zero(t.keypoints[8].state);
+            Assert.Zero(t.keypoints[8].x);
+            Assert.Zero(t.keypoints[8].y);
+        }
+
+        private void CreateFullyOccludedScene(KeypointTemplate template, GameObject cam)
+        {
             var cube = TestHelper.CreateLabeledCube(scale: 6, z: 8);
             SetupCubeJoints(cube, template);
 
             AddTestObjectForCleanup(cam);
             AddTestObjectForCleanup(cube);
             AddTestObjectForCleanup(blocker);
+        }
+
+
+        [UnityTest]
+        public IEnumerator Keypoint_Offscreen_DoesNotReport(
+            [Values(KeypointObjectFilter.VisibleAndOccluded, KeypointObjectFilter.Visible)] KeypointObjectFilter keypointObjectFilter)
+        {
+            var incoming = new List<List<KeypointLabeler.KeypointEntry>>();
+            var template = CreateTestTemplate(Guid.NewGuid(), "TestTemplate");
+            var texture = new RenderTexture(1024, 768, 16);
+            texture.Create();
+            var cam = SetupCamera(SetUpLabelConfig(), template, (frame, data) =>
+            {
+                incoming.Add(data);
+            }, texture, keypointObjectFilter);
+
+            var cube = TestHelper.CreateLabeledCube(scale: 6, z: -100);
+            SetupCubeJoints(cube, template);
+
+            cube.SetActive(true);
+            cam.SetActive(true);
+
+            AddTestObjectForCleanup(cam);
+            AddTestObjectForCleanup(cube);
+
+            yield return null;
+
+            //force all async readbacks to complete
+            DestroyTestObject(cam);
+
+            if (texture != null) texture.Release();
+
+            var testCase = incoming.Last();
+            Assert.AreEqual(0, testCase.Count);
+        }
+        [UnityTest]
+        public IEnumerator Keypoint_Offscreen_WithIncludeAll_ReportsProperly()
+        {
+            var incoming = new List<List<KeypointLabeler.KeypointEntry>>();
+            var template = CreateTestTemplate(Guid.NewGuid(), "TestTemplate");
+            var texture = new RenderTexture(1024, 768, 16);
+            texture.Create();
+            var cam = SetupCamera(SetUpLabelConfig(), template, (frame, data) =>
+            {
+                incoming.Add(data);
+            }, texture, KeypointObjectFilter.All);
+
+            var cube = TestHelper.CreateLabeledCube(scale: 6, z: -20);
+            SetupCubeJoints(cube, template);
+
+            cube.SetActive(true);
+            cam.SetActive(true);
+
+            AddTestObjectForCleanup(cam);
+            AddTestObjectForCleanup(cube);
 
             yield return null;
 
 
             var testCase = incoming.Last();
             Assert.AreEqual(1, testCase.Count);
+
             var t = testCase.First();
             Assert.NotNull(t);
             Assert.AreEqual(1, t.instance_id);
 
-            for (var i = 0; i < 8; i++)
-                Assert.AreEqual(1, t.keypoints[i].state);
-
-            for (var i = 0; i < 9; i++) Assert.AreEqual(i, t.keypoints[i].index);
-            Assert.Zero(t.keypoints[8].state);
-            Assert.Zero(t.keypoints[8].x);
-            Assert.Zero(t.keypoints[8].y);
+            for (var i = 0; i < 9; i++)
+            {
+                Assert.Zero(t.keypoints[i].state);
+                Assert.Zero(t.keypoints[i].x);
+                Assert.Zero(t.keypoints[i].y);
+            }
         }
 
         [UnityTest]
             Assert.AreEqual(screenPointCenterExpected.y, t.keypoints[8].y, Screen.height * .1);
             Assert.AreEqual(8, t.keypoints[8].index);
             Assert.AreEqual(2, t.keypoints[8].state);
+        }
+
+        static IEnumerable<(float scale, bool expectObject, int expectedState, KeypointObjectFilter keypointFilter, Vector2 expectedTopLeft, Vector2 expectedBottomRight)> Keypoint_OnBox_ReportsProperCoordinates_TestCases()
+        {
+            yield return (
+                1,
+                true,
+                2,
+                KeypointObjectFilter.Visible,
+                new Vector2(0, 0),
+                new Vector2(1023.99f, 1023.99f));
+            yield return (
+                1.001f,
+                true,
+                0,
+                KeypointObjectFilter.Visible,
+                new Vector2(0, 0),
+                new Vector2(0, 0));
+            yield return (
+                1.2f,
+                true,
+                0,
+                KeypointObjectFilter.Visible,
+                new Vector2(0, 0),
+                new Vector2(0, 0));
+            yield return (
+                0f,
+                false,
+                1,
+                KeypointObjectFilter.Visible,
+                new Vector2(512, 512),
+                new Vector2(512, 512));
+            yield return (
+                0f,
+                true,
+                1,
+                KeypointObjectFilter.VisibleAndOccluded,
+                new Vector2(512, 512),
+                new Vector2(512, 512));
+        }
+        [UnityTest]
+        public IEnumerator Keypoint_OnBox_ReportsProperCoordinates(
+            [ValueSource(nameof(Keypoint_OnBox_ReportsProperCoordinates_TestCases))]
+            (float scale, bool expectObject, int expectedState, KeypointObjectFilter keypointFilter, Vector2 expectedTopLeft, Vector2 expectedBottomRight) args)
+        {
+            var incoming = new List<List<KeypointLabeler.KeypointEntry>>();
+            var template = CreateTestTemplate(Guid.NewGuid(), "TestTemplate");
+            var frameSize = 1024;
+            var texture = new RenderTexture(frameSize, frameSize, 16);
+            var cam = SetupCamera(SetUpLabelConfig(), template, (frame, data) =>
+            {
+                incoming.Add(data);
+            }, texture, args.keypointFilter);
+
+            var camComponent = cam.GetComponent<Camera>();
+            camComponent.orthographic = true;
+            camComponent.orthographicSize = .5f;
+
+            var cube = TestHelper.CreateLabeledCube(scale: args.scale, z: 8);
+            SetupCubeJoints(cube, template);
+
+            cube.SetActive(true);
+            cam.SetActive(true);
+
+            AddTestObjectForCleanup(cam);
+            AddTestObjectForCleanup(cube);
+
+            yield return null;
+
+            //force all async readbacks to complete
+            DestroyTestObject(cam);
+            texture.Release();
+
+            var testCase = incoming.Last();
+            if (!args.expectObject)
+            {
+                Assert.AreEqual(0, testCase.Count);
+                yield break;
+            }
+            Assert.AreEqual(1, testCase.Count);
+            var t = testCase.First();
+            Assert.NotNull(t);
+            Assert.AreEqual(1, t.instance_id);
+            Assert.AreEqual(1, t.label_id);
+            Assert.AreEqual(template.templateID.ToString(), t.template_guid);
+            Assert.AreEqual(9, t.keypoints.Length);
+
+            CollectionAssert.AreEqual(Enumerable.Repeat(args.expectedState, 8), t.keypoints.Take(8).Select(k => k.state), "State mismatch");
+            Assert.AreEqual(args.expectedTopLeft.x, t.keypoints[0].x, k_Delta);
+            Assert.AreEqual(args.expectedBottomRight.y, t.keypoints[0].y, k_Delta);
+
+            Assert.AreEqual(args.expectedTopLeft.x, t.keypoints[1].x, k_Delta);
+            Assert.AreEqual(args.expectedTopLeft.y, t.keypoints[1].y, k_Delta);
+
+            Assert.AreEqual(args.expectedBottomRight.x, t.keypoints[2].x, k_Delta);
+            Assert.AreEqual(args.expectedTopLeft.y, t.keypoints[2].y, k_Delta);
+
+            Assert.AreEqual(args.expectedBottomRight.x, t.keypoints[3].x, k_Delta);
+            Assert.AreEqual(args.expectedBottomRight.y, t.keypoints[3].y, k_Delta);
         }
     }
 }

com.unity.perception/package.json

   "displayName": "Perception",
   "name": "com.unity.perception",
   "unity": "2019.4",
-  "version": "0.8.0-preview.2",
+  "version": "0.8.0-preview.3",
   "samples": [
     {
       "displayName": "Tutorial Files",

com.unity.perception/Editor/GroundTruth/JointLabelEditor.cs

+using System.Linq;
+using UnityEngine;
+using UnityEngine.Perception.GroundTruth;
+
+namespace UnityEditor.Perception.GroundTruth
+{
+    [CustomEditor(typeof(JointLabel))]
+    public class JointLabelEditor : Editor
+    {
+        public override void OnInspectorGUI()
+        {
+            base.OnInspectorGUI();
+#if UNITY_2020_1_OR_NEWER
+            //GetComponentInParent<T>(bool includeInactive) only exists on 2020.1 and later
+            if (targets.Any(t => ((Component)t).gameObject.GetComponentInParent<Labeling>(true) == null))
+#else
+            if (targets.Any(t => ((Component)t).GetComponentInParent<Labeling>() == null))
+#endif
+                EditorGUILayout.HelpBox("No Labeling component detected on parents. Keypoint labeling requires a Labeling component on the root of the object.", MessageType.Info);
+        }
+    }
+}

com.unity.perception/Editor/GroundTruth/JointLabelEditor.cs.meta

+fileFormatVersion: 2
+guid: 8b21d46736dd4cbb96bf827457752855
+timeCreated: 1616108507

com.unity.perception/Runtime/GroundTruth/Labelers/KeypointObjectFilter.cs

+namespace UnityEngine.Perception.GroundTruth
+{
+    /// <summary>
+    /// Keypoint filtering modes.
+    /// </summary>
+    public enum KeypointObjectFilter
+    {
+        /// <summary>
+        /// Only include objects which are partially visible in the frame.
+        /// </summary>
+        [InspectorName("Visible objects")]
+        Visible,
+        /// <summary>
+        /// Include visible objects and objects with keypoints in the frame.
+        /// </summary>
+        [InspectorName("Visible and occluded objects")]
+        VisibleAndOccluded,
+        /// <summary>
+        /// Include all labeled objects containing matching skeletons.
+        /// </summary>
+        [InspectorName("All objects")]
+        All
+    }
+}

com.unity.perception/Runtime/GroundTruth/Labelers/KeypointObjectFilter.cs.meta

+fileFormatVersion: 2
+guid: a1ea6aeb2b1c476288b4aa0cbc795280
+timeCreated: 1616179952

com.unity.perception/Documentation~/GroundTruth/KeypointLabeler.md

+# Keypoint Labeler
+
+The Keypoint Labeler captures the screen locations of specific points on labeled GameObjects. The typical use of this Labeler is capturing human pose estimation data, but it can be used to capture points on any kind of object. The Labeler uses a [Keypoint Template](#KeypointTemplate) which defines the keypoints to capture for the model and the skeletal connections between those keypoints. The positions of the keypoints are recorded in pixel coordinates.
+
+## Data Format
+The keypoints captured each frame are in the following format:
+```
+keypoints {
+  label_id:      <int>   -- Integer identifier of the label
+  instance_id:   <str>   -- UUID of the instance.
+  template_guid: <str>   -- UUID of the keypoint template
+  pose:          <str>   -- Current pose
+  keypoints [            -- Array of keypoint data, one entry for each keypoint defined in associated template file.
+    {
+      index:     <int>   -- Index of keypoint in template
+      x:         <float> -- X pixel coordinate of keypoint
+      y:         <float> -- Y pixel coordinate of keypoint
+      state:     <int>   -- Visibility state
+    }, ...
+  ]
+}
+```
+
+The `state` entry has three possible values: 
+* 0 - the keypoint either does not exist or is outside of the image's bounds
+* 1 - the keypoint exists inside of the image bounds but cannot be seen because the object is not visible at its location in the image
+* 2 - the keypoint exists and the object is visible at its location
+
+The annotation definition, captured by the Keypoint Labeler once in each dataset, describes points being captured and their skeletal connections. These are defined by the [Keypoint Template](#KeypointTemplate).
+```
+annotation_definition.spec {
+  template_id:       <str>           -- The UUID of the template
+  template_name:     <str>           -- Human readable name of the template
+  key_points [                       -- Array of joints defined in this template
+    {
+      label:         <str>           -- The label of the joint
+      index:         <int>           -- The index of the joint
+    }, ...
+  ]
+  skeleton [                         -- Array of skeletal connections (which joints have connections between one another) defined in this template
+    {
+      joint1:        <int>           -- The first joint of the connection
+      joint2:        <int>           -- The second joint of the connection
+    }, ...
+  ]
+}
+```
+
+## Setup
+The Keypoint Labeler captures keypoints each frame from each object in the scene that meets the following conditions:
+
+* The object or its children are at least partially visible in the frame 
+  * The _Object Filter_ option on the Keypoint Labeler can be used to also include fully occluded or off-screen objects
+* The root object has a `Labeling` component
+* The object matches at least one entry in the Keypoint Template by either:
+  * Containing an Animator with a [humanoid avatar](https://docs.unity3d.com/Manual/ConfiguringtheAvatar.html) whose rig matches a keypoint OR
+  * Containing children with Joint Label components whose labels match keypoints
+
+For a tutorial on setting up your project for keypoint labeling, see the [Human Pose Labeling and Randomization Tutorial](../HPTutorial/TUTORIAL.md).
+
+## Keypoint Template
+
+Keypoint Templates are used to define the keypoints and skeletal connections captured by the Keypoint Labeler. The Keypoint Template takes advantage of Unity's humanoid animation rig, and allows the user to automatically associate template keypoints to animation rig joints. Additionally, the user can choose to ignore the rigged points, or add points not defined in the rig.
+
+A [COCO](https://cocodataset.org/#home) Keypoint Template is included in the Perception package.
+
+### Editor
+
+The Keypoint Template editor allows the user to create/modify a Keypoint Template. The editor consists of the header information, the keypoint array, and the skeleton array.
+
+![Header section of the keypoint template](../images/keypoint_template_header.png)
+<br/>_Header section of the keypoint template_
+
+In the header section, a user can change the name of the template and supply textures that they would like to use for the keypoint visualization.
+
+![The keypoint section of the keypoint template](../images/keypoint_template_keypoints.png)
+<br/>_Keypoint section of the keypoint template_
+
+The keypoint section allows the user to create/edit keypoints and associate them with Unity animation rig points. Each keypoint record
+has 4 fields: label (the name of the keypoint), Associate to Rig (a boolean value which, if true, automatically maps the keypoint to
+the GameObject defined by the rig), Rig Label (only needed if Associate To Rig is true, defines which rig component to associate with
+the keypoint), and Color (RGB color value of the keypoint in the visualization).
+
+![Skeleton section of the keypoint template](../images/keypoint_template_skeleton.png)
+<br/>_Skeleton section of the keypoint template_
+
+The skeleton section allows the user to create connections between joints, basically defining the skeleton of a labeled object.
+
+#### Animation Pose Label
+
+This file is used to define timestamps in an animation to a pose label.