浏览代码

Fixes for PR comments

/keypoint_self_occlusion
Steve Borkman 3 年前
当前提交
c7a5c2ef
共有 4 个文件被更改,包括 17 次插入22 次删除
  1. 20
      com.unity.perception/Documentation~/GroundTruth/KeypointLabeler.md
  2. 4
      com.unity.perception/Runtime/GroundTruth/Labelers/KeypointLabeler.cs
  3. 13
      com.unity.perception/Runtime/GroundTruth/Labelers/KeypointOcclusionOverrides.cs
  4. 2
      com.unity.perception/Tests/Runtime/GroundTruthTests/KeypointGroundTruthTests.cs

20
com.unity.perception/Documentation~/GroundTruth/KeypointLabeler.md


The `state` entry has three possible values:
* 0 - the keypoint either does not exist or is outside of the image's bounds
* 1 - the keypoint is defined and inside of the image's bounds but cannot be seen because it is either occluded by
another object or itself
* 1 - the keypoint is defined and inside of the image's bounds but is occluded
* 2 - the keypoint exists and is visible at its location
The annotation definition, captured by the Keypoint Labeler once in each dataset, describes points being captured and their skeletal connections. These are defined by the [Keypoint Template](#KeypointTemplate).

A keypoint will be considered not visible if it outside of the camera's view, the camera cannot see it because
another object is occluding it (in between the camera and the keypoint), or another part of its own model is occluding it (for
example a human model's arm is raised and blocking its face from view). The keypoint is a point in space generally internal
to the model, for example the elbow joint is in the center of the arm volume. A self occlusion distance value has
been added to each keypoint allowing the keypoint to have depth. If a keypoint is occluded by the mesh that it is residing in,
then this value is added to keypoint location to see if that location is now closer than the mesh which is occluding the object.
If it is then the keypoint will be marked as visible, if it is not, then the keypoint will be marked as not visible. In the
case of the elbow, the distance should be enough that it is not blocked by the volume of the arm.
to the model, for example the elbow joint is in the center of the arm volume. Each keypoint has a self occlusion distance value
which provides depth to the keypoint. The keypoint distance can be assigned or modified via three different methods:
1) The unique distance can be defined to all keypoints defined in the keypoint template.
2) The occlusion distance for a particular keypont can be modified using a Joint Label component on a game object. If a Joint Label
is associated to a keypoint, the self occlusion service will use that value instead of the value found in the keypoint template file.
3) All of the keypoint distances in a model can be scaled for a model using the KeypointOcclusionOverrides component. The distanceScale will
applied to all of the keypoints for that model, a value of 1.0 means that the values are not modified.
If a keypoiant is occluded by the mesh that it is residing in, then this value is added to keypoint location to see if that
location is now closer than the mesh which is occluding the object. If it is then the keypoint will be marked as visible,
if it is not, then the keypoint will be marked as not visible. In the case of the elbow, the distance should be enough that
it is not blocked by the volume of the arm.
## Keypoint Template

4
com.unity.perception/Runtime/GroundTruth/Labelers/KeypointLabeler.cs


}
}
var occlusionOverrider = labeledEntity.GetComponent<KeypointOcclusionOverrides>();
var occlusionOverrider = labeledEntity.GetComponentInParent<KeypointOcclusionOverrides>();
cached.occlusionScalar = occlusionOverrider.overrideDistanceScale;
cached.occlusionScalar = occlusionOverrider.distanceScale;
}
m_KnownStatus[labeledEntity.instanceId] = cached;

13
com.unity.perception/Runtime/GroundTruth/Labelers/KeypointOcclusionOverrides.cs


public class KeypointOcclusionOverrides : MonoBehaviour
{
[Tooltip("Overrides the default occlusion distance values by a scalar. This is necessary for bodies with different body types (i.e. children should be less than one)")]
[SerializeField]
// ReSharper disable once InconsistentNaming, this needs to be human readable in the inspector
float distanceScale = 1.0f;
/// <summary>
/// The value to use to scale the values for keypoints distances defined in <see cref="KeypointTemplate"/>
/// </summary>
public float overrideDistanceScale
{
get => distanceScale;
internal set => distanceScale = value;
}
public float distanceScale = 1.0f;
}
}

2
com.unity.perception/Tests/Runtime/GroundTruthTests/KeypointGroundTruthTests.cs


SetupCubeJoint(cube, "Center", args.pointLocalPosition.x, args.pointLocalPosition.y, args.pointLocalPosition.z, args.checkDistance);
var kpOc = cube.AddComponent<KeypointOcclusionOverrides>();
kpOc.overrideDistanceScale = args.overrideScalar;
kpOc.distanceScale = args.overrideScalar;
cube.SetActive(true);
cam.SetActive(true);

正在加载...
取消
保存