Body tracking: data, avatar and role trackers

There are two different workflows: a ready skeleton from a body-tracking provider and separate waist/foot trackers. XRBodyTracker holds skeleton joints. Role trackers provide individual targets for application IK; they do not automatically assemble into XRBodyTracker or solve whole-body animation.

Check the provider first

Prepare XR startup. Obtain XRServer.GetTracker("/user/body_tracker") as XRBodyTracker. If null, establish provider support, tracker name and session lifecycle. Constructing new XRBodyTracker enables no tracking and connects no device. SDK classes do not establish that the built-in OpenXR module publishes a full skeleton on your platform.

Check three levels:

BodyFlags reports supported regions: UpperBodySupported, LowerBodySupported, HandsSupported. It does not replace joint flags. A device can supply only upper-body data or estimate some joint positions. VALID differs from TRACKED; choose whether your mechanic accepts estimated positions.

Confirmed constructor defect: initial JointFlags are uninitialized. This SDK has 87 joints; Joint.Max is the bound, not a joint. Gameplay must not enable HasTrackingData on an empty container. A custom provider must populate required joint flags/transforms first. Data loss retains old values; being readable does not make them current.

Read a joint relative to Root

Save BodyJointReader.cs. The example accepts Root and target transforms in one provider space, checks data and returns a Root-relative value copy. Pass Joint.Hips for the hips. This is neither a bone pose relative to its immediate parent nor a world transform.

using Reckfell;

public static class BodyJointReader
{
    // Provider transforms must share one space. Result is relative to Root, not a bone parent.
    public static bool TryRelativeToRoot(XRBodyTracker tracker, XRBodyTracker.Joint joint,
        out Transform3D relative)
    {
        relative = Transform3D.Identity;
        if (tracker is null || !tracker.HasTrackingData
            || (long)joint < 0 || (long)joint >= (long)XRBodyTracker.Joint.Max) return false;
        var root = XRBodyTracker.Joint.Root;
        const XRBodyTracker.JointFlags needed =
            XRBodyTracker.JointFlags.PositionValid | XRBodyTracker.JointFlags.OrientationValid;
        if ((tracker.GetJointFlags(root) & needed) != needed
            || (tracker.GetJointFlags(joint) & needed) != needed) return false;
        Transform3D rootTransform = tracker.GetJointTransform(root);
        Transform3D target = tracker.GetJointTransform(joint);
        float determinant = rootTransform.Basis.Determinant();
        if (!rootTransform.IsFinite() || !target.IsFinite()
            || !float.IsFinite(determinant) || Mathf.Abs(determinant) < 0.000001f) return false;
        Transform3D result = rootTransform.AffineInverse() * target;
        if (!result.IsFinite()) return false;
        relative = result;
        return true;
    }
}

The example requires PositionValid and OrientationValid, accepts estimated positions without TRACKED and does not assess accuracy. A singular Root, NaN/Infinity or an invalid index returns false without reading a nonexistent joint. BodyFlags is intentionally checked separately: capability metadata does not replace specific data validity. Interaction events additionally require application focus.

Connect an avatar

Supply a correct model skeleton, skin weights and the provider’s coordinate conventions:

XROrigin3D
└── AvatarRoot (Entity3D)
    └── Skeleton3D
        └── XRBodyModifier3D

Set XRBodyModifier3D.BodyTracker=/user/body_tracker before entering the tree. The modifier matches names such as Root, Hips, Spine, Chest, LeftUpperArm and RightFoot. Unmatched bones are skipped with warnings; it renames nothing and repairs no model weights.

BodyUpdate selects mapping regions: UpperBody, LowerBody, Hands; all three are enabled by default. Root and extra joints are not necessarily excluded by a zero mask, so use inherited Active=false to disable the modifier completely. BodyTracker switches the source without rebuilding bone mapping. Mapping depends on Skeleton and BodyUpdate, so a name change alone needs no remapping. Change BodyUpdate to select different regions.

Full updates positions and rotations. RotationOnly preserves most bone positions, but Hips still receives position updates for crouching/climbing. Root and required parent joints must be valid. Skeleton MotionScale applies to input positions; align it with the model and WorldScale.

Placing AvatarRoot in the gameplay world is separate. Do not assume XRBodyModifier3D automatically places the model at physical Root or that XRBodyTracker always publishes a positional default pose for XREntity3D. The provider must define both conventions explicitly. This page does not establish finished retargeting for your model.

Inspect modifier output

Skeleton3D applies modifiers temporarily, then restores base pose. For diagnosis read output inside ModificationProcessed. Save BonePoseProbe.cs, attach it to an ordinary Entity, then assign Skeleton, Modifier and an existing Bone index. LastModified is an event-result copy, not proof of current tracking.

using Reckfell;

public partial class BonePoseProbe : Entity
{
    [Export] public Skeleton3D Skeleton { get; set; } = null!;
    [Export] public SkeletonModifier3D Modifier { get; set; } = null!;
    [Export] public int Bone { get; set; }
    public Transform3D LastModified { get; private set; } = Transform3D.Identity;

    public override void _Ready()
    {
        if (Skeleton is null || Modifier is null || Modifier.GetSkeleton() != Skeleton
            || Bone < 0 || Bone >= Skeleton.GetBoneCount())
        {
            ReckfellApi.PushError("BonePoseProbe: assign a matching skeleton, modifier and bone index.");
            return;
        }
        Modifier.ModificationProcessed += Capture;
    }
    private void Capture() => LastModified = Skeleton.GetBonePose(Bone);
    public override void _ExitTree()
    {
        if (Modifier is not null && ReckfellObject.IsInstanceValid(Modifier))
            Modifier.ModificationProcessed -= Capture;
    }
}

Execution confirmed: Full updates positions and rotations; RotationOnly preserves the base Spine position but continues updating Hips position. HasTrackingData=false, invalid Root or a missing tracker skips the update; the hand-style cached pose is not replayed here.

BodyUpdate=0 left primary joints at base pose but still updated extra joint LowerChest. Active=false was checked for complete disabling. Changing BodyTracker selected another provider without bone remapping.

The test used a diagnostic skeleton: Spine under Hips, other nonroot bones under Root, all 87 names present. This checks transform processing, not a real anatomical hierarchy or model quality. Partially invalid parents were not established as a correct scenario; used chains must be consistent and fully valid.

When using role trackers

The built-in module contains profile /interaction_profiles/htc/vive_tracker_htcx requesting XR_HTCX_vive_tracker_interaction. Establish support on the target runtime/device separately. An old compatibility table is not a guarantee.

Example waist role: /user/vive_tracker_htcx/role/waist. In the action map, create a Pose action with this ToplevelPaths and binding /user/vive_tracker_htcx/role/waist/input/grip/pose in that profile. A foot uses a separate role, such as /user/vive_tracker_htcx/role/left_foot. Bind XREntity3D to the exact role name and configured pose name, then use it as an application IK target. Set Tracker before Pose. A Tracker inspector value alone is insufficient: map, data and validity checks are required.

Role names and coordinates do not define bone lengths, calibration, missing joint poses or body collision. Those need a separate solver, which is not implemented here.

Diagnosis and coverage

For an available tracker but static avatar check HasTrackingData, valid Root/parents, Bone/BodyUpdate and bone names. For an offset or stretched avatar check input space, AvatarRoot placement and MotionScale. For a retained old pose check tracking loss rather than relying on nonzero coordinates. Hands have a separate guide.

Nineteen checks covered synthetic XRBodyTracker state; fourteen more exercised a real Skeleton3D/XRBodyModifier3D with synthetic data. The exact BonePoseProbe ran in the engine. Initial JointFlags failure remains a negative result. No live provider, HTC role, IK, real Humanoid hierarchy, skin weights/deformation, animation blending, export or rendered image was checked.