OpenXR spatial entities: anchors, planes and markers

OpenXR spatial entities connect an application to a real room: anchors retain location bindings, planes describe surfaces, and markers describe detected codes. These are XR runtime data, not game-tree Entities or an ECS. Create an ordinary Entity bound to a tracker to display them.

Enable the required capability

First configure OpenXR. You need a running session, the corresponding spatial extensions and target-device permissions. Project settings request capabilities; they cannot add hardware support. Obtaining planes does not automatically enable passthrough, physics collisions or room scanning.

The keys below share the prefix xr/openxr/extensions/spatial_entity/. All are disabled by default; configure extension requests before OpenXR starts.

The last setting is a bitmask: QR=1, Micro QR=2, ArUco=4, AprilTag=8; for example 1 requests built-in QR handling and 15 requests all four. The runtime need not support every requested type. Select the matching marker-tracking dictionary for ArUco/AprilTag. Do not use the obsolete standalone enable_builtin_marker_tracking switch.

Start with built-in discovery for ordinary applications. Custom spatial contexts are for requirements that its query loop cannot meet; avoid two competing owners of the same discovery logic.

Bind trackers to the tree

The runtime registers OpenXRAnchorTracker, OpenXRPlaneTracker and OpenXRMarkerTracker in XRServer. All use the Anchor category and pose default. An ordinary XRPositionalTracker in that category is not necessarily a spatial tracker.

Add an Entity with this script directly below XROrigin3D. It creates empty XRAnchor3D siblings directly below the same origin. In OnChanged, after creating view, attach suitable content from your .rprefab, selected separately for anchors, planes or markers. The example itself creates no visible geometry and starts no discovery.

XROrigin3D
├── XRCamera3D
├── SpatialVisuals
└── XRAnchor3D (created at runtime)
    └── Your content

using System;
using System.Collections.Generic;
using Reckfell;

public partial class SpatialVisuals : Entity
{
    private XROrigin3D _origin = null!;
    private readonly Dictionary<StringName, (OpenXRSpatialEntityTracker Tracker, XRAnchor3D View)> _views = new();
    public int ViewCount => _views.Count;

    public static bool IsTracked(OpenXRSpatialEntityTracker tracker)
    {
        var pose = tracker.GetPose("default");
        return tracker.SpatialTrackingState == OpenXRSpatialEntityTracker.EntityTrackingState.Tracking
            && pose != null && pose.HasTrackingData;
    }

    public override void _EnterTree()
    {
        _origin = GetParent() as XROrigin3D
            ?? throw new InvalidOperationException("SpatialVisuals must be a direct child of XROrigin3D.");
        XRServer.TrackerAdded += OnChanged;
        XRServer.TrackerUpdated += OnChanged;
        XRServer.TrackerRemoved += OnRemoved;
        using var trackers = XRServer.GetTrackers((int)XRServer.TrackerType.Anchor);
        foreach (var key in trackers.Keys) OnChanged(key.AsStringName(), (long)XRServer.TrackerType.Anchor);
    }

    private void OnChanged(StringName name, long type)
    {
        OnRemoved(name, type);
        if (XRServer.GetTracker(name) is not OpenXRSpatialEntityTracker tracker) return;
        var view = new XRAnchor3D { Tracker = name, Pose = "default", Visible = false };
        _views.Add(name, (tracker, view));
        _origin.AddChild(view);
    }

    private void OnRemoved(StringName name, long type)
    {
        if (_views.Remove(name, out var item)) { item.View.Visible = false; item.View.QueueFree(); }
    }

    public override void _Process(double delta)
    {
        foreach (var item in _views.Values) item.View.Visible = IsTracked(item.Tracker);
    }

    public override void _ExitTree()
    {
        XRServer.TrackerAdded -= OnChanged;
        XRServer.TrackerUpdated -= OnChanged;
        XRServer.TrackerRemoved -= OnRemoved;
        foreach (var item in _views.Values) { item.View.Visible = false; item.View.QueueFree(); }
        _views.Clear();
    }
}

The manager subscribes before reading already registered trackers, covering objects found earlier. TrackerUpdated replaces an existing name binding. TrackerRemoved deletes only its owned view without another GetTracker call: the registry entry can still exist during that signal.

XRServer, trackers and their returned poses are borrowed here: do not Dispose/Free them. The manager owns only its created XRAnchor3D instances. QueueFree defers destruction safely while the old view is hidden immediately. Do not replace it with Free in _ExitTree: the parent may still be changing its children. This script runs on the main thread.

A view is shown only with SpatialTrackingState=Tracking and pose HasTrackingData. Paused/Stopped or invalid poses hide it instead of leaving stale content visible. Visible disables neither collision shapes, audio, input nor gameplay: disable these separately on tracking loss. For diagnostics expose ViewCount and tracker state; zero does not by itself establish missing support.

Create and persist an anchor

Use the registered OpenXRSpatialAnchorCapability, checking existence and IsSpatialAnchorSupported after session startup. Do not construct a new capability to query support. Do not free the engine-owned object.

1. Call CreateNewAnchor and check for null. Supply an OpenXR play-space pose. Reverse WorldScale/reference-frame conversion remains TODO in current source: do not pass an arbitrary GlobalTransform. Limit the simplest case to WorldScale=1 and identity reference frame/world origin; other conversions require separate verification.
2. Wait for SpatialTrackingState=Tracking and a valid pose. Receiving a tracker alone does not establish stable localization.
3. Check IsSpatialPersistenceSupported and a ready persistence context. Built-in handling uses its default context; creating your own context is asynchronous.
4. Call PersistAnchor, retain its future and wait for completion. Null means the request did not start. Check the finished request GetResultValue and HasUuid; only then save UUID together with your content identifier and application data.
5. On later discovery, match UUID to your saved record. Do not store tracker Name or RID as persistence keys: they identify the current session. A UUID in your JSON does not mean the device has restored its anchor.

LocalAnchors persistence scope is for the same app, user and device; it promises no cross-device transfer. SystemManaged provides access to system-persisted entities, not a mutable application store. Handle app deletion, data reset, changed rooms and localization loss separately.

To delete a persistent anchor, first UnpersistAnchor; wait for a successful result and UUID removal, then RemoveAnchor. Unpersist does not automatically remove the tracker. RemoveAnchor rejects an anchor that is still persistent. Freeing a visual Entity alone does not remove its stored spatial binding.

A future is neither a Task nor the operation result. GetStatus distinguishes Running, Finished and Cancelled; finished operations can fail. Current C# SDK Completed declares an OpenXRFutureResult parameter whereas native emission supplies result_value, such as bool or RID: do not assume that typed event correctly delivers such results. Check GetStatus/GetResultValue and operation state in your integration; real-device C# async delivery was not tested here. Do not issue persistence requests every frame or interpret a timeout as proof of unsupported hardware.

Planes and markers

For OpenXRPlaneTracker, listen to MeshChanged and update MeshInstance3D.Mesh and CollisionShape3D.Shape from GetMesh/GetShape. Apply GetMeshOffset to a shared child Entity3D containing visual and physical geometry: the offset relative to anchor pose can be nonidentity. Built-in geometry lies in local XY, so do not add an unconditional 90-degree rotation. Positive bounds permit a rectangular fallback; missing geometry/bounds can return null, requiring removal of stale mesh/collider. Mesh/shape wrappers are tracker-cached and borrowed. Do not expect another GetShape call to alter an already cached shape for different thickness parameters.

MeshChanged does not report every offset update: current SetMeshData with unchanged vertices/indices and a different origin changes GetMeshOffset without an event. While tracking is valid, refresh the geometry-child transform from GetMeshOffset each frame; keep mesh/shape rebuilding in the MeshChanged handler. This case was reproduced separately; engine source was not repaired.

Detection can report an earlier room scan rather than a current complete mesh. PlaneLabel and alignment require supplied data; an unknown label does not mean floor. A detected plane is no physics wall until you assign a shape to your StaticBody3D. Passthrough occlusion needs separate material and headset transparency/composition verification; an ordinary black material is no proof of correct virtual-object occlusion.

Check OpenXRMarkerTracker.MarkerType first. QR/Micro QR GetMarkerData can contain string or byte[]; explicitly handle an unknown data type. Match ArUco/AprilTag by type, selected dictionary and MarkerId rather than an assumed string. Do not treat QR content as an arbitrary resource path: map allowed application identifiers to known assets.

Custom backend access

OpenXRSpatialEntityExtension exposes SupportsCapability/SupportsComponentType, contexts, discovery/update snapshots and component reads. Each used capability needs actual support; a C# type existing is insufficient.

• CreateSpatialContext completes asynchronously with a context RID; check success and GetSpatialContextReady before queries.
• OpenXRSpatialQueryResultData must be first in component-data arrays; select other components using actual runtime support.
• Direct DiscoverSpatialEntities returns a caller-owned snapshot RID. Finish reading/copying buffers, then FreeSpatialSnapshot; update snapshots follow the same rule.
• Capability StartEntityDiscovery reads its snapshot and frees it after the user callback. Do not retain that RID or free it again; copy needed data inside the callback.
• A tracker owns its assigned spatial entity RID; SetEntity can free its predecessor. Do not FreeSpatialEntity for built-in tracker RIDs. Free your own contexts only after stopping requests and releasing dependents; do not free engine-owned default contexts.

Troubleshooting

Verification limits

Installed Windows Runtime 1.0.0 executed twenty headless checks of this exact script: synthetic tracker resources through real XRServer, existing/new registrations, replacement without duplicates, pose/state gating, hiding and deferred cleanup, bounded-plane local geometry and marker data. This verifies client logic, not real room discovery. Real headsets, anchor creation/restoration, async persistence, permissions, scanning, physical collisions, passthrough pixels and export remain untested. Engine Source, Runtime and SDK unchanged.