Godot XR Animation Recorder

Godot XR addon to facilitate recording of XR tracker data and creating an animation file.

Versions

Official releases are tagged and can be found here.

The following branches are in active development:

Overview

Godot XR Animation Recorder provides a set of nodes and scripts suitable for recording motion capture from XR tracker sources such as:

• Godot OpenXR
• Godot XR VMC Tracker
• Godot XR Axis Studio Tracker
• Godot XR MVN Tracker
• Godot XR Rokoko Tracker

These XR tracker sources provide different types of XR trackers:

• XRBodyTracker for tracking body movements (and sometimes hands)
• XRFaceTracker for tracking face movements
• XRHandTracker for tracking hand movements

Tracking data can be recorded, and then recombined (possibly including time-shifting) to create a Godot Animation file suitable for character animation.

Usage

The following steps show how to add the Godot XR Animation Recorder to a project.

Tracking Avatar

The tracking avatar shows the live XR tracking data. It's not strictly necessary, but can be useful to verify the XR tracking systems are functional. The tracker avatar should be driven by the same XR trackers intended for recording:

• XRBodyModifier3D driven by body tracker (E.G. /vmc/body_tracker for VMC)
• XRFaceModifier3D driven by face tracker (E.G. /vmc/face_tracker for VMC)
• XRHandModifier3Ds driven by hand trackers (E.G. /user/hand_tracker/left for OpenXR)

Tracker Recorder

A GodotXRTrackerRecorder node should be constructed to record the XR tracker sources.

Trackers may be left blank if not available - for example the hands may be skipped of the body-tracker provides hand-tracking.

Tracking data can be recorded using the start_recording() and stop_recording() methods. The tracker recordings are available after the recording finishes:

The recordings are Godot Resources and as such can be loaded and saved using the ResourceLoader.load() and ResourceSaver.save() methods.

Audio Recorder

A GodotXRAudioRecorder node may be constructed to record audio synchronous with the tracker performance.

Audio can be recorded using the start_recording() and stop_recording() methods. The GodotXRAudioRecorder.audio recording is available after the recording finishes.

The audio is a Godot AudioStreamWAV resource, and as such can be loaded and saved using the ResourceLoader.load() and ResourceSaver.save() methods.

Tracker Player

Once XR tracker recordings have been created using the Tracker Recorder they can be played into the Mixing Avatar using a GodotXRTrackerPlayer. Selecting different recordings allows for mixing new face and audio recordings on top of a previously recorded body performance.

Before starting playback, the recordings may be specified using the following fields:

Playback is performed using the play() and stop() methods.

When playing, the tracker will stream the tracking data out of new virtual trackers. The following are the default tracker names:

Mixing Avatar

The mixing avatar is an avatar driven by the trackers generated by the Tracker Player. It should be a close match to the final avatar the animation is targeting - specifically the face MeshInstance3D and the facial blend-shapes should be identical. It is also recommended to limit the avatars facial blend-shapes to the standard ARKit 52 blend-shapes. This will force the XRFaceModifier3D driving node to target ARKit animations.

Animation Recorder

A GodotXRAnimationRecorder node is used to record animations from the Mixing Avatar.

The start_recording() and stop_recording() methods control animation recording. Once complete the animation is available through the animation field.

The animation is a Godot Animation resource, and as such can be loaded and saved using the ResourceLoader.load() and ResourceSaver.save() methods.

Until optimization is implemented (when the Animation.optimize method is exposed) it is recommended that users manually perform the animation optimization on the saved resources using the Animation Edit/Optimize option.