Skip to content

Exporter: Vertex Animation Texture (VAT) — bake skeletal anim into RGB textures #371

Description

@fernandotonon

Overview

Add a Vertex Animation Texture (VAT) exporter so a static mesh + a small GPU shader can replay a skeletal animation without runtime skinning. This is the soft-body / per-vertex variant — works for any skinned animation and is what most VAT tutorials demonstrate.

Use cases: crowds, mobile/VR targets where bone palettes are expensive, sims where the deformation can't be expressed as a skin, drop-in playback in Godot/UE/Unity from a content pipeline.

Out of scope (separate issue if needed): rigid-body VAT (one quat+pivot per chunk) — a different baker.

Prerequisite

Block on Phase 5 slice B (#361) merging. The VAT baker reuses the same animation-stepping pattern the Blend baker introduced.

What we ship

A baker that, given an animated entity + animation name + frame range:

  1. Steps the entity's animation state in fixed-fps increments.
  2. After each step, calls entity->_updateAnimation() and reads the post-skinning vertex positions from entity->_getSkelAnimVertexData() / subEntity->_getSkelAnimVertexData().
  3. Encodes one row of the position texture per frame: pixel (vertexIndex, frameIndex) = (px, py, pz).
  4. Optionally: a normal texture (same layout) and a vertex-index UV channel baked into a duplicate of the source mesh.
  5. Writes a JSON sidecar describing the layout and bounds.

Encoding (selectable)

  • RGBA8 normalized — store (p − min) / (max − min) per axis; sidecar carries min/max. ~3 mm error on a 1 m model. Smallest, widest support.
  • RGBA16 normalized — same idea but 16-bit channels via PNG/TGA. ~50× more precise.
  • RGBA float (EXR) — store positions directly. Lossless. Larger files. Vendor TinyEXR (single header, MIT).

Target engine variant (selectable)

  • Engine-agnostic (default) — textures + JSON; docs explain the shader math.
  • Unity — Y-down UV, Z-up positions, separate position+normal textures, .meta sidecar.
  • Unreal — Niagara VAT module layout (vertex index in U, frame in V).
  • Godot — Godot shader template .gdshader snippet emitted alongside the textures.

The variant determines axis swizzle, UV orientation, and which sidecar files are emitted. Internally one baker function with a VATTarget enum.

File layout (per bake)

out/
  Run_pos.{png,exr}       # position texture (or _pos16.png / _pos.exr)
  Run_nrm.{png,exr}       # normal texture (optional)
  Run.json                # sidecar: frameCount, vertexCount, bounds, fps, target, encoding
  Run.meta                # only when target=unity
  Run.gdshader            # only when target=godot
  Run_mesh.{fbx,gltf}     # mesh with vertex-index UV channel

Files to add

  • src/VATBaker.h/.cpp — pure-data baker. Static bake(entity, animName, fps, frameRange, encoding, target, outDir, *errorMsg) returning VATBakeResult { posTex, nrmTex, meshPath, jsonPath, ok, error }. No QObject — easier to unit-test.
  • src/VATBaker_test.cpp — pure-data tests (encoding round-trip, JSON sidecar shape) + Ogre-fixture tests (sample vs. live-skinned position match within tolerance).
  • src/dependencies/tinyexr/ — vendor TinyEXR (single header, MIT).

Files to modify

  • src/CMakeLists.txt — add VATBaker.cpp + .h. Add TinyEXR include path.
  • tests/CMakeLists.txt — add VATBaker.cpp to the MaterialEditorQML_* test targets' duplicated source list (slice-A pattern).
  • src/CLIPipeline.cpp — new cmdVat() subcommand, wired in run() next to cmdAnim(). CLI: qtmesh vat model.fbx --anim Walk --fps 30 --encoding rgba8|rgba16|exr --target agnostic|unity|unreal|godot -o out/.
  • src/MCPServer.cpp — register bake_vat tool in toolHandlers(). Implement toolBakeVat() mirroring the CLI args.
  • src/MeshImporterExporter.cpp — re-export the source mesh with a vertex-index UV channel so the runtime shader can index into the position texture.
  • qml/PropertiesPanel.qml — new collapsible "Bake VAT" subgroup in the Animations section, mirroring the Blend subgroup.
  • src/PropertiesPanelController.cppQ_INVOKABLE bakeVAT(...) that dispatches to VATBaker::bake() on a QThread and emits vatBakeProgress / vatBakeFinished signals.
  • Sentry breadcrumbs: "file.export" from the baker, "ui.action" from the QML button.

Reuse map

  • Animation sampling: NormalVisualizer::updateAnimatedOverlays() (src/NormalVisualizer.cpp:249) — copy its entity->_updateAnimation() + entity->_getSkelAnimVertexData() pattern. The blender's bake() reads bone TRS, which is the wrong abstraction for VAT — we need post-skin positions.
  • Vertex iteration: EditableMesh::readVertexData() (src/EditableMesh.cpp:1486).
  • Image writing: 8/16-bit PNG via QImage::save(); EXR via TinyEXR.
  • CLI subcommand template: CLIPipeline::cmdAnim() (src/CLIPipeline.cpp:1281).
  • MCP tool registration: static map in MCPServer::toolHandlers() (src/MCPServer.cpp:385).
  • Async progress: BatchExporter::execute() (src/BatchExporter.cpp:13) emits progressChanged(int, int) — same shape we want.
  • QML pattern: Blend "Bake" button in qml/PropertiesPanel.qml (slice B).

Slices (1 PR each)

Slice 1 — Pure-data baker, CLI, RGBA8 only

  • VATBaker::bake() for engine-agnostic + RGBA8 + position only.
  • CLI qtmesh vat.
  • JSON sidecar.
  • Tests: encode/decode round-trip; sample vs. live position match within 4 mm at RGBA8.
  • No QML, no MCP yet. ~600 LOC.

Slice 2 — RGBA16 + EXR + normals

  • Vendor TinyEXR.
  • Add encoding and bakeNormals params.
  • Tests: precision matrix (RGBA8 vs. RGBA16 vs. EXR error bounds).

Slice 3 — Unity / Unreal / Godot variants

  • Per-target axis swizzle + UV orientation + sidecar emission.
  • A small media/vat/templates/ dir with the Godot .gdshader and Unity/Unreal docs.
  • Tests: per-target sidecar contents (snapshot tests against fixed input).

Slice 4 — Inspector UI + MCP tool + async pipeline

  • New VATBakerController (QObject + thread).
  • "Bake VAT" QML subgroup.
  • bake_vat MCP tool.
  • Sentry breadcrumbs.

Cross-cutting

Acceptance criteria

  • qtmesh vat <mesh> --anim <name> produces a position texture + JSON sidecar
  • All three encodings (rgba8 / rgba16 / exr) round-trip vertex positions within their stated tolerance
  • All four targets (agnostic / unity / unreal / godot) produce the right axis convention + sidecar files
  • Inspector "Bake VAT" subgroup runs the bake on a worker thread with a progress bar
  • MCP bake_vat tool exposes the same parameters
  • Loaded into a Godot test scene, the baked mesh plays the animation indistinguishably from the original at RGBA16 or higher
  • All new code covered by tests (pure-data + Ogre-fixture); SonarCloud quality gate stays passing

Metadata

Metadata

Assignees

No one assigned

    Labels

    enhancementNew feature or request

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions