Mocap Slice D (#873): MocapRecorder + CLI 'qtmesh mocap --face' + MCP capture_face_from_video - #882
Conversation
…ace_from_video Slice D of epic #869 — the first user-visible milestone: video file -> animated mesh, end-to-end. - MocapRecorder (src/Mocap/): FaceSample stream -> morph weight keyframes on a named clip via the #519 pipeline (new MorphAnimationManager:: writeWeightKeyOn public static — the entity-explicit core of setMorphWeightKeyframe, which now delegates to it), with epsilon run-length suppression, first/last anchoring, jump pre-anchors and >0.5s face-lost gaps held at both edges. Head pose is calibrated on the take's first confident frame and keyed as rotation deltas on the Head bone (canonicalIndexForBone role 5) in '<clip>_Head', or as node-TRS deltas via NodeAnimationManager for static meshes. - RecordMocapClipCommand (src/commands/): one undo step per take — first redo snapshots the pre-existing weight/head clips keyframe-for-keyframe, undo restores them exactly (verified by test). - CLI 'qtmesh mocap' (src/Mocap/MocapCLI.cpp, the SceneLightsCLI pattern): import mesh -> mapping table (matched/unmatched printed, never silently dropped) -> FileFrameSource or --frames-dir image sequence -> FaceCapPredictor -> One-Euro -> recordFace -> optional re-export; --json emits the FaceRecordReport. Non-MOCAP builds print the standard rebuild hint. Registered in the dispatcher, subcommand list and usage text. - MCP capture_face_from_video: heavy tool, live-scene entity (selected or entity_name), single undoable clip, optional output_path export, report JSON; clean error on non-MOCAP builds. - Fixes a latent CLI bug: writeCliError text was silently lost at _exit() (unflushed static QTextStream) — every subcommand's stderr errors printed nothing; now flushed. - Sentry ai.assist.mocap_face breadcrumbs; gamification noteOperation("mocap_face", {frames, keyframes}) on CLI + MCP. Verified end-to-end on macOS with the Slice A models: a 6-frame image sequence (portrait -> rotated portrait) onto (a) a static OBJ — 5 head keys on the node path, glb exported — and (b) a minimal glTF with jawOpen/ mouthSmileLeft targets — 2 channels matched, 9 weight keys, exported glb carries the 'FaceCap' morph-weights animation (path:"weights"). Known upstream gap noted: the Assimp glTF exporter doesn't emit extras.targetNames, so reimported targets alias to Shape_N. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5c5bbf9d12
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const QString clip = options.clipName + QStringLiteral("_Head"); | ||
| if (nam) { | ||
| if (options.replaceExisting) | ||
| nam->deleteClip(clip); // no-op when absent |
There was a problem hiding this comment.
When recording head pose for a static mesh and a scene-level node animation named <clipName>_Head already exists (for example, a user-authored FaceCap_Head clip), this deletes that clip before creating the mocap clip. Unlike the mesh and skeletal clips, RecordMocapClipCommand never snapshots scene-level node clips and its undo path only deletes the newly recorded clip, so undoing the single mocap step permanently loses the user's pre-existing node animation. Please snapshot/restore the node clip or reject the name collision before deletion.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
Fixed in #909: recordFace no longer deletes a pre-existing scene-level node clip. It rejects the name collision (new report.headError, surfaced in CLI text + JSON) so undo can never lose a user clip it didn't snapshot; a fresh clip name proceeds normally.
|
Consolidated into #909 (single epic PR targeting master). Review findings from this PR are addressed there — see the latest commit. |
1 participant
Part of epic #869. Closes #873. Stacked on #881 (Slice C) — merge order: #880 → #881 → this.
What this adds
The first user-visible milestone: one command turns a face video into a playing, exportable morph animation.
MocapRecorder—FaceSamplestream → weight keyframes on a named morph clip through the Anim: Slice B — Mesh / vertex animation (Alembic + Ogre VAT_POSE clips) #519 pipeline (dope sheet / timeline / glTF export all work downstream for free). Epsilon run-length suppression (constant channels → 2 keys), jump pre-anchoring, first/last always keyed, face-lost gaps > 0.5 s held at both edges. Head pose neutral-calibrates on the first confident frame and lands on the Head bone (<clip>_Headskeletal clip,canonicalIndexForBonerole) or as node TRS for static meshes.MorphAnimationManager::writeWeightKeyOn— small refactor exposing the entity-explicit keyframe writer (setMorphWeightKeyframenow delegates to it); the shared-track pose-reference subtlety stays in one place.RecordMocapClipCommand— a whole take is ONE undo step (snapshot pattern; pre-existing clips restored keyframe-for-keyframe, covered by test).qtmesh mocap(separateMocapCLI.cpp, the SceneLightsCLI precedent): mapping table printed (unmatched channels reported, never dropped),--frames-dirheadless path,--jsonreport, clear errors for every failure mode, "rebuild with -DENABLE_MOCAP" on non-mocap builds.capture_face_from_video(heavy): live-scene entity, single undoable clip, optional export, full report JSON.CLIPipeline::writeCliErroroutput was silently lost at_exit()(unflushed static QTextStream) — affected every existing subcommand's error text.Verification (run locally on macOS with the Slice A models)
framesProcessed 6, framesNoFace 0, headTarget "node", 5 head keys, valid glb exported.jawOpen/mouthSmileLefttargets: 2 channels matched, 9 weight keys, exported glb contains theFaceCapmorph-weights animation ("path":"weights"— the Anim: Slice B — Mesh / vertex animation (Alembic + Ogre VAT_POSE clips) #519 round-trip).MocapRecorder_test.cppsuite (keyframe times/values, epsilon suppression, gap holds, unmatched reporting, single-undo round-trip, head-target resolution) needs Ogre → runs in the Linux CI lane.extras.targetNames, so a REimported mesh showsShape_Ntarget names — a--mapsidecar re-binds them.Real-webcam-video + ARKit-avatar verification is the Slice F/G QA pass (needs a hosted model + a Ready Player Me head; the pipeline pieces are each proven above).
🤖 Generated with Claude Code