Face-Rig Slice C (#891): NonRigidICP — native template→user fit - #899
Face-Rig Slice C (#891): NonRigidICP — native template→user fit#899fernandotonon wants to merge 1 commit into
Conversation
Slice C of epic #889. Pure-data, Ogre-free, ZERO new dependencies, headless-tested — the native C++ port of the NRICP the spike (#897) proved. - src/FaceRig/NonRigidICP.{h,cpp}: Amberg 2007 optimal-step non-rigid ICP. Fits a template mesh to a user neutral head of arbitrary topology, producing per-template-vertex positions on the user surface (the correspondence the deformation transfer #892 consumes). Per-vertex 3x4 affine A_i; minimizes data (A_i·v_i -> closest user-surface point) + annealed stiffness (alpha·(A_i - A_j) over template edges, 50->0.5). Self-contained: * median-split KD-tree over user-triangle centroids (broad phase), * Ericson point-to-triangle projection (exact closest point), * CSR sparse matrix + conjugate-gradient on the normal equations (AtA x = At b) — no Eigen, no scipy, nothing vendored. Reports per-vertex residual + mean/max + bbox diag so callers can gate on fit quality (the humanoid check in Slice E). Rigid centroid+bbox-scale pre-align, then the non-rigid warp. - 6 headless tests: fit-to-self near-identity, fit onto a translated target, **fit onto a DIFFERENT-topology target** (10x10 template -> 14x14 user, <5% residual, finite), stiffness-bounded under noise, degenerate input no-NaN, residual/diag reporting. All pass (~5-13 ms each). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
|
Important Review skippedAuto reviews are disabled on base/target branches other than the default branch. Please check the settings in the CodeRabbit UI or the ⚙️ Run configurationConfiguration used: defaults Review profile: CHILL Plan: Pro Run ID: You can disable this status message by setting the Use the checkbox below for a quick retry:
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ca318634b3
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
| const int cf = tree.nearest(X[i]); | ||
| // refine: check that triangle + a few neighbors would need the | ||
| // full tree; centroid-nearest triangle is a good approximation | ||
| // for a fitted template already close to the surface. | ||
| target[i] = closestPointTriangle(X[i], utri[cf][0], utri[cf][1], utri[cf][2]); |
There was a problem hiding this comment.
When the user mesh has non-uniform triangles, tree.nearest(X[i]) only minimizes distance to triangle centroids, not to the triangles themselves. A point can lie on a large triangle whose centroid is far away while a small nearby triangle/artifact has a closer centroid, so ICP snaps that correspondence to the wrong surface patch; the final residual pass uses the same heuristic, making the bad fit look clean. Please expand the broad phase to consider all triangles whose bounds can beat the current closest point, or use a triangle BVH before choosing the target.
Useful? React with 👍 / 👎.
- NonRigidICP: correspondence search now takes the K=4 nearest triangle CENTROIDS and picks by exact point-triangle distance — a single centroid winner mis-corresponds next to large/sliver triangles (PR #899). - DeformationTransfer: reject malformed buffers (trailing floats, indices outside [0,N)) before dereferencing; anchor the translation gauge of EVERY connected component, not just vertex 0 — the ICT template is dozens of islands (eyeballs, corneas, teeth) and each needs its own anchor row + rhs (PR #900). - FaceRigger: '--max-residual' now gates the MAX fit residual directly (it silently allowed 6x the supplied value); mean gated at a quarter of it. Healthy fits (max <= ~4%) pass the default 8% unchanged (PR #901). - FaceRigAttach::extractGeometry: skip a sharedVertexData pool no submesh references — orphan vertices joined the fit with no triangles (PR #901). - ArkitTemplate::ensureModelBlocking: a synchronous startDownload failure no longer blocks for the full 5-minute timeout ('done' guard, the LLM CLI pattern); Sentry breadcrumbs on download start/ok/fail (PR #898). - export-arkit-template.py: document that tongueOut is deliberately absent — ICT-FaceKit has no tongue expression, 51 real shapes (PR #898). - docs/FACE_RIG_SPIKE.md: escape |Δ| pipes that broke the results table (PR #897). Already addressed by earlier commits (noted for the record): the .arkit.json sidecar is now consumed on import (PR #903, commit e7abe01) and re-rigging replaces existing same-named targets instead of stacking (PR #902). FaceRig sources build into UnitTests via src/CMakeLists.txt — the 34 FaceRig/NRICP/DT tests run green. Verified: reference rig max residual 0.35% under the stricter gate; anchored Rumba sim 3.27%. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
|
Consolidated into #903 (single epic PR, retargeted to master). Review findings from this PR are addressed there — see commit d234722d. |
1 participant
Part of epic #889. Closes #891. Stacked on #898 (Slice B) — merge that first.
The hard slice: the native C++ non-rigid ICP the spike (#897) proved, fitting the ARKit template to an arbitrary user head to produce the correspondence deformation transfer (#892) needs.
What this adds
src/FaceRig/NonRigidICP.{h,cpp}— Amberg 2007 optimal-step NRICP, pure-data, Ogre-free, zero new dependencies:A_i·v_i→ closest user-surface point) + annealed stiffness (α·(A_i − A_j)over template edges, 50→0.5).Verification
6 headless tests pass (~5–13 ms each):
Matches the spike's proven behavior (which hit mean 0.003% on the real ICT↔decimated-head fit); real-mesh C++ validation lands in Slice E where it's wired to actual entities.
Next: Slice D (#892) — DeformationTransfer (the per-shape transfer using this correspondence).
🤖 Generated with Claude Code