[WIP] Mortar-based periodic boundary conditions#62
Draft
rcarson3 wants to merge 52 commits into
Draft
Conversation
Probably should have captured more of the intermediate updates through-out this journey of "vibe-coding"... However, I used Claude Opus 4.7 and Claude's project / github integration with its web chat to drive an initial implementation of a mortar based periodic boundary conditions. I initially had it work things out in pyMFEM and create a plan to build things up from 2D serial to parallel to creating a ton of tests to then moving onto 3D with more tests. Some of these tests involved multi-material elastic simulations to break up in homogeneity and to actually test the methods. Outside of the tests here, I did validate what it was doing with visualizations making sure the solutions at each stage of the way made sense and if not would iterate with Claude until we got something decent. After the python implementation looked more or less good enough, we moved over to C++. Here, I had it work on the initial port and verify that things matched up with the Python version. After we did that, I had it move on to a more scaleable set-up and assembly phase for the constraint matrix as it was initially doing everything on a single rank or broadcasting everything to all ranks. Once that seemed like in a good state, we moved onto an element assembly formulation as that seemed like a good potential path to get us on the GPU and that was verified against the existing sparse matrix / hypre implementation. An initial stab at GPU support also happened. I know this will need to be expanded on. Finally, I had it add support for 3D non-conforming mesh faces. It needed Axom's BVH and similar spatial search algorithms for this which is why we know support Axom. The solutions there seem to line up with the expected results and when the meshes and as the mesh gets closer to conformal the distortional displacement required in the solution approached 0 and matched the conformal variation which was a good indication that the method was doing what we would expect. Also, this work added a ton of new tests which is good but yeah there are a ton of new tests to work through...
For nonconformal tests added the checkerboard and stripe test. Outside that relaxed the F_tol test condition so the tests would pass. Then the mortar saddle point API was updated to make it more useable for when ExaConstit would havee to use it.
Claude created a manager that controls all of the fun-ness that we'll need to manage for the Mortar PBCs in both the SystemDriver and elsewhere.
…PBCs and add test
…t matrix Had Claude add some functionality to create the boundary sub-mesh, add a simple function to compute the current deformation gradient, and then finally some other things to get out the Fbar and Fdot_bar terms.
… fact The mortar method should now have several different capabilities to check the fluctuation field or the hill-mandel condition and a number of other things kinda related to this stuff.
…thod into SystemDriver and nonlinear solvers This was over a couple different iterations with Claude but the above has most of the framework in-place to where we should be able to in the very near term test the new mortar-based periodic boundary conditions on real problems.
After lots of debugging finally got a working example with periodic boundary conditions when using ExaConstit directly. Initial tests are pretty much just an isotropic elastic examples using ExaCMech but it's a start. The good things is we do see converged results which is super awesome.
…ic boundary conditions Had Claude add a number of useful validation / diagnostic type fields as part of the periodic boundary conditions. Also had it add all of the affine and fluctuation fields for the velocity field so we can see how the PBCs have driven things to be different.
Pushing Claude to move towards "semi-periodic" BCSs so we don't end up with overly constrained systems and we can relax the constraint matrix to allow for more natural relaxation of faces for like in monotonic deformation modes
So, I found in testing that the default method of clamping all of the corner DOFs to be way to restrictive and wouldn't really allow me to do like monotonic tension tests easily. Therefore, I had Claude drive things so that we could have what is like semi-periodic BCs by allowing the constraint matrix to be set up based on which DOFs you actually want. Overall, the whole thing works and we now have something that should be useable by the broader community.
…ewton/Krylov wiring
Add the Phase 5.11 saddle-residual scaling stack for mortar PBC solves,
including the residual scaler, scaled saddle/Jacobian/preconditioner
wrappers, trust-region dogleg support for scaled coordinates, and
per-Newton-iteration diagnostic logging.
This change also wires the scaling path through SystemDriver and the
mortar manager, adds sub-block partition support for lambda rows, and
extends the options path to configure saddle scaling from TOML.
While debugging the new path, fix several solver-state and lifecycle
issues that could make first-step behavior nondeterministic or unsafe:
- zero Newton/Krylov correction buffers before inner solves so the
linear solver never consumes stale data as an initial guess
- propagate iterative_mode through wrapped solvers so zero-initial-guess
semantics survive the wrapper stack
- remove a duplicate physical residual evaluation in the scaling
pre-attempt path; the residual callback is stateful and must not be
probed twice before Newton starts
- refresh scaled wrapper state, TRDOG offsets, and diagnostic wiring
after periodic-BC spec changes that resize the lambda block
- remove the temporary inspecting iterative solver used during
root-cause analysis once the underlying initialization bug was fixed
At a high level this commit introduces the new saddle scaling
infrastructure, preserves diagnostic visibility for the scaled solve,
and fixes the wrapper/solver initialization bugs that were causing
inconsistent cycle-1 residual behavior and occasional MINRES NaNs.
…aling checks and fix a cmake issue
Add SimulationState accessors for boundary-submesh FE spaces and LOR-refined boundary submeshes, including lor_depth=1 aliasing and lor_depth>1 fresh refinement to avoid mutating the direct boundary cache. Update mesh option validation for Phase 6 LOR combinations and add a focused mortar_pbc test covering accessor aliasing, depth-2 refinement behavior, surface FES properties, and validation rules.
Add a Phase 6 BoundaryClassifier3D constructor that consumes a pre-built boundary ParSubMesh and surface H1(P1) vector FE space. The new path reads coordinates, attributes, and TDOFs directly from the boundary submesh/FES, while preserving the existing parent-mesh constructor for legacy callers. Extend the boundary classifier test with parity coverage against the legacy constructor and rerun the classifier plus LOR boundary-submesh tests.
Introduce SurfaceProjector as the setup-time mapping between parent volume true DOFs and boundary-submesh true DOFs for the unified Phase 6 mortar PBC architecture. The projector builds snapped-coordinate maps from low-order boundary/LOR vertices to coincident parent boundary Lagrange nodes and exposes both direct lookup tables and diagnostic Mult/MultTranspose operations. Add focused mortar_pbc tests for p=1 direct projection and p=2 once-refined LOR projection, including constant and linear field reproduction, local/global map consistency, and transpose scatter-add behavior. Register the projector in the production library and add the new test target to the mortar PBC test suite.
Extend MortarConstraintOperator with a Phase 6 constructor that accepts a boundary-submesh classifier, SurfaceProjector, and parent volume FES. The operator now translates classifier-side submesh true DOFs to parent-FES true DOFs during setup, so import/export topology and flat row arrays are built against the runtime parent vector space while Mult and MultTranspose remain unchanged in the Krylov hot path. Preserve the legacy parent-classifier constructor as an identity projection path and share the setup implementation through Initialize(). Document the parent-FES indexing contract, projector-mediated setup semantics, and ownership/lifetime behavior in the operator header. Add a focused direct-path regression test comparing the projector-backed operator against the legacy operator at p=1 / lor_depth=1, accounting for submesh row enumeration order while still verifying Mult values and MultTranspose parent-column accumulation.
Extend ConstraintBuilder3D with a Phase 6 constructor that accepts a boundary/LOR-submesh classifier, SurfaceProjector, and parent volume FES. The builder now translates classifier-side submesh true DOFs into parent volume true DOFs when emitting constraint matrix columns, keeping the assembled HypreParMatrix path aligned with the projector-aware MortarConstraintOperator. Preserve the legacy constructor as an identity/direct path and update the builder documentation to describe the parent-FES column-space contract, projector ownership, and which APIs are affected. Build() now sizes the replicated matrix by the parent global true-vector size, and BuildHypreParMatrix() uses the parent FES true-DOF partition for its column starts when projector construction is used. Add a focused p=1 / lor_depth=1 regression test that builds the projector-backed submesh classifier path and verifies its assembled matrix action matches the legacy parent-classifier builder up to row permutation, while also checking the projected HypreParMatrix uses the parent FES column count.
Migrate MortarPbcManager construction to the Phase 6 boundary/LOR pipeline. The manager now builds its classifier on the LOR boundary submesh, constructs a SurfaceProjector back to the parent volume FE space, and wires the projector-aware constraint builder and EA constraint operator into the saddle system. Add projector-aware corner-pinning helpers so both default and spec-filtered corner essential TDOF lists are returned in parent-FES local true-DOF numbering. This keeps mechanics Dirichlet application, constraint columns, and C^T residual contributions in the parent solution space while allowing mortar topology to live on the LOR surface. Extend the manager corner-pinning tests with direct-path comparisons between the legacy parent classifier and the new boundary-submesh plus SurfaceProjector path, including full XYZ and X-only filtered specs. Document the Phase 6 ownership and index-space contracts in the manager header and implementation.
Move MortarSaddlePointSystem and MortarSaddlePreconditioner to store shared constraint-operator handles instead of raw references. This matches the Phase 6 manager ownership model, where the projector-aware MortarConstraintOperator is owned behind a shared_ptr and passed through the saddle solve stack. Keep reference-based compatibility constructors by wrapping references in non-owning shared_ptr aliases, so existing tests and legacy call sites continue to work during the migration. Update MortarPbcManager to pass its shared operator handle directly into MortarSaddlePointSystem. Add focused tests that construct the saddle system and saddle preconditioner from shared operator handles, reset the operator filter, and verify Refresh/SetOperator observe the updated lambda-block size. Also make the clipped triangular face-mortar conforming check compare sparse rows algebraically by row/column value rather than requiring identical CSR insertion structure. Verify by building and running the full mortar_pbc test suite; all 30 registered mortar_pbc tests pass.
Replace the arithmetic reconstruction of parent y/z true DOFs from an
x-component true DOF in MortarConstraintOperator. The previous logic
assumed a component-block global numbering layout, which is not valid
for the byNODES vector spaces used by the mechanics FES. Under MPI this
could make the EA import/export topology ask a peer rank for a component
true DOF that the peer did not own.
Cache the actual parent-side {x,y,z} true DOF tuple during operator
initialization by translating BoundaryClassifier3D component tuples
through SurfaceProjector. Use that cache when packing imported values
and accumulating transpose contributions, preserving the node-level
ownership invariant without assuming a global numbering formula.
Extend mortar constraint operator coverage for MPI by making the small
mesh construction test tolerate ranks with zero local lambda rows, by
including Hypre offd columns in the MPI Schur diagonal comparison, and
by running the P2 tet LOR affine constraint smoke test successfully at
4 and 7 ranks.
Verified with the full registered mortar_pbc CTest suite, the
MortarConstraintOperator executable under flux at np=4 and np=7, and
the reported 4-rank voce_ea mechanics reproducer past the previous
peer-request assertion.
Add Phase 6.1.D coverage for component-restricted periodic constraints on the projected P2 tetrahedral LOR path. The new operator test builds a P2 tet parent space, a once-refined linear boundary submesh, the submesh classifier, SurfaceProjector, projected ConstraintBuilder3D, and projected MortarConstraintOperator, then resets the operator to an X-only active spec. Verify that the filtered operator height matches the filtered builder row count, that row-factor metadata and RHS sizing match the reset operator, that all emitted rows are x-component rows, and that the manufactured affine field still satisfies the filtered constraint RHS. Run the same executable at np=1, np=4, and np=7 to cover the MPI import/export path under the reduced filter. Document the higher-order component-filter invariant in MortarConstraintOperator: projector-aware Reset keeps the parent-FES domain unchanged, reduces the lambda block to the filtered rows, and must preserve affine reproduction against filtered EmitRowFactors metadata. Verify by building all mortar_pbc targets and running the full registered mortar_pbc CTest suite; all 30 tests pass.
Add AMGF solver option parsing and validation gates Introduce AMGF and AMGF_AUG_LAGRANGIAN as linear preconditioner choices and parse the companion AMGF options: - amgf_gamma - amgf_subspace_executor Reject AMGF configurations that cannot work with the current implementation: AMGF requires FULL assembly and CPU/OPENMP runtime because it needs assembled HypreParMatrix operators for BoomerAMG and the filtered subspace solve. This check runs before the existing GPU/EA/PA logic can silently rewrite the preconditioner to Jacobi. Document the new options in src/options.toml and add test_amgf_options to verify: - AMGF keyword parsing - AMGF_AUG_LAGRANGIAN keyword parsing - AMGF option TOML parsing - rejection of EA/PA/GPU AMGF configurations without silent Jacobi fallback - rejection of invalid subspace executor strings Tests: - ctest -R test_amgf_options --output-on-failure
Add MortarConstraintOperator::GetConstraintCoupledDofIndices(), returning the sorted unique global displacement true DOFs that appear as nonzero columns in the active mortar constraint operator C. Build the index set lazily from the same filter-aware flat arrays used by Mult and MultTranspose. This keeps the AMGF index set consistent with: - active pair-label filters - component masks - projector/LOR parent-space TDOF translation - off-rank mortar imports - sentinel rows/components Invalidate the cache in Reset() so sub-XYZ periodicity changes rebuild the index set on demand. Update test_mortar_constraint_operator to compare the accessor against the reference HypreParMatrix nonzero-column set for both full and filtered constraint specs. This avoids stale geometric assumptions about higher-order legacy classifiers; higher-order coverage remains through the existing LOR projector path. Tests: - ctest -R test_mortar_constraint_operator --output-on-failure
Add exaconstit::amgf::BuildBooleanRestrictionProlongation(), which converts The utility collectively unions rank-local coupled-DOF views before assigning compact AMGF columns. This is required because a rank-local constraint row can touch off-rank mortar-side displacement columns, while the Boolean column must be owned exactly once by the rank that owns the corresponding K row. Register the new utility in src/CMakeLists.txt and add test_amgf_utils to verify: - P has the expected Boolean Mult and MultTranspose behavior - P^T A P extracts the expected principal submatrix for a diagonal test matrix SurfaceProjector was reviewed for reuse. It remains the correct LOR bridge from classifier/submesh TDOFs to parent-volume TDOFs, and the new accessor benefits from that because it reads the already-translated flat arrays. It is not a substitute for this utility because AMGF requires a compact-column HypreParMatrix over the active coupled-DOF subset, while SurfaceProjector is an Operator trace map over the full boundary/LOR surface. Tests: - ctest -R test_amgf_utils --output-on-failure Current validation run for changed tests: ctest -R 'test_amgf_options|test_mortar_constraint_operator' --output-on-failure ctest -R test_amgf_utils --output-on-failure
Introduce exaconstit::amgf::GinkgoDirectSubspaceSolver, an mfem::Solver adapter that lets MFEM's AMGFSolver apply a Ginkgo sparse direct solve on the filtered AMGF subspace operator. The adapter converts the local diagonal block of the HypreParMatrix subspace operator to a Ginkgo CSR matrix during SetOperator(), then applies a cached Ginkgo experimental::solver::Direct solve from Mult(). Support both symmetric and nonsymmetric setup modes. The symmetric path uses Ginkgo Cholesky for the expected K-block / augmented-K-block cases, while the nonsymmetric path uses Ginkgo LU for future non-associated-flow tangents. Add MakeGinkgoExecutor() for AMGF subspace executor selection. The current implementation keeps AMGF host-resident: "auto" selects OpenMP when the configured Ginkgo package provides it and falls back to the reference executor otherwise; explicit GPU executor strings abort with a clear message because the hybrid GPU/full-assembly path is future work. The CMake configuration now exports EXACONSTIT_GINKGO_HAS_OMP when the Ginkgo package reports an OMP backend. Document the adapter limitations in the new mortar_pbc header. In particular, this partial step intentionally factors HypreParMatrix::GetDiag() only, matching the initial AMGF wiring skeleton and the current single-rank correctness tests. Distributed production subspace-solve behavior should be revisited when the full AMGFSolver integration is exercised across ranks. Add a focused unit test covering both the Cholesky and LU adapter paths on small HypreParMatrix systems. Register the test with the mortar_pbc CTest suite. Update the new AMGF utility/test matrix-construction paths to use MFEM memory-safe Read()/ReadI()/ReadJ()/ReadData() accessors instead of raw GetData()/GetI()/GetJ() pointer access, so these setup paths synchronize host data correctly if future full matrix assembly uses device-resident memory.
Introduce mortar_pbc::MortarSaddlePreconditionerAMGF as the Path-A block preconditioner for the mortar saddle Jacobian. The class owns an mfem::AMGFSolver, keeps the Boolean AMGF transfer matrix alive, holds the caller-provided filtered-subspace solver, and configures BoomerAMG through the same systems-option path used by the existing full-assembly K-block AMG setup. SetOperator now validates that the saddle operator is a 2x2 mfem::BlockOperator with a HypreParMatrix K block, routes K into AMGF, refreshes the Jacobi-style K probe, and preserves the existing Path-A Schur diagonal computation through MortarConstraintOperator:: ComputeInvDiagSchur. The lower block is installed as a DiagonalScaler inside a BlockDiagonalPreconditioner. Path D constructor parameters are kept for the later augmented-Lagrangian step, but Path D currently aborts clearly instead of silently taking an incomplete path. Add a focused single-rank class-level unit test. The test builds the mortar constraint operator, constructs the Boolean AMGF transfer P, installs a Ginkgo direct subspace solver, verifies setup dimensions and gamma state, and checks that the AMGF preconditioner stores the same Path-A inverse Schur diagonal as the existing diagonal-probe path. Register the new source/header with exaconstit_static and add the new test to the mortar_pbc CTest suite.
Add the SystemDriver branch for LinearSolver preconditioner AMGF on the mortar-PBC saddle path. The AMGF branch constructs the Ginkgo direct filtered-subspace solver and installs MortarSaddlePreconditionerAMGF as the saddle J_prec, while preserving the existing MortarSaddlePreconditioner path for all non-AMGF preconditioners. Extend MortarSaddlePreconditionerAMGF with a deferred-transfer constructor for production use. SystemDriver does not have the assembled Hypre K block when the saddle preconditioner is constructed, so the class now rebuilds the Boolean AMGF transfer operator in SetOperator() from the current K row partition and MortarConstraintOperator::GetConstraintCoupledDofIndices(). The explicit-P constructor remains available for focused unit tests. Keep AMGF_AUG_LAGRANGIAN as an explicit early abort in SystemDriver. Path D requires the augmented residual/scaler wiring and remains a later partial step. Strengthen the AMGF saddle preconditioner test so it exercises Mult(), not only SetOperator(). The test uses warning-mode Hypre error handling only for its artificial pinned elasticity matrix; production construction still defaults to aborting on Hypre setup/solve errors. The test verifies finite AMGF upper-block output and exact preservation of the existing Path-A Schur diagonal action.
Add a post-linear-solve diagnostic sink to ExaNewtonSolver and wire it into the mortar saddle Newton diagnostic logger. The logger now records linear_iterations, linear_final_norm, and linear_converged on the same per-Newton-iteration CSV row as the residual block decomposition. Remove the rejected Caliper-style Krylov iteration reporting from the SystemDriver path. The linear solve data now stays associated with the Newton iteration that produced it, and converged Newton rows with no linear solve are written explicitly with -1 sentinel values. Clarify AMGF filtered-subspace sizing: n_filter is the unique set of constraint-coupled displacement true DOFs, not the number of lambda constraint rows. Extend the AMGF preconditioner test to assert that relationship and print the n_filter/n_lam ratio with the correct label. Add a focused SaddleNewtonDiagnosticLogger test covering the new linear solve CSV columns and the no-linear-solve sentinel row.
Add validation that prevents AMGF preconditioners from being paired with MINRES. MFEM's AMGFSolver is a filtered/multiplicative preconditioner and is not guaranteed to satisfy MINRES' symmetric preconditioner contract. Apply the check both to the K-block Krylov solver options and to the mortar SaddlePoint solver options when mortar PBC is active. This catches the actual AMGF mortar configuration rather than only the generic Krylov table. Extend the AMGF option test to cover rejected K-block MINRES, rejected mortar SaddlePoint MINRES, and accepted mortar SaddlePoint GMRES.
Add a dedicated SaddlePointMethod option so the augmented-Lagrangian formulation is selected through [Solvers.SaddlePoint] rather than being encoded as an AMGF preconditioner variant. Parse method = "AUGMENTED_LAGRANGIAN" plus augmented_lagrangian_gamma, document the new deck shape, and print the selected saddle method in the options summary. Keep AMGF as a K-block preconditioner choice. The legacy AMGF_AUG_LAGRANGIAN spelling remains parsed for compatibility, but new Phase D wiring should use the saddle method switch and then apply the selected K-block preconditioner, whether AMG or AMGF, to K_gamma. Update the system-driver guard so augmented-Lagrangian requests fail explicitly until the Phase D operator/RHS path is implemented. Move the focused AMGF option coverage under test/mortar_pbc and extend it to cover saddle-method parsing and to verify that the augmented-Lagrangian method by itself does not inherit the AMGF + MINRES restriction.
Add MortarConstraintOperator::BuildCTransposeC() as the Phase D setup hook for augmented-Lagrangian saddle solves. The method assembles the currently active EA constraint operator into a temporary HypreParMatrix from the flat row arrays, then forms C^T C with MFEM/Hypre transpose and parallel sparse matrix multiplication. Preserve the current mortar filter semantics during assembly: active pair labels and component masks from Reset are honored, sentinel true DOFs are skipped, and off-rank mortar-side columns are resolved through the existing import topology. The returned matrix uses the parent FES true-DOF row and column partition so it can be combined with the K block when K_gamma wiring is added. Extend the mortar constraint operator test to verify the assembled normal matrix action against the matrix-free reference C^T(Cu), including both the full XYZ constraint set and an X-only filtered constraint set. Register a Flux-backed np=4 CTest entry for the C^T C coverage so the off-rank column, Hypre transpose, and parallel matrix multiply paths are tested with multiple MPI ranks.
Extend MortarSaddlePreconditioner with an opt-in augmented-Lagrangian setup path for the saddle solver method. In augmented mode the preconditioner builds K_gamma = K + gamma C^T C and refreshes the selected K-block preconditioner on K_gamma instead of K. This keeps the augmented-Lagrangian formulation independent of the K-block preconditioner choice, so AMG, Jacobi, ILU, and later AMGF can be tested against the same augmented saddle method. Replace the lambda-block action with the trivial gamma I scaling in augmented mode. Positive gamma overrides are used directly; non-positive gamma requests the trace-scaled default tr(K) / tr(C^T C) * n_lambda / n_u, with a defensive gamma=1 fallback for degenerate traces. Add mortar preconditioner coverage that uses a recording K-block solver to verify SetOperator receives K_gamma and verifies Mult applies both the recorded K_gamma upper-block action and the gamma-scaled lambda block.
Enable the augmented-Lagrangian saddle method for the non-AMGF mortar preconditioner path. The driver now treats [Solvers.SaddlePoint] method = "AUGMENTED_LAGRANGIAN" as a saddle solver formulation choice and passes the configured augmented_lagrangian_gamma to MortarSaddlePreconditioner, where K_gamma and the gamma-I lambda block are built during preconditioner setup. Keep the AMGF preconditioner path separate. Requests that combine AMGF with the augmented saddle method now fail with a targeted runtime error until the AMGF-specific augmented setup is implemented, preserving the intended split between saddle method selection and K-block preconditioner selection. Document the standard versus augmented construction behavior at the driver call site so future AMGF wiring can follow the same option split.
Introduce augmented-Lagrangian saddle wrappers for the Phase D Newton linear solve. The new operator wrapper preserves the physical mortar saddle residual returned by Mult, so nonlinear convergence checks, diagnostics, line search, and trust-region acceptance continue to use the true residual. Its GetGradient path returns an augmented Jacobian action that adds gamma C^T C to the displacement block. Add a matching RHS solver wrapper that applies the gamma C^T r_lambda shift immediately before the inner Krylov solve. The wrapper supports both unscaled and saddle-residual-scaled solve paths by temporarily mapping scaled right-hand sides back to physical units before applying the augmented RHS correction. Wire the wrappers through SystemDriver for the AUGMENTED_LAGRANGIAN saddle method, refresh them across active periodic-spec changes, and guard the current implementation to FULL assembly because K_gamma setup requires HypreParMatrix K blocks. Update MortarSaddlePreconditioner to unwrap augmented Jacobians before extracting K so the preconditioner builds K_gamma from the original mechanics block instead of adding gamma C^T C twice. Add focused mortar tests for the augmented Jacobian action and RHS shift, and register the test under test/mortar_pbc.
Extend MortarSaddlePreconditionerAMGF so the AMGF K-block preconditioner can be used with the augmented-Lagrangian saddle method. In augmented mode the AMGF wrapper unwraps AugmentedLagrangianSaddleJacobian back to the original saddle BlockOperator, extracts the unaugmented K block, builds K_gamma = K + gamma C^T C, and refreshes AMGFSolver on K_gamma. This avoids double-counting the penalty term while keeping AMGF as a K-block preconditioner choice rather than a separate saddle method. Replace the AMGF lambda-block diagonal Schur approximation with gamma I when augmented mode is active. Positive gamma overrides are used directly; non-positive gamma requests the same trace-scaled default used by the generic mortar saddle preconditioner. Wire SystemDriver so AMGF is no longer rejected with [Solvers.SaddlePoint] method = "AUGMENTED_LAGRANGIAN". The saddle method now controls the augmented formulation, while [Solvers.Krylov] preconditioner = "AMGF" controls the K-block preconditioner. Add focused AMGF coverage for the augmented setup, including a 4-rank CTest registration to exercise distributed C^T C assembly and AMGF filtered-transfer construction. Expand the generic MortarSaddlePreconditioner documentation to match the verbose style used throughout src/mortar_pbc, including file-level design notes, helper documentation, setup comments, and private member documentation.
Update the mortar SolveInit direct saddle solve so it uses the same augmented-Lagrangian linear algebra as the regular Newton path. When [Solvers.SaddlePoint] method = "AUGMENTED_LAGRANGIAN" is active, or the legacy AMGF_AUG_LAGRANGIAN preconditioner spelling is used, SolveInit now builds K_gamma = K + gamma C^T C and shifts the displacement RHS by gamma C^T r_lambda before calling SaddlePointSolver. Keep the physical constraint residual unchanged. The augmentation is applied only to the direct linear solve operator and RHS, matching the regular Newton wrapper path and preserving the interpretation of r_lambda = C x_prev - g. Refresh the K-Jacobi probe against the same operator passed to SaddlePointSolver: K in standard mode and K_gamma in augmented mode. Add local documentation for the SolveInit trace-scaled gamma default because this path bypasses the regular saddle preconditioner setup where gamma is normally selected.
Teach the Newton linear-solve diagnostic extractor to unwrap AugmentedLagrangianRhsSolver and report diagnostics from its inner solver. The augmented RHS wrapper only applies the algebraic augmented-Lagrangian residual shift before delegating to the real Krylov solve, so the inner solver owns the meaningful iteration count, final residual norm, and convergence flag. This restores populated newton_iters.csv linear diagnostics for AUGMENTED_LAGRANGIAN saddle runs and makes Phase D integrated validation directly comparable across AMG standard, AMGF standard, AMG augmented, and AMGF augmented cases.
…s 6 and AMGF The SurfaceProjector had a few issues where it had a few issues with assumptions related to the child and parent FES which were not necessarily correct anymore and this was evident with the tet meshes but not the auto-generated meshes. Next, a few issues were noted where it assumed all ranks would have data rather than there being some empty rank data sets.
The Gingko solver had a ton of issues in that I didn't realize it was only good for a single MPI rank even though their website made it sound like it could work over distributed ranks :/ So, I had Claude swap us over to SuperLU and copy over for the time being the MFEM parallel direct solver utility package. This works and does at least provide us with a path to get on the GPU with a direct sparse solver but still kinda sad couldn't leverage Gingko... Outside of that through Claude / Codex, I found out that the non-conforming meshes had a pretty glaring issue. It turns out that there was never a ghost element layer accounted for which meant that the non-conformal stuff was missing contributions from elements not on the chosen processor...
Had Claude update the install scripts such that SuperLU is used instead of Gingko.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
So this has been completely driven Claude Opus 4.7 writing all of the code and tests for this feature as I'm in no way an expert on mortar methods, and I've been very much interested in supporting periodic boundary conditions in ExaConstit for a while. The code was largely inspired by https://doi.org/10.1016/j.cma.2021.113930 which was a really interesting paper I came across on periodic boundary conditions and a way to enforce periodic boundary conditions through mortar methods which allows you to have non-conforming mesh faces or heterogeneous materials across various faces.
So, I largely "vibe" coded in the sense that I did not write anything but directed where things should go, what assumptions it was making were bad, when it should be doing a deeper research of the field to ensure we were doing things correctly as well as many, many different things. The initial prototype of this code started as a python miniapp using pyMFEM where I could quickly iterate with it and verify what it was doing worked starting from 2D and moving up to 3D with a linear elastic problem or a bi-material elastic material with a soft and hard phase. After doing that, I felt comfortable moving onto the C++ port and bringing over all of the python tests to C++ to verify the port was working as expected. The C++ side of things took a lot of work as we needed scalable algorithms (largely for the constraint systems), GPU-capable algorithms (still getting there), usable preconditioners, support for non-conforming mesh faces, proper translation from the total lagrangian framework these methods were written in to the updated lagrangian framework, and then like a ton of tests to make sure things actually work and help to track bugs down later.
This is still a work in progress as I need to get it working with higher-order elements as we're currently limited to just linear elements. Outside of that, I'm also working on a number of extensions to the nonlinear / linear solvers so that we see faster convergences in some polycrystal simulations I've been running with only a handful of grains and linear tets, which are definitely a harder test for these models. Once I get a lot of this done, I will work on having Claude create some better / extensive standalone documentation to go with this new feature as this is not a small feature addition and there's a lot of things that need to be documented on how it works and what Claude was pulling from while working on this.