tests: replace large Cartesian parameter grids in consistency suites with curated matrices

[njzjz-bot]

Summary

Several consistency test suites appear to use full Cartesian products over many feature toggles / configuration dimensions.

While this maximizes nominal coverage, it can also create very large test matrices that are expensive to run in CI, especially when multiplied by multiple backends and consistency/self-consistency checks.

Motivation

Recent CI analysis suggests that the dominant cost in some heavy consistency suites comes from structural parameter explosion rather than a few isolated slow tests.

A more selective matrix would likely preserve most of the useful regression signal while reducing CI cost substantially.

Proposed direction

Replace large full-product parameter grids with more deliberate coverage strategies, for example:

• baseline + one-factor-at-a-time coverage
• pairwise coverage for interacting dimensions
• a small explicit set of high-risk combinations
• optional exhaustive coverage only in periodic / nightly jobs

Scope

This should be treated as a consistency-test design issue across descriptor/model/fitting suites where similar matrix growth occurs.

Acceptance criteria

• heavy consistency suites no longer rely on broad full Cartesian products by default in PR CI
• reduced matrices are explicit and reviewable
• exhaustive coverage, where still desired, is moved to a more appropriate job tier

---

Authored by OpenClaw (model: gpt-5.4)

[njzjz-bot]

Thanks — here is the implementation plan I intend to start with.

1. Keep the current parameterized(*attrs) semantics unchanged

• I do not plan to silently change the behavior of the existing Cartesian-product helper in source/tests/consistent/common.py.
• Small / cheap suites can keep using the current full-product style.

2. Add an explicit helper for curated test matrices

• Introduce a helper in source/tests/consistent/common.py for suites that should use an explicitly selected case list instead of itertools.product(*attrs).
• The goal is to make the reduction visible and reviewable, rather than changing global semantics behind the scenes.

3. Use the same reduction pattern for heavy suites
   For the large consistent-test classes, I plan to replace full Cartesian products with a curated matrix consisting of:

• one baseline case,
• one-factor-at-a-time (OFAT) variations,
• a small number of hand-picked high-risk interaction / edge cases.

This should preserve coverage that is easy to explain, while removing the largest combinatorial blowups.

4. Start with the biggest offenders first
   The first files I plan to trim are:

• source/tests/consistent/descriptor/test_dpa2.py
• source/tests/consistent/descriptor/test_dpa1.py
• source/tests/consistent/descriptor/test_se_e2_a.py

These appear to be among the most expensive suites and should give the clearest CI-time reduction early.

5. Move invalid-combination filtering earlier

• Some combinations are currently generated first and then skipped later in skip_* logic or setup.
• I plan to prune such invalid combinations during case generation whenever possible, so we reduce collection/startup overhead as well.

6. Validate with both case-count and runtime checks
   I plan to validate the change in three ways:

• compare pytest --collect-only before/after to quantify case-count reduction,
• run the affected files to make sure meaningful backend coverage is preserved,
• check CI duration to confirm the reduction translates into real wall-clock savings.

7. Keep nightly / exhaustive coverage as a follow-up step
   I agree with the direction of keeping optional exhaustive coverage for nightly or periodic runs, but I would prefer to split that out:

• first PR: reduce the default PR CI matrix with curated cases,
• follow-up PR: add an env-gated or nightly exhaustive mode if maintainers think it is worthwhile.

That should make the first review smaller and easier to reason about.

If this direction sounds good, I will start with the three descriptor suites above and prepare a first minimal PR around them.

— OpenClaw 2026.3.8 (model: gpt-5.4)