fix: skip empty frame properties when dumping deepmd format (#977)

[wanghan-iapcm]

Summary

Fixes #977.

When an ABACUS SCF run converges but does not compute forces/stress (e.g. cal_force/cal_stress disabled — which is what users hit with the new GPU cusolver ks_solver), dpdata's ABACUS parser correctly produces a LabeledSystem with size-0 forces (this is an intended state, covered by TestABACUSLabeledOutputNoFS).

The bug is in the deepmd dump→load round-trip:

• dump did np.reshape(forces, [nframes, -1]), which for empty forces writes force.npy of shape (nframes, 0).

• On load, np.reshape((nframes, 0)-array, [nframes, natoms, 3]) raises:

  ValueError: cannot reshape array of size 0 into shape (20,64,3)
  

This is exactly the traceback reported in #977 (surfacing in dpgen's post_fp_check_fail).

Root cause investigation

• ABACUS (LTSv3.10.0) prints forces via ModuleIO::print_force independent of the KS solver; cusolver fills the wavefunctions just like genelpa/scalapack_gvx, and forces are computed from the density matrix afterward. There is no code path that disables force output for cusolver/device gpu.
• dpdata parses the v3.10.0 FmtTable force block correctly when present.
• The crash therefore only requires a converged log with no TOTAL-FORCE block, which dpdata represents as empty forces — and the deepmd round-trip then corrupts it.

Fix

In both dpdata/formats/deepmd/comp.py (npy) and dpdata/formats/deepmd/raw.py (raw) dump, skip an optional frame property whose array is empty while the system still has frames, rather than writing a meaningless (nframes, 0) array:

if nframes > 0 and np.asarray(data[dtype.name]).size == 0:
    continue

A missing force/virial file is already a supported state on load (_cond_load_data returns None and skips it), so the round-trip is now consistent.

Tests

Added two tests to TestABACUSLabeledOutputNoFS (reusing the existing INPUT.ch4-noforcestress fixture):

• test_noforcestress_deepmd_roundtrip
• test_noforcestress_deepmd_raw_roundtrip

Both fail on master with the reported ValueError and pass with this change. abacus + deepmd comp/raw/mixed/empty suites pass (312 tests); no new failures in the full suite.

Out of scope (user/dpgen side)

This makes dpdata robust to force-less ABACUS results. Producing usable training labels still requires cal_force 1/cal_stress 1 in the generated INPUT and adding cusolver to dpgen's ks_solver whitelist.

Summary by CodeRabbit

Bug Fixes

• Fixed an issue where empty optional properties (such as forces and virials) were incorrectly exported as invalid empty arrays in DeePMD format files. These properties are now properly skipped during export when data is unavailable.

Tests

• Added validation tests for proper handling of missing optional properties during DeePMD format conversions.

[codspeed-hq]

Merging this PR will not alter performance

⚠️ Unknown Walltime execution environment detected

Using the Walltime instrument on standard Hosted Runners will lead to inconsistent data.

For the most accurate results, we recommend using CodSpeed Macro Runners: bare-metal machines fine-tuned for performance measurement consistency.

✅ 2 untouched benchmarks

---

_{Comparing wanghan-iapcm:fix-abacus-missing-force (b774c0b) with master (7d75096)}

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 86.76%. Comparing base (7d75096) to head (b774c0b).
⚠️ Report is 1 commits behind head on master.

Additional details and impacted files

@@           Coverage Diff           @@
##           master     #982   +/-   ##
=======================================
  Coverage   86.75%   86.76%           
=======================================
  Files          89       89           
  Lines        8093     8097    +4     
=======================================
+ Hits         7021     7025    +4     
  Misses       1072     1072           

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[coderabbitai]

📝 Walkthrough

Walkthrough

Two early-continue guards are added to the dump() routines in dpdata/formats/deepmd/comp.py and dpdata/formats/deepmd/raw.py. When a frame-level optional property has an empty array (size == 0) but nframes > 0, the property is now skipped rather than written as an invalid (nframes, 0) array. Two new round-trip tests validate this for ABACUS SCF no-force/stress data.

Changes

Empty-property skip fix and tests

Layer / File(s)	Summary
Empty-property guards in comp and raw dump dpdata/formats/deepmd/comp.py, dpdata/formats/deepmd/raw.py	Both dump() routines now check each optional frame-level dtype: if the system has frames but the property array is empty (size == 0), that property's file is skipped instead of being reshaped into an unusable (nframes, 0) result.
Round-trip tests for no-force/stress SCF tests/test_abacus_pw_scf.py	tempfile import added; two new methods in TestABACUSLabeledOutputNoFS write the ABACUS SCF system to deepmd/npy and deepmd/raw, reload it, and assert frame count preservation, empty forces, and absent virials.

Estimated code review effort

🎯 2 (Simple) | ⏱️ ~10 minutes

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title 'fix: skip empty frame properties when dumping deepmd format (#977)' accurately describes the main change - implementing logic to skip writing empty frame properties during deepmd format dumping.
Linked Issues check	✅ Passed	The PR addresses the core requirement from issue #977: making dpdata robust to ABACUS results without force/stress data by skipping empty frame properties during deepmd dump, allowing successful round-trip without reshaping errors.
Out of Scope Changes check	✅ Passed	All changes are directly scoped to the issue: modifications to deepmd format dumping in comp.py and raw.py, plus tests validating the fix work correctly for force-less ABACUS results.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

✨ Finishing Touches

🧪 Generate unit tests (beta)

• Create PR with unit tests

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

[coderabbitai]

Actionable comments posted: 2

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@dpdata/formats/deepmd/comp.py`:
- Around line 155-160: The file has B028 ruff linting violations due to missing
explicit stacklevel keyword arguments in warnings.warn() calls at two locations
in the file. Fix these violations by adding the stacklevel=2 parameter to both
warnings.warn() calls that are missing it. This ensures that warnings are
properly attributed to the caller's code location rather than the warnings
module itself, which is required for compliance with the coding guidelines.

In `@dpdata/formats/deepmd/raw.py`:
- Around line 139-144: Fix the two ruff linting violations (B028) in the file by
adding the explicit stacklevel=2 keyword argument to both warnings.warn() calls.
Locate the two warnings.warn() calls in the file and add stacklevel=2 as a
keyword argument to each one (for example, warnings.warn(..., stacklevel=2)).
This will satisfy the ruff linting requirement to pass ruff check before
committing.

🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: 5ab6af88-73fd-461d-8443-b71962a9c2a4

📥 Commits

Reviewing files that changed from the base of the PR and between 7d75096 and b774c0b.

📒 Files selected for processing (3)

• dpdata/formats/deepmd/comp.py
• dpdata/formats/deepmd/raw.py
• tests/test_abacus_pw_scf.py

[njzjz-bot]

I reviewed this PR and the change looks correct to me.

Why I think this is the right fix:

• The failing case is an optional frame property (forces/virials) represented as a zero-size array while nframes > 0. Dumping that to DeePMD as (nframes, 0) creates a file that the loader later tries to reshape to (nframes, natoms, 3) or (nframes, 3, 3), which is exactly the reported failure mode.
• Skipping such empty optional data on dump is consistent with the existing load-side behavior: a missing force/virial file is already treated as absent optional data.
• The guard is placed before reshape, so normal non-empty per-frame properties keep the existing behavior; zero-frame systems are also not affected by the new condition.
• The new regression tests cover both deepmd/npy and deepmd/raw round-trips for the existing ABACUS no-force/no-stress fixture.

I also ran the relevant local tests:

cd tests && uv run --with pytest pytest -q \
  test_abacus_pw_scf.py::TestABACUSLabeledOutputNoFS \
  test_deepmd_comp.py test_deepmd_raw.py test_deepmd_mixed.py

Result: 301 passed.

No blocking comments from me.

Reviewed by OpenClaw 2026.6.8 (model: custom-chat-jinzhezeng-group/gpt-5.5).