Feature: Spack support for ABACUS

[QuantumMisaka]

Background

Spack is a multi-platform package manager that builds and installs multiple versions and configurations of software. It works on Linux, macOS, and many supercomputers. Spack is non-destructive: installing a new version of a package does not break existing installations, so many configurations of the same package can coexist.

Many scientific modeling packages such as LAMMPS, ABINIT, CP2K have built-in package manage script in Spack.

There is a Spack built-in scripts for installing and managing ABACUS https://github.com/spack/spack/blob/develop/var/spack/repos/builtin/packages/abacus/package.py, but this script is for a much old version of abacus.

Describe the solution you'd like

An up-to-date Spack built-in management script for ABACUS is needed and need to be maintained

Task list only for developers

• Notice possible changes of behavior
• Explain the changes of codes in core modules of ESolver, HSolver, ElecState, Hamilt, Operator or Psi

Notice Possible Changes of Behavior (Reminder only for developers)

No response

Notice any changes of core modules (Reminder only for developers)

No response

Notice Possible Changes of Core Modules (Reminder only for developers)

No response

Additional Context

No response

Task list for Issue attackers (only for developers)

• Review and understand the proposed feature and its importance.
• Research on the existing solutions and relevant research articles/resources.
• Discuss with the team to evaluate the feasibility of implementing the feature.
• Create a design document outlining the proposed solution and implementation details.
• Get feedback from the team on the design document.
• Develop the feature following the agreed design.
• Write unit tests and integration tests for the feature.
• Update the documentation to include the new feature.
• Perform code review and address any issues.
• Merge the feature into the main branch.
• Monitor for any issues or bugs reported by users after the feature is released.
• Address any issues or bugs reported by users and continuously improve the feature.

[yizeyi18]

Some questions:

• Should makefile install be supported in the script now, in future, or never?
  Makefile is far much faster than cmake, as spack could automatically fill the vars, and my preference is In Future. One lying problem is source/makefile seems not updated for long, and lacks a lot of optional choices: to enable elpa or not, to enable lcao or not, etc. Fixing them would take some time and delay the time the new script to be released.
• Should the script support BLAS/LAPACK implementations in addition to OpenBLAS/MKL?
  On spack, a bunch of BLAS/LAPACK implementations are served, including reference BLAS/LAPACK, BLIS/LibFlame, Cray Libsci, ARM Performance Libraries, ATLAS, etc. Realizing support for them with makefile is easy, but with cmake needs some works. Would the script implement support for them, or at lesat some, or none?
• Should the script support CUDA?
  Compatibility to some old cards may need test, or just leave this to the community?

EDIT:

• Some choices are defined in CMakeLists.txt but not the Document: USE_ROCM, ENABLE_PAW, ENABLE_FLOAT_FFTW, ENABLE_RAPIDJSON.
  Are they ready to be used currently?

[yizeyi18]

A draft implementation is finished and visible at https://github.com/yizeyi18/spack.

[QuantumMisaka]

A draft implementation is finished and visible at https://github.com/yizeyi18/spack.

@yizeyi18 For Spack management of ABACUS installation, we may not need to manage the packages flexibly for now, but a usable implementation is needed first. We need gcc- and intel-packages toolchain in Spack, but this can be simple and we can put the extendable job later.

Also, installation package management is a somehow hard work, more support may need like:
0. Package compatibility of ABACUS, which was not clear now

1. Choices in CMakeLists.txt needed to be in docs and also be accessible in Spack
2. GPU installation in Spack, especially compatibility problem.

We need more discussion and cooperate @caic99 @pxlxingliang

[caic99]

Hi @yizeyi18 ,
In my humble opinion:

• Should makefile install be supported in the script now, in future, or never?
  Makefile is far much faster than cmake, as spack could automatically fill the vars, and my preference is In Future. One lying problem is source/makefile seems not updated for long, and lacks a lot of optional choices: to enable elpa or not, to enable lcao or not, etc. Fixing them would take some time and delay the time the new script to be released.

The current supporting policy is that: Makefile is provided for backward compatible, and only supports the core components of ABACUS; CMake supports advanced features and is the mainstream of developing. The former is considered to be deprecated.

• Should the script support BLAS/LAPACK implementations in addition to OpenBLAS/MKL?

Yes. They share the same interface. For conda-forge packages, at build time the NETLIB’s reference implementations are used, and users can select a runtime library. (ref)

On spack, a bunch of BLAS/LAPACK implementations are served, including reference BLAS/LAPACK, BLIS/LibFlame, Cray Libsci, ARM Performance Libraries, ATLAS, etc. Realizing support for them with makefile is easy, but with cmake needs some works. Would the script implement support for them, or at lesat some, or none?

That would be better!

• Should the script support CUDA?

Yes, but maybe with low priority.

Compatibility to some old cards may need test, or just leave this to the community?

The latter is OK.

EDIT:

• Some choices are defined in CMakeLists.txt but not the Document: USE_ROCM, ENABLE_PAW, ENABLE_FLOAT_FFTW, ENABLE_RAPIDJSON.
  Are they ready to be used currently?

Any features not documented are considered as not production-ready.

[yizeyi18]

@caic99 @QuantumMisaka I think the spack script is ready to be released, as rocm install part has been checked. All features which don't depend on other DP libs(like, LibRI) get implemented; additional BLAS/LAPACK support almost done, just waiting for #3623. Do you think any additional work needed now?

[caic99]

Do you think any additional work needed now?

@yizeyi18 LGTM. Let's start from a minimum available build. Thanks!

[mohanchen]

We will close this issue since no obvious solution can be found in a short time. We always welcome new issues.