Numerical instability of Davidson iterative eigensolver in solving LR-TDDFT excited states

[maki49]

Describe the bug

Now lr_solver=dav_subspace has mainly two problems in solving LR-TDDFT excited states:

1. "Assertion ( psi_norm > 0 ) failed" when doing normalizaiton in cal_grad (also appears in lr_solver=dav)
2. Large nonphysical negative eigenvalues (e.g., ~-1e10)

Both of them already exist before #5199, after which the problem 1 become worse.

To Reproduce

In tests/integrate/291_NO_KP_LR, enlarge lr_nstates (for example, 10), or use more k-points.

Possible Solution (with Refactor)

I'm going to try the following solutions that needs some refactor:

• Theoretically the number of initial-guess subspace basis vectors can be larger than the number of eigenpairs (bands or states) to be solved (expecially when the first several eigenpairs degenerate) , but in the present implementation, only n_bands initial vectors are copied in.
• Pass a precondition function similar to hpsi_func to enable custom preconditioner (which is hard-coded at present), such as sTDA or rid-preconditioner

[Cstandardlib]

• Theoretically the initial guess of subspace basis vectors can be larger than the number of eigenpairs (bands or states) to be solved (expecially when the first several eigenpairs degenerate) , but in the present implementation, only n_bands initial vectors are copied in.

What does it mean to pass larger initial guess of subspace basis than the number of eigenpairs (bands or states) to be solved?

• Is the size of subspace during the main loop n_initial_guess or n_band? For example we can use a subspace of size n_initial_guess when doing Rayleigh-Ritz and use n_band only when checking convergence.

[mohanchen]

Try to solve the issue by yourselves @maki49 @Cstandardlib @haozhihan

[haozhihan]

Will the same numerical instability be triggered for CG and Davidson?