Implement Principal Component Analysis (PCA) module

[JAi-SATHVIK]

Motivation

WHY NEEDED

Principal Component Analysis is the workhorse of dimensionality reduction in modern scientific computing (e.g., climate patterns, fluid dynamics modes, machine learning preprocessing). Currently, users must manually combine stdlib_stats_cov, stdlib_stats_mean, and stdlib_linalg_svd to perform this common task. Adding a dedicated PCA module would significantly lower the barrier to entry for data science workflows in Fortran, bringing it closer to ecosystems like scikit-learn or MATLAB.

PLAN

I have analyzed the current capabilities of stdlib_stats and stdlib_linalg and found we have all the prerequisites. I propose implementing a pca derived type or set of routines that offers:

Dual Algorithms:
Eigendecomposition of Covariance Matrix: Leveraging existing stdlib_stats_cov and stdlib_linalg_eig.
SVD Method: Performing SVD directly on the centered data matrix (often more numerically stable) using stdlib_linalg_svd.
API Design:
pca_fit(X, n_components): Computes the principal components (eigenvectors), singular values, and explained variance ratios.
pca_transform(X, pca_model): Projects new data into the reduced dimensional space.
pca_inverse_transform(): Reconstructs original data from the reduced space.
Integration: This module would sit naturally in stdlib_stats, acting as a bridge between the statistics and linear algebra modules.

Prior Art

https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html
https://www.mathworks.com/help/stats/pca.html

Additional Information

No response

[jalvesz]

Excellent idea @JAi-SATHVIK if you have a plan for it you can start a draft PR to polish the design.

My advice, lets try to iterate first on the high level API which is the one thing that once settles is harder to change, and then polish the inner details.

Given the dependency on both the stats and linalg modules, just be careful with the dependencies as there is an ongoing effort to modularize the build. As of now the stats module does not depend on either blas or lapack nor the higher level linalg APIs. But indeed a PCA module would make sense to live closer to the stats module rather than linalg conceptually speaking.

Poke @jvdp1 @perazz @loiseaujc

[JAi-SATHVIK]

Thank you @jalvesz

On Dependencies: I understand the concern regarding stdlib_stats not currently depending on linalg (BLAS/LAPACK). To maintain this separation, perhaps we could consider placing this in a sub-module or a distinct stdlib_stats_pca module. This way, users who only need basic statistics don't pull in the linear algebra overhead, but those doing multivariate analysis have a seamless experience. I'll look into how stdlib handles these optional-style dependencies.

On API Design: To start the iteration on the high-level API, I am envisioning a derived type to maintain state (like the mean and components). Here is a rough sketch:

type :: pca_model
    private
    real(wp), allocatable :: mean(:)       ! For centering
    real(wp), allocatable :: components(:,:) ! Principal axes
    real(wp), allocatable :: variance(:)   ! Explained variance
    ! ... other metadata
end type

interface pca_fit
    module procedure pca_fit_svd
    module procedure pca_fit_cov
end interface

I will start a Draft PR to formalize this structure and the specific signatures for pca_transform and pca_inverse_transform. This will give us a concrete place to discuss whether we should favor a functional approach or this object-oriented approach.

[jvdp1]

Thank you @JAi-SATHVIK for this proposal. I think it is a really good idea.

Re: dependencies with stdlib_blas, stdlib_lapack, I suggest to not worry about that. It is true that stdlib_stats does not depend currently on BLAS/LAPACK. But at a long term, procedures like (co)variance could also benefit on BLAS/LAPACK. Therefore, adding a submodule for PCA could be a good approach.

Re: API: procedural API, like for mean, var, svd, ... should be prefered over DT API, whenever it is possible. DT API can be added at a later stage on top of the existing API.

Openinig a draft PR is indeed a good way to discuss these details.