cedalion.sigdecomp.multimodal.cca_models
Module for CCA-like models
Functions
|
"Estimate the canonical vectors Wx, and Wy for the datasets X and Y. |
|
Extracts the top singular vectors from X using an iterative power method. |
|
Compute the inverse square root of a covariance matrix C. |
|
Compute the leading (sparse) singular vector pair and value (u, sigma, v) of a matrix X using an alternating power method. |
|
Apply soft thresholding elementwise to an array X. |
Classes
|
Perform CCA between two datasets X and Y. |
|
Perform Elastic Net Canonical Correlation Analysis (CCA) between two datasets X and Y. |
Class for decomposing multimodal data, X and Y, into latent sources using linear filters. |
|
|
Perform PLS between two datasets X and Y. |
|
Perform CCA between two datasets X and Y with L2 regularization, a.k.a ridge CCA. |
|
Perform Sparse CCA between two datasets X and Y with L1 regularization, a.k.a sparse CCA, based on Parkhomenko et al. [PTB09]. |
|
Perform Partial Least Squares (PLS) between two datasets X and Y with L1 regularization, a.k.a sparse PLS, based on a combination from Parkhomenko et al. [PTB09] and Witten et al. [WTH09]. |
|
Perform structured sparse Canonical Correlation Analysis (ssCCA) between two datasets X and Y. |
- class cedalion.sigdecomp.multimodal.cca_models.MultimodalSourceDecomposition(
- N_components: int = None,
- max_iter: int = 100,
- tol: float = 1e-06,
- scale: bool = True,
Bases:
objectClass for decomposing multimodal data, X and Y, into latent sources using linear filters.
This main class is inherited by other source decomposition methods, such as ElasticNetCCA, ssCCA, and PLS. It implements methods to validate input dimensions, apply normalization, and transform data from two modalities using filters learned during training.
- Parameters:
N_components (int, optional) – Number of components to extract. If None, the number of components is set to the minimum number of features between modalities.
max_iter (int) – Maximum number of iterations for the algorithm.
tol (float) – Tolerance for convergence.
scale (bool) – Whether to scale the data during normalization to unit variance. Defaults to True.
- validate_inputs_fit(
- X: DataArray,
- Y: DataArray,
Validates input data of fit function and returns it with the correct dimensions and labels.
This method ensures that the input data to the fit function, X and Y, i.e. those used for training, have the expected dimension labels and sizes and returns them with the dimensions ordered as (sample_name, feature_name). It also initializes the number of samples, and features.
- Parameters:
X (DataArray) – Input data for modality X.
Y (DataArray) – Input data for modality Y.
- Returns:
- A tuple (X, Y) where:
X (DataArray): Input data for modality X ordered as (sample_name, featureX_name). Y (DataArray): Input data for modality Y ordered as (sample_name, featureY_name).
- Return type:
tuple
- validate_inputs_transform(
- X: DataArray,
- Y: DataArray,
Validates that the to-be-transformed data have the expected dimension labels and sizes.
This method ensures that X and Y have the same dimension labels and number of features than the ones used during training. The number of time points, however, can be different.
- Parameters:
X (DataArray) – Input data for modality X.
Y (DataArray) – Input data for modality Y.
- Returns:
- A tuple (X, Y) where:
X (DataArray): Input data for modality X ordered as (sample_name, featureX_name). Y (DataArray): Input data for modality Y ordered as (sample_name, featureY_name).
- Return type:
tuple
- normalization_fit(
- X: DataArray,
- Y: DataArray,
Normalize input data and save normalization parameters (mean and std) for later.
This method centers and scales the data for both modalities along the sample dimension. It computes the mean and standard deviation for X and Y using the provided standardization function, updating the corresponding class attributes.
- Parameters:
X (DataArray) – Input data for modality X.
Y (DataArray) – Input data for modality Y.
- Returns:
A tuple (X, Y) of standardized data arrays for modalities X and Y.
- Return type:
tuple
- normalization_transform(
- X: DataArray,
- Y: DataArray,
Applies normalization input data using trained parameters.
This method standardizes the input data arrays X and Y using the normalization parameters (mean and standard deviation) computed during the fitting process.
- Parameters:
X (DataArray) – Input data for modality X.
Y (DataArray) – Input data for modality Y.
- Returns:
A tuple (X, Y) of normalized data arrays.
- Return type:
tuple
- convert_filters_to_DataArray(
- Wx: ndarray,
- Wy: ndarray,
- X: DataArray,
- Y: DataArray,
Convert filters Wx and Wy in numpy array format to DataArray with right dimensions and coordinates.
- Parameters:
Wx (ndarray) – Filter matrix for modality X with shape (Nx, N_components).
Wy (ndarray) – Filter matrix for modality Y with shape (Ny, N_components).
X_features (DataArray) – DataArray containing the features of modality X.
Y_features (DataArray) – DataArray containing the features of modality Y.
- Returns:
A tuple (Wx_xr, Wy_xr) containing the DataArray versions of Wx and Wy respectively.
- Return type:
tuple[DataArray, DataArray]
- transform(
- X: DataArray,
- Y: DataArray,
Apply the linear transformation on the input data using learnt filters.
This method validates the dimension labels and sizes of the input data to ensure consistency with the training data, applies normalization using the stored parameters, and then projects the normalized data onto a lower-dimensional space using the learned filters Wx and Wy. It retrieves the transformed arrays, a.k.a reconstructed sources.
- Parameters:
X (DataArray) – Input data for modality X. Expected to have dimensions (sample_name, featureX_name).
Y (DataArray) – Input data for modality Y. Expected to have dimensions (sample_name, featureY_name).
- Returns:
- A tuple (X_new, Y_new) where:
X_new (DataArray): Transformed data for modality X. Y_new (DataArray): Transformed data for modality Y.
- Return type:
tuple
- class cedalion.sigdecomp.multimodal.cca_models.ElasticNetCCA(
- N_components: int = None,
- l1_reg: float | list[float, float] = 0,
- l2_reg: float | list[float, float] = 0,
- max_iter: int = 100,
- tol: float = 1e-06,
- scale: bool = True,
- pls: bool = False,
Bases:
MultimodalSourceDecompositionPerform Elastic Net Canonical Correlation Analysis (CCA) between two datasets X and Y.
Apply CCA with L1 + L2 regularization, a.k.a elastic net. The algorithm finds sparse (L1) and normalized (L2) vectors Wx, and Wy as the solution to the following constrained optimization problem:
maximize Wx^T Cxy Wy subject to Wx^T Cx Wx = 1, Wy^T Cy Wy = 1,
||Wx||_1 <= c1x, ||Wy||_1 <= c1y, ||Wx||^2_2 <= c2x, ||Wy||^2_2 <= c2y
where Cx, Cy, and Cxy are the individual and cross-covariance matrices between X and Y datasets, and the last four constraints correspond to the standard L1-norm and L2-norm penalization terms. c1x and c1y controls sparsity while c2x and c2y controls the magnitude of the vectors. PLS algorithms are also captured by this algorithm by sending Cx and Cy to the identity matrices.
For the one-unit algorithm (sparse) SVD decomposition is performed on the whitened cross-covariance matrix K = Cx^(-1/2) Cxy Cy^(-1/2) (reduced to K = Cxy for PLS), using the following standard alternating power method (based on Parkhomenko et al. [PTB09]):
- Update u:
u <- K * v
u <- u / ||u||
- If L1:
u <- SoftThresholding(u, lambda_u/2) u <- u / ||u||
- Update v:
v <- K^T * u
v <- v / ||v||
- If L1:
v <- SoftThresholding(v, lambda_v/2) v <- v / ||v||
The resulting u and v are the leading left and right singular vectors of K which are nothing but individual components of the filters Wx and Wy. The softthresholding function bring some components to zero. If L2 regularization is used, prior to computing K, Cx and Cy are shifted by Cx <- Cx + alpha_x I and Cy <- Cy + alpha_y I.
Multiple components are obtained via a deflation method, subtracting from K its 1-rank approximation on each iteration. The returned vectors Wx and Wy are ordered in desceding order w.r.t. the singular values, which coincide with the canonical correlations.
- Parameters:
N_components (int, optional) – Number of components to extract. If None, the number of components is set to the minimum number of features between modalities.
l1_reg (float or list of floats) – list containing lambda_u and lambda_v (see above). If a single float is provided,
lambda_v. (then lambda_u =)
l2_reg (float or list of floats) – list containing alpha_x and alpha_y (see above). If a single float is provided,
alpha_y. (then alpha_x =)
max_iter (int) – Maximum number of iterations for the algorithm.
tol (float) – Tolerance for convergence.
scale (bool) – Whether to scale the data during normalization to unit variance. Defaults to True.
pls (bool) – Whether to perform PLS regression. Defaults to False.
- Wx[source]
Linear filters for dataset X with dimensions (featureX_name, latent_featureX_name)
- Type:
DataArray
- Wy[source]
Linear filters for dataset Y with dimensions (featureY_name, latent_featureY_name).
- Type:
DataArray
- fit(
- X: DataArray,
- Y: DataArray,
- sample_name: str = 'time',
- featureX_name: str = 'channel',
- featureY_name: str = 'channel',
Find the canonical vectors Wx, and Wy for the datasets X and Y.
- Parameters:
X (DataArray) – Input data for modality X. Expected to have dimensions (sample_name, featureX_name).
Y (DataArray) – Input data for modality Y. Expected to have dimensions (sample_name, featureY_name).
sample_name (str, optional) – Label for sample dimension, set to ‘time’ by default.
featureX_name (str, optional) – Label for X-feature dimension, set to ‘channel’ by default.
featureY_name (str, optional) – Label for Y-feature dimension, set to ‘channel’ by default.
- class cedalion.sigdecomp.multimodal.cca_models.StructuredSparseCCA(
- N_components: int = None,
- Lx: ndarray = None,
- Ly: ndarray = None,
- l1_reg: float | list[float, float] = 0,
- l2_reg: float | list[float, float] = 0,
- max_iter: int = 100,
- tol: float = 1e-06,
- scale: bool = True,
- pls: bool = False,
Bases:
MultimodalSourceDecompositionPerform structured sparse Canonical Correlation Analysis (ssCCA) between two datasets X and Y.
The ssCCA algorithm is based on Chen et al. [CBL+13] and it assumes the underlying X and Y features are linked through a graph structure. It finds sparse (L1) vectors Wx, and Wy as the solution to the following constrained optimization problem:
maximize Wx^T Cxy Wy subject to Wx^T Cx Wx = 1, Wy^T Cy Wy = 1,
||Wx||_1 <= c1x, ||Wy||_1 <= c1y, Wx^T Lx Wx <= c2x, Wy^T Ly Wy <= c2y
where Cx, Cy, and Cxy are the individual and cross-covariance matrices between X and Y datasets. The second constraint is the standard L1-norm penalization term, while the last constraint incorporates local information of the spatial distribution of the features trough the Laplacian matrices Lx and Ly. These terms encaurage filter components that are linked on the graphical structure to have similar values, making them to vary smoothly across the graph. The c1x and c1y controls sparsity while c2x and c2y controls the relative importante of the graph structure.
For the one-unit algorithm, first Cx and Cy are shifted by Cx <- Cx + alpha_x Lx and Cy <- Cy + alpha_y Ly, and then SVD decomposition is performed on the whitened cross-covariance matrix K = Cx^(-1/2) Cxy Cy^(-1/2), using the following standard alternating power method (based on Parkhomenko et al. [PTB09]):
- Update u:
u <- K * v
u <- u / ||u||
- If L1:
u <- SoftThresholding(u, lambda_u/2) u <- u / ||u||
- Update v:
v <- K^T * u
v <- v / ||v||
- If L1:
v <- SoftThresholding(v, lambda_v/2) v <- v / ||v||
The resulting u and v are the leading left and right singular vectors of K which are nothing but individual components of the filters Wx and Wy. The softthresholding function bring some components to zero.
Multiple components are obtained via a deflation method, subtracting from K its 1-rank approximation on each iteration. The returned vectors Wx and Wy are ordered in desceding order w.r.t. the singular values, which coincide with the canonical correlations.
- Parameters:
N_components (int, optional) – Number of components to extract. If None, the number of components is set to the minimum number of features between modalities.
Lx (ndarray) – Laplacian matrix for modality X.
Ly (ndarray) – Laplacian matrix for modality Y.
l1_reg (float or list of floats) – list containing lambda_u and lambda_v (see above). If a single float is provided,
lambda_v. (then lambda_u =)
l2_reg (float or list of floats) – list containing alpha_x and alpha_y (see above). If a single float is provided,
alpha_y. (then alpha_x =)
max_iter (int) – Maximum number of iterations for the algorithm.
tol (float) – Tolerance for convergence.
scale (bool) – Whether to scale the data during normalization to unit variance. Defaults to True.
pls (bool) – Whether to perform PLS regression. Defaults to False.
- Wx[source]
Linear filters for dataset X with dimensions (featureX_name, latent_featureX_name)
- Type:
DataArray
- Wy[source]
Linear filters for dataset Y with dimensions (featureY_name, latent_featureY_name).
- Type:
DataArray
- fit(
- X: DataArray,
- Y: DataArray,
- sample_name: str = 'time',
- featureX_name: str = 'channel',
- featureY_name: str = 'channel',
Find the canonical vectors Wx, and Wy for the datasets X and Y.
- Parameters:
X (DataArray) – Input data for modality X. Expected to have dimensions (sample_name, featureX_name).
Y (DataArray) – Input data for modality Y. Expected to have dimensions (sample_name, featureY_name).
sample_name (str, optional) – Label for sample dimension, set to ‘time’ by default.
featureX_name (str, optional) – Label for X-feature dimension, set to ‘channel’ by default.
featureY_name (str, optional) – Label for Y-feature dimension, set to ‘channel’ by default.
- class cedalion.sigdecomp.multimodal.cca_models.RidgeCCA(
- N_components: int = None,
- l2_reg: float | list[float, float] = 0,
- max_iter: int = 100,
- tol: float = 1e-06,
- scale: bool = True,
Bases:
ElasticNetCCAPerform CCA between two datasets X and Y with L2 regularization, a.k.a ridge CCA.
This algorithm is a particular case of the one implemented in the ElasticNetCCA class. See there for a detailed explanation of the algorithm.
- Parameters:
N_components (int, optional) – Number of components to extract. If None, the number of components is set to the minimum number of features between modalities.
l2_reg (float or list of floats) – list containing alpha_x and alpha_y (see above). If a single float is provided,
alpha_y. (then alpha_x =)
max_iter (int) – Maximum number of iterations for the algorithm.
tol (float) – Tolerance for convergence.
scale (bool) – Whether to scale the data during normalization to unit variance. Defaults to True.
- class cedalion.sigdecomp.multimodal.cca_models.SparseCCA(
- N_components: int = None,
- l1_reg: float | list[float, float] = 0,
- max_iter: int = 100,
- tol: float = 1e-06,
- scale: bool = True,
Bases:
ElasticNetCCAPerform Sparse CCA between two datasets X and Y with L1 regularization, a.k.a sparse CCA, based on Parkhomenko et al. [PTB09].
This algorithm is a particular case of the one implemented in the ElasticNetCCA class. See there for a detailed explanation of the algorithm.
- Parameters:
N_components (int, optional) – Number of components to extract. If None, the number of components is set to the minimum number of features between modalities.
l1_reg (float or list of floats) – list containing lambda_u and lambda_v (see above). If a single float is provided,
lambda_v. (then lambda_u =)
max_iter (int) – Maximum number of iterations for the algorithm.
tol (float) – Tolerance for convergence.
scale (bool) – Whether to scale the data during normalization to unit variance. Defaults to True.
- class cedalion.sigdecomp.multimodal.cca_models.CCA(
- N_components: int = None,
- max_iter: int = 100,
- tol: float = 1e-06,
- scale: bool = True,
Bases:
ElasticNetCCAPerform CCA between two datasets X and Y.
This algorithm is a particular case of the one implemented in the ElasticNetCCA class. See there for a detailed explanation of the algorithm.
- Parameters:
N_components (int, optional) – Number of components to extract. If None, the number of components is set to the minimum number of features between modalities.
max_iter (int) – Maximum number of iterations for the algorithm.
tol (float) – Tolerance for convergence.
scale (bool) – Whether to scale the data during normalization to unit variance. Defaults to True.
- class cedalion.sigdecomp.multimodal.cca_models.SparsePLS(
- N_components: int = None,
- l1_reg: float | list[float, float] = 0,
- max_iter: int = 100,
- tol: float = 1e-06,
- scale: bool = True,
Bases:
ElasticNetCCAPerform Partial Least Squares (PLS) between two datasets X and Y with L1 regularization, a.k.a sparse PLS, based on a combination from Parkhomenko et al. [PTB09] and Witten et al. [WTH09].
In Witten’s paper, the algorithm is presented as a particular case of their Penalized Matrix Decomposition (PMD) method, called PMD(L1, L1) or Sparse CCA. However, the latter name is misleading since in this problem we use identity matrices for the L2-norm constraints, rather than correlation matrices. That difference makes the method truly a SparsePLS one. Here, Witten’s method is modified by adding normalization on each iteration and dividing L1 parameters by 2.
This algorithm is a particular case of the one implemented in the ElasticNetCCA class. See there for a detailed explanation of the algorithm.
- Parameters:
N_components (int, optional) – Number of components to extract. If None, the number of components is set to the minimum number of features between modalities.
l1_reg (float or list of floats) – list containing lambda_u and lambda_v (see above).
provided (If a single float is)
lambda_v. (then lambda_u =)
max_iter (int) – Maximum number of iterations for the algorithm.
tol (float) – Tolerance for convergence.
scale (bool) – Whether to scale the data during normalization to unit variance. Defaults to True.
- class cedalion.sigdecomp.multimodal.cca_models.PLS(
- N_components: int = None,
- max_iter: int = 100,
- tol: float = 1e-06,
- scale: bool = True,
Bases:
SparsePLSPerform PLS between two datasets X and Y. This algorithm is a particular case of the one implemented in the SparsePLS class when no penalty is imposed. See there for a detailed explanation of the algorithm.
- Parameters:
N_components (int, optional) – Number of components to extract. If None, the number of components is set to the minimum number of features between modalities.
max_iter (int) – Maximum number of iterations for the algorithm. Defaults to 100.
tol (float) – Tolerance for convergence. Defaults to 1e-6.
scale (bool) – Whether to scale the data during normalization to unit variance. Defaults to True.
- cedalion.sigdecomp.multimodal.cca_models.estimate_filters(
- X: ndarray,
- Y: ndarray,
- N_components: int,
- l1_reg: list[float, float] = 0,
- l2_reg: list[float, float] = 0,
- Lx: ndarray = None,
- Ly: ndarray = None,
- pls: bool = False,
“Estimate the canonical vectors Wx, and Wy for the datasets X and Y.
Main function that estimates the canonical vectors Wx, and Wy for the datasets X and Y using an Elastic Net CCA algorithm with the option of incorporating Laplace matrices, Lx and Ly that transforms the algorithm into a structured sparse CCA. It assumes X and Y have shapes (samples, features).
- Parameters:
X (ndarray) – Input data for modality X with shape (Nt, Nx).
Y (ndarray) – Input data for modality Y with shape (Nt, Ny).
N_components (int) – Number of components to extract.
l1_reg (list of floats) – list containing lambda_u and lambda_v.
l2_reg (list of floats) – list containing alpha_x and alpha_y.
Lx (ndarray, optional) – Laplacian matrix for modality X. Defaults to None.
Ly (ndarray, optional) – Laplacian matrix for modality Y. Defaults to None.
pls (bool, optional) – Whether to perform PLS regression. Defaults to False.
- Returns:
A tuple (Wx, Wy) with the canonical vectors for X and Y, ordered by descending singular values.
- Return type:
tuple
- cedalion.sigdecomp.multimodal.cca_models.inv_sqrt_cov(C: ndarray, eps: float = 1e-10) ndarray[source]
Compute the inverse square root of a covariance matrix C.
Given a (symmetric) covariance matrix C, it computes C^{-1/2} = U Lambda^{-1/2} U^T using eigen-decomposition and clipping the inverse diagonal entries to eps to avoid division by zero and instabilities.
- Parameters:
C (ndarray) – Convariance matrix. Expected to be square and symmetric.
eps (float, optional) – Small value to avoid division by zero during inversion.
- Returns:
Inverse square root of C of the same size as input matrix.
- cedalion.sigdecomp.multimodal.cca_models.get_singular_vectors(
- X: ndarray,
- N_components: int,
- l1_reg: float | list[float, float] = 0,
Extracts the top singular vectors from X using an iterative power method.
The function iteratively extracts one sparse singular component at a time. On each iteration, it computes the leading singular pair using an alternating power method, subtracts the rank-1 approximation from X, and stores the component. Sparsity is enforced via L1 regularization when l1_reg > 0.
- Parameters:
X (ndarray) – Input matrix of shape (M, N) from which singular vectors are extracted.
N_components (int) – Number of singular components to extract.
l1_reg (float or list of floats, optional) – Regularization parameter for L1 sparsity. If scalar, the same value is applied for both u and v. Defaults to 0 (no sparsity).
- Returns:
U (ndarray): Matrix of left singular vectors with shape (m, N_components). S (ndarray): Array of singular values in the diagonal with length N_components. V (ndarray): Matrix of right singular vectors with shape (n, N_components).
- Return type:
tuple
- cedalion.sigdecomp.multimodal.cca_models.leading_singular_pair_power_method(
- X: ndarray,
- l1_reg: float | list[float, float] = 0,
- max_iter: int = 1000,
- tol: float = 1e-06,
Compute the leading (sparse) singular vector pair and value (u, sigma, v) of a matrix X using an alternating power method.
The method alternates between updating the left singular vector (u) and the right singular vector (v) until convergence following the rules:
- Update u:
u <- K * v
u <- u / ||u||
- If L1:
u <- SoftThresholding(u, lambda_u/2) u <- u / ||u||
- Update v:
v <- K^T * u
v <- v / ||v||
- If L1:
v <- SoftThresholding(v, lambda_v/2) v <- v / ||v||
Sparsity is enforced via soft-thresholding if the corresponding regularization parameters (lambda_u and lambda_v) encoded in l1_reg are set to a nonzero value.
- Parameters:
X (ndarray) – Input matrix of shape (m, n).
l1_reg (int, float, or list, optional) – L1 regularization parameter(s) for sparsity. If a scalar, the same value is applied to both u and v; if a list of two values, the first is used for u and the second for v. Defaults to 0 (no sparsity).
max_iter (int, optional) – Maximum number of iterations. Defaults to 1000.
tol (float, optional) – Convergence tolerance. Defaults to 1e-6.
- Returns:
u (np.ndarray): Leading left singular vector of shape (m, 1). sigma (float): Leading singular value. v (np.ndarray): Leading right singular vector of shape (n, 1).
- Return type:
tuple