manuotel / infomax-ica

Likelihood ICA for the ML project at CCU.

Infomax Independent Component Analysis (ICA) is a mathematical technique used to separate a mixed signal into its individual components. It is based on the principle of maximizing the mutual information between the input signal and the estimated sources.

ICA can be used to separate a mixed signal into its underlying sources, provided that the sources are statistically independent of each other. This is useful in a variety of applications, such as audio and image processing, where the goal is to extract individual sources from a mixed signal.

The basic idea behind ICA is to find a linear combination of the original signals that is maximally independent. This is done by minimizing the mutual information between the mixed signals and the estimated sources.

To implement ICA, we first need to define a set of basis functions that span the space of the mixed signals. These basis functions can be chosen in a variety of ways, such as using principal component analysis (PCA) or using a set of predefined basis functions.

Once the basis functions have been defined, we can use an optimization algorithm to find the linear combination of the basis functions that maximizes the mutual information between the mixed signals and the estimated sources. This optimization can be done using a variety of techniques, such as gradient descent or genetic algorithms.

Once the optimization is complete, we can use the linear combination of the basis functions to estimate the original sources. This can be done by applying the linear combination to the mixed signals and reconstructing the estimated sources.

Overall, ICA is a powerful technique for separating mixed signals into their underlying sources, and it has a wide range of applications in signal processing and machine learning.