lyouqi / rmt4ml

Matlab Notebook for visualizing random matrix theory results and their applications to machine learning

RMT4ML

This repository contains MATLAB and Python codes for visualizing random matrix theory results and their applications to machine learning, in Random Matrix Theory for Machine Learning.

In each subfolder (named after the corresponding section) there are:

• a .html file containing the MATLAB or IPython Notebook demos

• a .m or .ipynb source file

• Chapter 1 Introduction

• Chapter 2 Random Matrix Theory

  • Section 2.1 Fundamental objects
  • Section 2.2 Foundational random matrix results
    • Section 2.2.1 Key lemma and identities: Matlab code and Python code
    • Section 2.2.2 The Marcenko-Pastur and semicircle laws: Matlab code and Python code
    • Section 2.2.3 Large sample covariance matrices and generalized semicircles: Matlab code and Python code
  • Section 2.3 Advanced spectrum considerations for sample covariances: Matlab code and Python code
  • Section 2.4 Preliminaries on statistical inference
    • Section 2.4.1 Linear eigenvalue statistics: Matlab code and Python code
    • Section 2.4.2 Eigenvector projections and subspace methods: Matlab code and Python code
  • Section 2.5 Spiked model: Matlab code and Python code
  • Section 2.6 Information-plus-noise, deformed Wigner, and other models
  • Section 2.7 Beyond vectors of independent entries: concentration of measure in RMT
  • Section 2.8 Concluding remarks
  • Section 2.9 Exercises

• Chapter 3 Statistical Inference in Linear Models

  • Section 3.1 Detection and estimation in information-plus-noise models
    • Section 3.1.1 GLRT asymptotics: Matlab code and Python code
    • Section 3.1.2 Linear and Quadratic Discriminant Analysis: Matlab code and Python code
    • Section 3.1.1 Subspace methods: the G-MUSIC algorithm: Matlab code and Python code
  • Section 3.2 Covariance matrix distance estimation: Matlab code and Python code
  • Section 3.3 M-estimator of scatter: Matlab code and Python code
  • Section 3.4 Concluding remarks
  • Section 3.5 Practical course material:
    • The Wasserstein distance estimation: Matlab code and Python code
    • Robust portfolio optimization via Tyler estimator: Matlab code and Python code

• Chapter 4 Kernel Methods

  • Section 4.1 Basic setting
  • Section 4.2 Distance and inner-product random kernel matrices
    • Section 4.2.1 Main intuitions
    • Section 4.2.2 Main results: distance random kernel matrices: Matlab code and Python code
    • Section 4.2.3 Motivations: alpha-beta random kernel matrices
    • Section 4.2.4 Main results: alpha-beta random kernel matrices: Matlab code and Python code
  • Section 4.3 Properly scaling kernel model: Matlab code and Python code
  • Section 4.4 Implications to kernel methods
    • Section 4.4.1 Application to kernel spectral clustering: Matlab code and Python code
    • Section 4.4.2 Application to semi-supervised kernel learning: Matlab code and Python code
    • Section 4.4.3 Application to kernel ridge regression: Matlab code and Python code
    • Section 4.4.4 Summary of Section 4.4
  • Section 4.5 Concluding remarks
  • Section 4.6 Practical course material
    • Complexity-performance trade-off in spectral clustering with sparse kernel: Matlab code and Python code
    • Towards transfer learning with kernel regression: Matlab code and Python code

• Chapter 5 Large Neural Networks

  • Section 5.1 Random neural networks
    • Section 5.1.1 Regression with random neural networks: Matlab code and Python code
    • Section 5.1.2 Delving deeper into limiting kernels: Matlab code and Python code
  • Section 5.2 Gradient descent dynamics in learning linear neural nets: Matlab code and Python code
  • Section 5.3 Recurrent neural nets: echo-state networks: Matlab code and Python code
  • Section 5.4 Concluding remarks
  • Section 5.5 Practical course material: performance of large dimensional random Fourier features: Matlab code and Python code

• Chapter 6 Large Dimensional Convex Optimization

  • Section 6.1 Generalized linear classifier: Matlab code and Python code
  • Section 6.2 Large dimensional support vector machines
  • Section 6.3 Concluding remarks
  • Section 6.4 Practical course material: phase retrieval: Matlab code and Python code

• Chapter 7 Community Detection on Graphs

  • Section 7.1 Community detection in dense graphs
    • Section 7.1.1 The stochastic block model: Matlab code and Python code
    • Section 7.1.2 The degree-correlated stochastic block model: Matlab code and Python code
  • Section 7.2 From dense to sparse graphs: a different approach: Matlab code and Python code
  • Section 7.3 Concluding remarks
  • Section 7.4 Practical course material: asymptotic Gaussian fluctuations of the SBM dominant eigenvector

• Chapter 8 Universality and Real Data: Matlab code and Python code