xiaochunlove / multiview_learning

Multi-view learning methods with code

 Datasets attached with the code can be found at the end of the page.

Part A: general multi-view methods with code

1. NMF (non-negative matrix factorization) based methods

 NMF factorizes the non-negative data matrix into two non-negative matrices.

• 1.1 AAAI17 Multi-View Clustering via Deep Matrix Factorization (matlab)

  • Deep Matrix Factorization is a variant of NMF.

• 1.2 ICPR16 Partial Multi-View Clustering Using Graph Regularized NMF (matlab)

• 1.3 ICDM16 Multi-View Clustering via Concept Factorization with Local Manifold Regularization (matlab)

  • Concept Factorization is a variant of NMF.

• 1.4 TC19 Individuality- and Commonality-Based Multiview Multilabel Learning (matlab)

• 1.5 AAAI14 Partial Multi-View Clustering (matlab)

• 1.6 TNNLS15 Partially Shared Latent Factor Learning With Multiview Data (matlab)

• 1.7 S18 Multi-view Discriminative Learning via Joint Non-negative Matrix Factorization (matlab)

• 1.8 ICDM13 Multi-View Clustering via Joint Nonnegative Matrix Factorization (matlab)

• 1.9 KBS20 Multi-view clustering by non-negative matrix factorization with co-orthogonal constraints (matlab)

• 1.10 KBS20 Semi-supervised Multi-view Clustering with Graph-regularized Partially Shared Non-negative Matrix Factorization (matlab)

• 1.11 NC18 Adaptive Structure Concept Factorization for Multiview Clustering (matlab)

  • Concept Factorization is a variant of NMF.

• 1.12 ICDE20 A Novel Approach to Learning Consensus and Complementary Information for Multi-View Data Clustering (matlab)

• 1.13 ECCV20 SPL-MLL: Selecting Predictable Landmarks for Multi-Label Learning (python)

• 1.14 PR20 Auto-weighted Multi-view Clustering via Deep Matrix Decomposition (matlab)

• 1.15 TKDE(Early Access) Learning Inter- and Intra-manifolds for Matrix Factorization-based Multi-Aspect Data Clustering (matlab)

• 1.16 IPM22 Co-consensus semi-supervised multi-view learning with orthogonal non-negative matrix factorization (matlab)

2. Graph based methods

 It contains two kinds of methods. The first kind is using a predefined or leaning graph (also resfer to the traditional spectral clustering), and performing post-processing spectral clustering or k-means. And the second kind is to learn the graph and the index matrix simultaneously. 

• 2.1 ICDM19 Consistency Meets Inconsistency: A Unified Graph Learning Framework for Multi-view Clustering (matlab)

• 2.2 TIP19 Multiview Consensus Graph Clustering (matlab)

• 2.3 TIP18 Auto-Weighted Multi-View Learning for Image Clustering and Semi-Supervised Classification (matlab)(python)

  • The conference variant is AAAI17 Multi-View Clustering and Semi-Supervised Classification with Adaptive Neighbours.

• 2.4 TKDE19 GMC Graph-based Multi-view Clustering (matlab)

• 2.5 BD17 Multi-View Graph Learning with Adaptive Label Propagation (matlab)

• 2.6 TC18 Graph Learning for Multiview Clustering (matlab)

• 2.7 IJCAI16 Parameter-Free Auto-Weighted Multiple Graph Learning (matlab)

• 2.8 TC18 Incomplete Multiview Spectral Clustering With Adaptive Graph Learning (matlab)

• 2.9 TKDE19 Graph structure fusion for multiview clustering (matlab)

• 2.10 ACML19 Latent Multi-view Semi-Supervised Classification (matlab)

• 2.11 NN20 Partition level multiview subspace clustering (matlab)

• 2.12 KBS20 Multi-graph Fusion for Multi-view Spectral Clustering (matlab)

• 2.13 TIP17 Flexible Multi-view Dimensionality co-Reduction (matlab)

• 2.14 ICML19 COMIC: Multi-view Clustering Without Parameter Selection (python)

• 2.15 AAAI20 Multi-View Clustering in Latent Embedding Space (matlab)

• 2.16 PR19 Multi-view Subspace Clustering with Intactness-Aware Similarity (matlab)

• 2.17 IF20 Multi-view spectral clustering via integrating nonnegative embedding and spectral embedding (matlab)

• 2.18 N19 Auto-weighted multi-view constrained spectral clustering (matlab)

• 2.19 KBS19 A Study of Graph-based System for Multi-view Clustering (matlab)

• 2.20 PR19 Auto-weighted Multi-view Clustering via Kernelized Graph Learning (matlab)

• 2.21 TKDE21 Measuring Diversity in Graph Learning: A Unified Framework for Structured Multi-view Clustering (matlab)

• 2.22 IJCAI21 Graph Filter-based Multi-view Attributed Graph Clustering (python)

• 2.23 TCYB21 Structured Graph Learning for Scalable Subspace Clustering: From Single-view to Multi-view (matlab)

• 2.24 TKDE20 Multi-View Spectral Clustering with High-Order Optimal Neighborhood Laplacian Matrix (matlab)

• 2.25 TKDE21 Consensus Graph Learning for Multi-view Clustering (matlab&python)

• 2.26 AAAI20 CGD: Multi-view Clustering via Cross-view Graph Diffusion (matlab)

• 2.27 TKDE21 Multi-view Attributed Graph Clustering (python)

• 2.28 TPAMI (Early Access) Improved Normalized Cut for Multi-view Clustering (matlab)

• 2.29 TIP22 Fast Parameter-free Multi-view Subspace Clustering with Consensus Anchor Guidance (matlab)

• 2.30 TCYB19 Dubbed Contextual Correlation Preserving Multi-View Featured Graph Clustering (matlab)

3. Self-representation based methods

 Self-representation means that each data sample is expressed by a linear combination of other samples in the same subspace.

• 3.1 AAAI18 Consistent and Specific Multi-View Subspace Clustering (matlab)

• 3.2 The method in 2.8 is also a self-representation based method.

• 3.3 PR18 Multi-view Low-rank Sparse Subspace Clustering (matlab)

• 3.4 CVPR15 Diversity-induced Multi-view Subspace Clustering (matlab)

• 3.5 TIP19 Split Multiplicative Multi-view Subspace Clustering (matlab)

• 3.6 CVPR17 Exclusivity-Consistency Regularized Multi-view Subspace Clustering (matlab)

• 3.7 TPAMI20 Generalized Latent Multi-view Subspace Clustering (matlab)

  • The conference variant is CVPR17 Latent Multi-view Subspace Clustering.

• 3.8 IS21 Multi-view Subspace Clustering via Partition Fusion (matlab)

• 3.9 TNNLS21 Multiview Subspace Clustering via Co-Training Robust Data Representation (matlab)

• 3.10 TKDE20 Consensus One-step Multi-view Subspace Clustering (matlab)

• 3.11 NCAA21 Smoothed Multi-View Subspace Clustering (matlab)

• 3.12 TCSVT21 Generalized Multi-view Collaborative Subspace Clustering (matlab)

• 3.13 AAAI22 Efficient One-pass Multi-view Subspace Clustering with Consensus Anchors (matlab)

• 3.14 IJCAI19 Flexible Multi-View Representation Learning for Subspace Clustering (matlab)

• 3.15 ICCV19 Reciprocal Multi-Layer Subspace Learning for Multi-View Clustering (matlab)

4. Tensor based methods

 The tensor is the generalization of the matrix concept. And the matrix case is a 2-order tensor.

• 4.1 TNNLS18 Multiview Subspace Clustering via Tensorial t-Product Representation (matlab)

• 4.2 ICCV15 Low-Rank Tensor Constrained Multiview Subspace Clustering (matlab)

• 4.3 TIP19 Essential tensor learning for multi-view spectral clustering (matlab)

• 4.4 IJCV20 Tensorized Multi-View Subspace Representation Learning (matlab)

  • The conference variant may be ICCV15 Low-Rank Tensor Constrained Multiview Subspace Clustering.

• 4.5 IJCV18 On Unifying Multi-view Self-Representations for Clustering by Tensor Multi-rank Minimization (matlab)

• 4.6 TCYB20 Hyper-Laplacian Regularized Multilinear Multi-View Self-Representation for Clustering and Semi-supervised Learning (matlab)

• 4.7 TCSVT21 Multi-View Spectral Clustering Tailored Tensor Low-Rank Representation (matlab)

• 4.8 TKDE20 TCCANet: Tensor Canonical Correlation Analysis Networks for Multi-view Remote Sensing Scene Recognition (matlab)

5. Kernel learning based methods

 It often uses kernel representation for each view, and then incorporates different views by seeking optimal combination of multiple kernels of different views. 

• 5.1 N18 Local kernel alignment based multi-view clustering using extreme learning machine (matlab)

• 5.2 TKDE20 Optimal Neighborhood Multiple Kernel Clustering with Adaptive Local Kernels (matlab)

6. Dictionary learning based methods

• 6.1 Access18 Multi-view Analysis Dictionary Learning for Image Classification (matlab)

• 6.2 TIP16 Multimodal Task-Driven Dictionary Learning for Image Classification(matlab)

7. Deep learning based or network based methods

   Part A 11 self-supervised learning (or contrastive learning) is also based on Deep learning.

• 7.1 TIP19 Multi-view Deep Subspace Clustering Networks (python)

• 7.2 NIPS19 CPM-Nets: Cross Partial Multi-View Networks (python)

• 7.3 AAA18 Deep Multi-View Spatial-Temporal Network for Taxi Demand Prediction (python)

• 7.4 TKDE20 MV-RNN: A Multi-View Recurrent Neural Network for Sequential Recommendation (python)

• 7.5 TIP19 Multi-View Linear Discriminant Analysis Network (python)

• 7.6 TIP19 Deep Multi-View Learning Using Neuron-Wise Correlation-Maximizing Regularizers (python)

• 7.7 ICCV15 Multi-view Convolutional Neural Networks for 3D Shape Recognition (matlab)

• 7.8 CVPR19 AE2-Nets:Autoencoder in Autoencoder Networks (python)

• 7.9 IJCAI19 Multi-view Spectral Clustering Network (python)

• 7.10 ICCV19 Reciprocal Multi-Layer Subspace Learning for Multi-View Clustering (matlab)

• 7.11 ICLR20 Learning Robust Representations via Multi-View Information Bottleneck (python)

• 7.12 SIAM19 Deep Multi-view Information Bottleneck (python)

• 7.13 TIP21 Deep Spectral Representation Learning From Multi-View Data (python)

  • The conference variant is IJCAI19 Multi-view Spectral Clustering Network (7.9).

• 7.14 TM21 Self-supervised Graph Convolutional Network For Multi-view Clustering (python)

• 7.15 BD21 CONAN: Contrastive Fusion Networks for Multi-view Clustering (python)

• 7.16 NN22 Multi-view graph embedding clustering network: Joint self-supervision and block diagonal representation (python)

8. SVM based methods

• 8.1 TNNLS18 Multiview Privileged Support Vector Machines (matlab)

• 8.2 KBS18 Multi-view learning based on Nonparallel Support Vector Machine (matlab)

• 8.3 IS19 Coupling Privileged Kernel Method for Multi-view Learning (matlab)

9. Co-training based methods

• 9.1 JMLR20 Self-paced Multi-view Co-training (python)

10. Metric Learning based methods

• 10.1 IJCAI18 FISH-MML: Fisher-HSIC Multi-View Metric Learning(matlab)

11. Self-supervised Learning based methods

• 11.1 ICLR21 Self-supervised Learning from a Multi-view Perspective (python)

• 11.2 ECCV20 Contrastive Multiview Coding (python)

• 11.3 ICML20 Contrastive Multi-View Representation Learning on Graphs (python)

• 11.4 CVPR21 Partially View-aligned Representation Learning with Noise-robust Contrastive Loss (python)

• 11.5 CVPR21 COMPLETER: Incomplete Multi-view Clustering via Contrastive Prediction (python)

• 11.6 AAAI21 Contrastive Clustering (python)

• 11.7 The method in 7.14 is also a self-supervised Learning based method.

12. Least squares regression based methods

• 12.1 PR19 Adaptive-Weighting Discriminative Regression for Multi-View Classification (matlab)

• 12.2 TC20 Multiview Classification With Cohesion and Diversity (matlab)

• 12.3 TIP17 Scalable multi-view semi-supervised classification via adaptive regression (matlab)

• 12.4 ML20 Joint Consensus and Diversity for Multi-view Semi-supervised Classification (matlab)

13. Discriminant analysis based methods

• 13.1 TPAMMI16 Multi-view discriminant analysis (matlab)

14. Boosting based methods

• 14.1 N19 Multiview Boosting by Controlling the Diversity and the Accuracy of View-specific Voters (python)

15. SNE (Stochastic Neighbour Embedding) based methods

• 15.1 arXiv21 Multi-view Data Visualisation via Manifold Learning (python)

Part B: multi-view applications with code

1. Incomplete or partial multi-view learning

 Some views of samples are missing.

• 1.1 AAAI19 Unified Embedding Alignment with Missing Views Inferring for Incomplete Multi-View Clustering (matlab)

• 1.2 ECML15 Multiple Incomplete Views Clustering via Weighted Nonnegative Matrix Factorization with L2,1 Regularization (matlab)

• 1.3 BD16 Online Multi-view Clustering with Incomplete Views (matlab)

• 1.4 IJCAI16 Incomplete Multi-Modal Visual Data Grouping (matlab)

• 1.5 TC20 Generalized Incomplete Multiview Clustering With Flexible Locality Structure Diffusion (matlab)

• 1.6 IJCAI19 Spectral Perturbation Meets Incomplete Multi-view Data (matlab)

• 1.7 TPAMI, in press, Deep Partial Multi-View Learning (python)

  • The conference variant is NIPS19 CPM-Nets: Cross Partial Multi-View Networks.

• 1.8 TPAMI20 Efficient and Effective Regularized Incomplete Multi-view Clustering (matlab)

• 1.9 TPAMI19 Late Fusion Incomplete Multi-view Clustering (matlab)

• 1.10 ICME21 Tensor-based Multi-view Block-diagonal Structure Diffusion for Clustering Incomplete Multi-view Data (matlab)

• 1.11 NeurIPS20 Partially View-aligned Clustering (python)

• 1.12 TAI22 Incomplete Multiview Clustering with Cross-view Feature Transformation (matlab)

• 1.13 AAAI22 Deep Incomplete Multi-View Clustering via Mining Cluster Complementarity (python)

• 1.14 TKDE22 Incomplete Multi-view Clustering with Sample-level Auto-weighted Graph Fusion (matlab)

2. Person Re-Identification

• 2.1 TPAMI18 Person Re-Identification by Cross-View Multi-Level Dictionary Learning (matlab)

3. Outlier detection

• 3.1 TKDD18 Multi-View Low-Rank Analysis with Applications to Outlier Detection (matlab)

• 3.2 AAAI18 Partial Multi-View Outlier Detection (matlab)

4. Zero shot learning

• 4.1 ECCV14 Transductive Multi-view Embedding for Zero-Shot Recognition and Annotation (matlab)

5. Multi-label learning or Weak-label learning

• 5.1 SIAM SDM18 Multi-view Weak-label Learning based on Matrix Completion (matlab)

  • Weak-label learning is an important branch of multi-label learning.

• 5.2 Access19 Multi-View Multi-Label Learning With View-Label-Specific Features (matlab)

• 5.3 The method in 1.4 is also a multi-label learning method.

• 5.4 IJCAI18 Incomplete Multi-View Weak-Label Learning (matlab)

• 5.5 IJCAI20 Weakly-Supervised Multi-view Multi-instance Multi-label Learning (matlab)

6. Online learning

• 6.1 ICDM16 Online Unsupervised Multi-view Feature Selection (matlab)

7. Multi-Instance learning

• 7.1 AAA19 Multi-View Multi-Instance Multi-Label Learning based on Collaborative Matrix Factorization (matlab)

8. Large-scale clustering

• 8.1 AAAI20 Large-scale Multi-view Subspace Clustering in Linear Time (matlab)

• 8.2 AAAI15 Large-scale multi-view spectral clustering via bipartite graph (matlab)

9. Non-independently and Non-identically Distributed Complex Noise

• 9.1 TNNLS19 Robust Multi-view Subspace Learning with Non-independently and Non-identically Distributed Complex Noise (matlab)

10. Multiview training boost Single-view test

• 10.1 TPAMI20 Multiview Feature Selection for Single-view Classification (matlab)

11. Fuzzy clustering

• 11.1 PR21 Collaborative feature-weighted multi-view fuzzy c-means clustering (matlab)

12. Trusted (or reliable) learning

• 12.1 ICLR21 Trusted Multi-View Classification (python)

• 12.2 AAAI18 Reliable multi-view clustering (matlab)

13. 3D point cloud registration

• 13.1 CVPR20 Learning multiview 3D point cloud registration (python)

14. Recommendation

• 14.1 SIGIR20 MVIN: Learning Multiview Items for Recommendation (python)

15. Shape Reconstruction

• 15.1 MIPS19 Multiview Aggregation for Learning Category-Specific Shape Reconstruction (python)

16. 3D Object Recognition

• 16.1 CVPR20 Views Self Supervised and Regularized Learning for 3D Object Recognition (python)

17. Remote Sensing Scene Representations

• 17.1 CVPR21 Self-Supervised Learning of Remote Sensing Scene Representations Using Contrastive Multiview Coding (python)

18. View generation

• 18.1 ECCV18 Multi-view to Novel view: Synthesizing Novel Views with Self-Learned Confidence (python)

19. Dialog Intent Induction

• 19.1 EMNLP19 Dialog Intent Induction with Deep Multi-View Clustering (python)

20. Partially View-unaligned Problem

• 20.1 TPAMI(Early Access) Robust Multi-view Clustering with Incomplete Information (python)

21. Cancer Subtype Identification

• 21.1 TCBB22 Multi-view Robust Graph-based clustering for Cancer Subtype Identification (matlab)

Part C: Others

• 1.1 Multi-view dataset

    example: dataset name (the mark refers to the paper listed above)
  

  • yale_mtv (Part A 1.1)

  • 20newsgroups, 3sources, BBC4view_685, BBCSport2view_544, WikipediaArticles (Part A 1.3)

• 1.2 famous authors in the field of multi-view learning

    private opinions
  

  • Feiping Nie (Google Scholar Citations) (home)

• 1.3 other github pages with code

  • awesome multi-view clustering (page)