margauxberthaud / high-dimensional-deep-learning

Science des données Saison 4 : Apprentissage en grande dimension, Données fonctionnelles, Détection d'anomalies, Introduction au Deep Learning

| Mathématiques Appliquées, Science des Données

High Dimensional and Deep Learning

Presentation :

The main theme of the course is learning methods, especially deep neural networks, for processing high dimensional data, such as signals or images. We will cover the following topics:

• Neural networks and introduction to deep learning: definition of neural networks, activation functions, multilayer perceptron, backpropagation algorithms, optimization algorithms, regularization
  Application : Implementation of a mlp with one layer with Numpy

• Convolutional neural networks: convolutional layer, pooling, dropout, convolutional network architectures (ResNet, Inception), transfer learning and fine tuning, applications for image or signal classification.
  Application : Image classification on MNIST and CatsVsDogs data with Tensorflow

• Encoder-decoder, Variational auto-encoder, Generative adversarial networks

• Functional decomposition on splines, Fourier or wavelets bases: cubic splines, penalized least squares criterion, Fourier basis, wavelet bases, applications to nonparametric regression, linear estimators and nonlinear estimators by thresholding, links with the LASSO method.

• Anomaly detection for functional data: One Class SVM, Random Forest, Isolation Forest, Local Outlier Factor. Applications to anomaly detection in functional data.

---

Objectives :

At the end of this module, the student will have understood and be able to explain (main concepts):

• Using deep learning methods for classification in high dimension
• Classification of signal and images
• Estimation of the prediction error
• Dimension reduction by projections onto orthonormal bases
• Anomaly detection
• Application of deep learning methods on real data sets

The student will be able to:

• Fit a deep neural network for signal or image classification
• Implement deep learning methods in high dimension on real data sets with the software Python or R’s libraries.

Organisation :

• Lectures : 15 H .

• Practical works : 25 H applications on real data sets with the softwares R and Python's libraries Scikit Learn and Keras -Tensorflow.

Evaluation

• written exam (50 %) - 09/12/2020

• project (oral presentation 25% - 22/01/2021 + notebook (25%)
  The main of this project is to apply the knowledge you acquired during this course by:

  • Selecting a deep learning algorithm you haven't seen in this course.
  • Explaining how this algorithm works (oral presentation).
  • Apply these algorithm on a dataset and explain their performances (notebook and oral presentation).

You can choose a deep learning algorithm among the following list.
This list is not exhaustive and you can suggest other algorithms (that's actually a good idea).
Also, the code proposed on those examples are not necessarily the official code nor the one propose by the authors.

Example of algorithms

• Detection & segmentation

  • Yolo paper, code
  • U-net paper, code
  • Mask Rcnn, Fast Rcnn, Faster Rcnn paper, code

• One shot learning

  • Siamese Network paper, code

• Outlier detection

  • Deep roc

• Style Transfer

  • A Neural Algorithm of Artistic Style paper, code
  • Cycle gan paper, code

• Generative model

  • Pixel Cnn/++ paper, code
  • Gan variation paper, code

• Anomaly detection

  • Deep robust One -Class classification paper, code