limapsd / ltcc-advanced-computational-methods-in-statistics

LTCC-Course

Advanced Computational Methods in Statistics

This course will provide an overview of Monte Carlo methods when used for problems in Statistics. After an introduction to simulation, its purpose and challenges, we will cover in more detail Importance Sampling, Markov Chain Monte Carlo and Sequential Monte Carlo. Whilst the main focus will be on the methodology and its relevance to applications, we will often mention relevant theoretical results and their importance for problems in practice.

Keywords: Simulation, Variance reduction, Monte Carlo methods, Importance Sampling, Markov Chain Monte Carlo, Sequential Monte Carlo, Particle filters, Stochastic filtering, Hidden Markov models.

Part of the material below is delivered every year as part of the London Taught Course Centre (LTCC) for PhD students in mathematical sciences. Here is the official link of the course: http://www.ltcc.ac.uk/courses/advanced-computational-methods-in-statistics/

The lecture notes can be found here and the Matlab code used in the examples is provided above.

Immediately below you can find the course outline together the with links to slides and video-lectures (~20hrs). I have listed all the video lectures in this playlist. At the bottom of this page you can find more details on assessment, reading list, prerequisites, delivery format for the LTCC course.

Introduction

We will split the introduction into two parts:

• Introduction to Monte Carlo slides and video

• Basic Indirect Sampling methods (Rejection Sampling and Importance Sampling) slides and video

Markov Chain Monte Carlo (MCMC)

We will mainly discuss various topics and provide some some basics on theory and practice.

• Introduction to Markov Chain Monte Carlo
  • Metropolis-Hastings, Gibbs sampling, diagnostics
  • video and slides
• Some more methodology
  • Computing the normalising constant, Adaptive MCMC, Pseudo marginal MCMC
  • video and slides
• Theoretical topics
  • Understanding MCMC from basics of Markov Chain theory
  • Diffusions, MCMC and the 0.234 rule of thumb
  • video and slides and scribles

Hidden Markov models and the filtering problem

• Hidden Markov models and filtering: slides and video

• The Kalman filter and extensions: slides and video

Sequential Monte Carlo (SMC)

• Sequential Importance sampling: slides and video

• Introduction to Particle filtering: slides and video

• Some extensions to the basic Particle filter

  • adaptive resampling, resample move and auxiliary particle filters
  • slides and video

• Particle Smoothing: slides and video

• Parameter estimation methods for static parameters of Hidden Markov models

  • Bayesian methods and particle MCMC: slides and video
  • likelihood estimation: slides and video

• SMC sampling for fixed dimensional state spaces: slides and video

LTCC Course organisation and assessment

Delivery

The course will be delivered in person at Hardy room in De Morgan House every Monday 10:50-12:50 from 7 November to 5 December 2022.

The rescheduled lecture from 21/11 will take place on Monday the 12th December at Imperial College, room 144 Huxley Building, at 12-2pm.

Huxley Building has an entrance from 180 Queensgate. See here for more details on how to get to the building and here for a floorplan. Once you reach the straircase or lift head towards floor 1 at the basement.

Registration is compulsory, please vist http://www.ltcc.ac.uk/registration/

Coursework

You may find the 2022 coursework instructions here

Deadline: around 18 January (about a month)

Page limit: 10 pages, recommended length around 6-8 pages

Submit by email to n.kantas at imperial.ac.uk using subject: LTCC coursework submission

References

Relevant introductory graduate textbooks and edited volumes:

• Chopin and Papaspiliopoulos (2020). An introduction to sequential Monte Carlo, Springer Series in Statistics.

• R. Douc, E. Moulines, & D. S. Stoffer (2014). Nonlinear Time Series Theory, Methods and Applications with R Examples, CRC Press.

• S. Sarkka (2013) Bayesian filtering and smoothing, CUP Cambridge.

• Cappé, O., Moulines, E. and Rydén, T. (2005). Inference in Hidden Markov Models. New York: Springer-Verlag.

• Doucet, de Freitas, Gordon (2001) Sequential Monte Carlo Methods in Practice, Springer.

• Liu (2001) Monte Carlo strategies in scientific computing, Springer.

• Robert and Casella (1999) Monte Carlo Statistical Methods, Springer.

• Gillks, Richardson, Spiegelhalter (1996) Markov Chain Monte Carlo in Practice, Chapman Hall.

Prerequisites:

• Basic knowledge of Statistics and Probability.

• Basic knowledge of programming in any language appropriate for scientific computing.

• Familiarity and exposure to Markov Chains or stochastic processes will be useful.

Format:

• There will be optional exercises or small courseworks posed as quizes or homeworks. There will be no separate problem sheets. Some problems will require the use of some programming.

• Lecture/computer session/tutorial/discussion hours split: 10/ 0 /0 /0

Lecturer contact details:

• Nikolas Kantas, Imperial College

• Email: n.kantas at imperial.ac.uk

• website: http://wwwf.imperial.ac.uk/~nkantas/