anwarvic / mathematics-for-ml-specialization

This is the solution of the "Mathematics for Machine Learning Specialization" made by Coursera

Mathematics-for-Machine-Learning-Specialization

This repo contains my solution to the 3-course Coursera Mathematics for Machine Learning Specialization created by Imperial College in London

In this repo you can find:

• The original assignments without solution.
• The original quizzes without solution.
• My solution to the assignment.
• My solution to the quizzes.

About this Specialization

For a lot of higher level courses in Machine Learning and Data Science, you find you need to freshen up on the basics in mathematics - stuff you may have studied before in school or university, but which was taught in another context, or not very intuitively, such that you struggle to relate it to how it’s used in Computer Science. This specialization aims to bridge that gap, getting you up to speed in the underlying mathematics, building an intuitive understanding, and relating it to Machine Learning and Data Science.

In the first course on Linear Algebra we look at what linear algebra is and how it relates to data. Then we look through what vectors and matrices are and how to work with them.

The second course, Multivariate Calculus, builds on this to look at how to optimize fitting functions to get good fits to data. It starts from introductory calculus and then uses the matrices and vectors from the first course to look at data fitting.

The third course, Dimensionality Reduction with Principal Component Analysis, uses the mathematics from the first two courses to compress high-dimensional data. This course is of intermediate difficulty and will require Python and numpy knowledge.

At the end of this specialization you will have gained the prerequisite mathematical knowledge to continue your journey and take more advanced courses in machine learning.

Specialization Courses

There are 3 Courses in this Specialization:

• Mathematics for Machine Learning: Linear Algebra: In this course on Linear Algebra we look at what linear algebra is and how it relates to vectors and matrices. Then we look through what vectors and matrices are and how to work with them, including the knotty problem of eigenvalues and eigenvectors, and how to use these to solve problems. Finally we look at how to use these to do fun things with datasets - like how to rotate images of faces and how to extract eigenvectors to look at how the Pagerank algorithm works.

  Since we're aiming at data-driven applications, we'll be implementing some of these ideas in code, not just on pencil and paper. Towards the end of the course, you'll write code blocks and encounter Jupyter notebooks in Python, but don't worry, these will be quite short, focussed on the concepts, and will guide you through if you’ve not coded before. At the end of this course you will have an intuitive understanding of vectors and matrices that will help you bridge the gap into linear algebra problems, and how to apply these concepts to machine learning.

• Mathematics for Machine Learning: Multivariate Calculus This course offers a brief introduction to the multivariate calculus required to build many common machine learning techniques. We start at the very beginning with a refresher on the “rise over run” formulation of a slope, before converting this to the formal definition of the gradient of a function. We then start to build up a set of tools for making calculus easier and faster. Next, we learn how to calculate vectors that point up hill on multidimensional surfaces and even put this into action using an interactive game. We take a look at how we can use calculus to build approximations to functions, as well as helping us to quantify how accurate we should expect those approximations to be. We also spend some time talking about where calculus comes up in the training of neural networks, before finally showing you how it is applied in linear regression models. This course is intended to offer an intuitive understanding of calculus, as well as the language necessary to look concepts up yourselves when you get stuck. Hopefully, without going into too much detail, you’ll still come away with the confidence to dive into some more focused machine learning courses in future.

• Mathematics for Machine Learning: PCA This intermediate-level course introduces the mathematical foundations to derive Principal Component Analysis (PCA), a fundamental dimensionality reduction technique. We'll cover some basic statistics of data sets, such as mean values and variances, we'll compute distances and angles between vectors using inner products and derive orthogonal projections of data onto lower-dimensional subspaces. Using all these tools, we'll then derive PCA as a method that minimizes the average squared reconstruction error between data points and their reconstruction.

  At the end of this course, you'll be familiar with important mathematical concepts and you can implement PCA all by yourself. If you’re struggling, you'll find a set of jupyter notebooks that will allow you to explore properties of the techniques and walk you through what you need to do to get on track. If you are already an expert, this course may refresh some of your knowledge. The lectures, examples and exercises require: 1. Some ability of abstract thinking 2. Good background in linear algebra (e.g., matrix and vector algebra, linear independence, basis) 3. Basic background in multivariate calculus (e.g., partial derivatives, basic optimization) 4. Basic knowledge in python programming and numpy Disclaimer: This course is substantially more abstract and requires more programming than the other two courses of the specialization. However, this type of abstract thinking, algebraic manipulation and programming is necessary if you want to understand and develop machine learning algorithms.

mathematics-for-ml-specialization's Issues

PCA/week 2/General inner products: lengths and distances/question 1

The correct answer to PCA/week 2/General inner products: lengths and distances/question 1 is sqrt(26) and nor sqrt(29)

Pass a programming assignment

For some reason I can't get any programming assignments to pass for PCA. Can you help?

Grader fails on all tests, seems like type info is getting lost, isfinite datatype check can't be applied.

The graded function specifically returns:

    return covariance_matrix.astype('float64')

not sure how the grader changes the type, this is failing for all 4 tests.

======================================================================
ERROR: test_cov (week1_tests.Test)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/tmp/autograde_pnpmxbbt/week1_tests.py", line 32, in test_cov
    np.testing.assert_allclose(result, cov(X))
  File "/opt/conda/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 1533, in assert_allclose
    verbose=verbose, header=header, equal_nan=equal_nan)
  File "/opt/conda/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 794, in assert_array_compare
    val = comparison(x, y)
  File "/opt/conda/lib/python3.7/site-packages/numpy/testing/_private/utils.py", line 1528, in compare
    equal_nan=equal_nan)
  File "<__array_function__ internals>", line 6, in isclose
  File "/opt/conda/lib/python3.7/site-packages/numpy/core/numeric.py", line 2257, in isclose
    xfin = isfinite(x)
TypeError: ufunc 'isfinite' not supported for the input types, and the inputs could not be safely coerced to any supported types according to the casting rule ''safe''

It appears to fail for functions defined in this notebook here as well.

PCA Week 1: Mean/Covariance of a data set and effect of a linear transformation

(shapes (2, 1), (3,) mismatch)
x: array([[0.666667],
[1.333333]])
y: array([0.5, 1.5, 1. ])

def mean(X):
D, N = X.shape
mean = np.zeros((D,1))
mean = np.mean(X, axis=1, keepdims=True)
return mean