jermwatt / machine_learning_refined

Notes, examples, and Python demos for the 2nd edition of the textbook "Machine Learning Refined" (published by Cambridge University Press).

Machine Learning Refined: Notes, Exercises, Presentations, and Sample Chapters

Below you will find a range of resources - including extensive interactive Collab / Jupyter notebooks - that complement the 2nd edition of Machine Learning Refined (published by Cambridge University Press).

Table of Contents

• Sample chapters from the 2nd edition
• A sampler of widgets / pedagogy
• Online notes (jupyter notebooks)
• What is new in the second edition?
• How to use the book
• Technical prerequisites
• Coding exercises
• Slides and additional instructor resources
• Errata
• Get a copy of the book
• Reviews and Endorsements
• Software installation and dependencies
• Contact

A sampler of widgets and our pedagogy

(Back to top)

We believe mastery of a certain machine learning concept/topic is achieved only when the answer to each of the following three questions is affirmative.

1. Intuition Can you describe the idea with a simple picture?
2. Mathematical derivation Can you express your intuition in mathematical notation and derive underlying models/cost functions?
3. Implementation Can you code up your derivations in a programming language, say Python, without using high-level libraries?

Intuition comes first. Intuitive leaps precede intellectual ones, and because of this we have included over 300 color illustrations in the book that have been meticulously designed to enable an intuitive grasp of technical concepts. Many of those illustrations are snapshots of animations that show convergence of certain algorithms, evolution of certain models from underfitting all the way to overfitting, etc. This sort of concepts can be illustrated and intuited best using animations (as opposed to static figures). You'll find a large number of such animations in this repository -- which you can modify yourself too via the raw Jupyter notebook version of these notes. Here are just a few examples:

Cross-validation (regression)	Cross-validation (two-class classification)	Cross-validation (multi-class classification)

K-means clustering	Feature normalization	Normalized gradient descent

Rotation	Convexification	Dogification!

A nonlinear transformation	Weighted classification	The moving average

Batch normalization	Logistic regression

Polynomials vs. NNs vs. Trees (regression)	Polynomials vs. NNs vs. Trees (classification)

Changing gradient descent's steplength (1d)	Changing gradient descent's steplength (2d)

Convex combination of two functions	Taylor series approximation

Feature selection via regularization	Secant planes

Function approximation with a neural network	A regression tree

Mathematical optimization: the workhorse of machine learning. We highly emphasize the importance of mathematical optimization in our treatment of machine learning. Optimization is the workhorse of machine learning and is fundamental at many levels – from the tuning of individual models to the general selection of appropriate nonlinearities via cross-validation. Because of this a strong understanding of mathematical optimization is requisite if one wishes to deeply understand machine learning, and if one wishes to be able to implement fundamental algorithms. Part I of the book provides a complete introduction to mathematical optimization, covering zero-, first-, and second-order methods, that are relied upon later in deriving and tuning machine learning models.

Learning by doing. We place significant emphasis on the design and implementation of algorithms throughout the text with implementations of fundamental algorithms given in Python. These fundamental examples can then be used as building blocks for the reader to help complete the text’s programming exercises, allowing them to ”get their hands dirty” and ”learn by doing,” practicing the concepts introduced in the body of the text. While in principle any programming language can be used to complete the text’s coding exercises, we highly recommend using Python for its ease of use and large support community. We also recommend using the open-source Python libraries NumPy, autograd, and matplotlib, as well as the Jupyter notebook editor to make implementing and testing code easier. A complete set of installation instructions, datasets, as well as starter notebooks can be found in this repository.

Collab / Jupyter notes

(Back to top)

A select number of Chapters/Sections are highlighted below and are linked to fully interactive Collab / Jupyter notebooks that served as early drafts for the second edition of the textbook. You can find these html files as well as Jupyter notebooks which created them in the notes subdirectory.

Chapter 1. Introduction to Machine Learning

1.1 Introduction
1.2 Distinguishing Cats from Dogs: a Machine Learning Approach
1.3 The Basic Taxonomy of Machine Learning Problems
1.4 Mathematical Optimization
1.5 Conclusion

Chapter 2. Zero-Order Optimization Techniques

2.1 Introduction
2.2 The Zero-Order Optimality Condition
2.3 Global Optimization Methods
2.4 Local Optimization Methods
2.5 Random Search
2.6 Coordinate Search and Descent
2.7 Conclusion
2.8 Exercises

Chapter 3. First-Order Optimization Techniques

3.1 Introduction
3.2 The First-Order Optimality Condition
3.3 The Geometry of First-Order Taylor Series
3.4 Computing Gradients Efficiently
3.5 Gradient Descent
3.6 Two Natural Weaknesses of Gradient Descent
3.7 Conclusion
3.8 Exercises

Chapter 4. Second-Order Optimization Techniques

4.1 The Second-Order Optimality Condition
4.2 The Geometry of Second-Order Taylor Series
4.3 Newton’s Method
4.4 Two Natural Weaknesses of Newton’s Method
4.5 Conclusion
4.6 Exercises

Chapter 5. Linear Regression

5.1 Introduction
5.2 Least Squares Linear Regression
5.3 Least Absolute Deviations
5.4 Regression Quality Metrics
5.5 Weighted Regression
5.6 Multi-Output Regression
5.7 Conclusion
5.8 Exercises
5.9 Endnotes: Probabilistic interpretation of linear regression \

Chapter 6. Linear Two-Class Classification

6.1 Introduction
6.2 Logistic Regression and the Cross Entropy Cost
6.3 Logistic Regression and the Softmax Cost
6.4 The Perceptron
6.5 Support Vector Machines
6.6 Which Approach Produces the Best Results?
6.7 The Categorical Cross Entropy Cost
6.8 Classification Quality Metrics
6.9 Weighted Two-Class Classification
6.10 Conclusion
6.11 Exercises

Chapter 7. Linear Multi-Class Classification

7.1 Introduction
7.2 One-versus-All Multi-Class Classification
7.3 Multi-Class Classification and the Perceptron
7.4 Which Approach Produces the Best Results?
7.5 The Categorical Cross Entropy Cost Function
7.6 Classification Quality Metrics
7.7 Weighted Multi-Class Classification
7.8 Stochastic and Mini-Batch Learning
7.9 Conclusion
7.10 Exercises

Chapter 8. Linear Unsupervised Learning

8.1 Introduction
8.2 Fixed Spanning Sets, Orthonormality, and Projections
8.3 The Linear Autoencoder and Principal Component Analysis
8.4 Recommender Systems
8.5 K-Means Clustering
8.6 General Matrix Factorization Techniques
8.7 Conclusion
8.8 Exercises
8.9 Endnotes

Chapter 9. Feature Engineering and Selection

9.1 Introduction
9.2 Histogram Features
9.3 Feature Scaling via Standard Normalization
9.4 Imputing Missing Values in a Dataset
9.5 Feature Scaling via PCA-Sphering
9.6 Feature Selection via Boosting
9.7 Feature Selection via Regularization
9.8 Conclusion
9.9 Exercises

Chapter 10. Principles of Nonlinear Feature Engineering

10.1 Introduction \
10.2 Nonlinear Regression
10.3 Nonlinear Multi-Output Regression
10.4 Nonlinear Two-Class Classification
10.5 Nonlinear Multi-Class Classification
10.6 Nonlinear Unsupervised Learning
10.7 Conclusion
10.8 Exercises

Chapter 11. Principles of Feature Learning

11.1 Introduction
11.2 Universal Approximators
11.3 Universal Approximation of Real Data
11.4 Naive Cross-Validation
11.5 Efficient Cross-Validation via Boosting
11.6 Efficient Cross-Validation via Regularization
11.7 Testing Data
11.8 Which Universal Approximator Works Best in Practice?
11.9 Bagging Cross-Validated Models
11.10 K-Fold Cross-Validation
11.11 When Feature Learning Fails
11.12 Conclusion
11.13 Exercises

Chapter 12. Kernel Methods

12.1 Introduction
12.2 Fixed-Shape Universal Approximators
12.3 The Kernel Trick
12.4 Kernels as Measures of Similarity
12.5 Optimization of Kernelized Models
12.6 Cross-Validating Kernelized Learners
12.7 Conclusion
12.8 Exercises

Chapter 13. Fully Connected Neural Networks

13.1 Introduction
13.2 Fully Connected Neural Networks
13.3 Activation Functions
13.4 The Backpropagation Algorithm
13.5 Optimization of Neural Network Models
13.6 Batch Normalization
13.7 Cross-Validation via Early Stopping
13.8 Conclusion
13.9 Exercises

Chapter 14. Tree-Based Learners

14.1 Introduction
14.2 From Stumps to Deep Trees
14.3 Regression Trees
14.4 Classification Trees
14.5 Gradient Boosting
14.6 Random Forests
14.7 Cross-Validation Techniques for Recursively Defined Trees
14.8 Conclusion
14.9 Exercises

Appendix A. Advanced First- and Second-Order Optimization Methods

A.1 Introduction
A.2 Momentum-Accelerated Gradient Descent
A.3 Normalized Gradient Descent
A.4 Advanced Gradient-Based Methods
A.5 Mini-Batch Optimization
A.6 Conservative Steplength Rules
A.7 General Steepest Descent
A.8 Newton’s Method, Regularization, and Nonconvex Functions
A.9 Hessian-Free Methods

Appendix B. Derivatives and Automatic Differentiation

B.1 Introduction
B.2 The Derivative
B.3 Derivative Rules for Elementary Functions and Operations
B.4 The Gradient
B.5 The Computation Graph
B.6 The Forward Mode of Automatic Differentiation
B.7 The Reverse Mode of Automatic Differentiation
B.8 Higher-Order Derivatives
B.9 Taylor Series
B.10 Using the autograd Library

Appendix C. Linear Algebra

C.1 Introduction
C.2 Vectors and Vector Operations
C.3 Matrices and Matrix Operations
C.4 Eigenvalues and Eigenvectors
C.5 Vector and Matrix Norms

What is new in the second edition?

(Back to top)

The second edition of this text is a complete revision of our first endeavor, with virtually every chapter of the original rewritten from the ground up and eight new chapters of material added, doubling the size of the first edition. Topics from the first edition, from expositions on gradient descent to those on One-versusAll classification and Principal Component Analysis have been reworked and polished. A swath of new topics have been added throughout the text, from derivative-free optimization to weighted supervised learning, feature selection, nonlinear feature engineering, boosting-based cross-validation, and more. While heftier in size, the intent of our original attempt has remained unchanged: to explain machine learning, from first principles to practical implementation, in the simplest possible terms.

How to use the book?

(Back to top)

Example ”roadmaps” shown below provide suggested paths for navigating the text based on a variety of learning outcomes and university courses taught using the present book.

Recommended study roadmap for a course on the essentials of machine learning, including requisite chapters (left column), sections (middle column), and corresponding topics (right column). This essentials plan is suitable for time-constrained courses (in quarter-based programs and universities) or self-study, or where machine learning is not the sole focus but a key component of some broader course of study.

Recommended study roadmap for a full treatment of standard machine learning subjects, including chapters, sections, as well as corresponding topics to cover. This plan entails a more in-depth coverage of machine learning topics compared to the essentials roadmap given above, and is best suited for senior undergraduate/early graduate students in semester-based programs and passionate independent readers.

Recommended study roadmap for a course on mathematical optimization for machine learning and deep learning, including chapters, sections, as well as topics to cover.

Recommended study roadmap for an introductory portion of a course on deep learning, including chapters, sections, as well as topics to cover.

Technical prerequisites

(Back to top)

To make full use of the text one needs only a basic understanding of vector algebra (mathematical functions, vector arithmetic, etc.) and computer programming (for example, basic proficiency with a dynamically typed language like Python). We provide complete introductory treatments of other prerequisite topics including linear algebra, vector calculus, and automatic differentiation in the appendices of the text.

Coding exercises

(Back to top)

In the mlrefined_exercises directory you can find starting wrappers for coding exercises from the first and second editions of the text.

Slides and additional instructor resources

(Back to top)

Slides for the 2nd edition of the text are available in pptx, jupyter, and reveal.js formats. Slides for the 1st edition of the text are also available.

Instructors may request a copy of this text for examination from the publisher's website. Cambridge University Press can also provide you with the solution manual to both editions of the text.

Errata

(Back to top)

Here you can find a regularly updated errata sheet for the second edition of the text. Please report any typos, bugs, broken links, etc., in the Issues Section of this repository or by contacting us directly via email (see contact section for more info).

Get a copy of the book

(Back to top)

• Free sample chapters in pdf format
• From Cambridge University Press
• From Amazon
• From Barnes & Noble
  
  

Reviews and Endorsements

(Back to top)

An excellent book that treats the fundamentals of machine learning from basic principles to practical implementation. The book is suitable as a text for senior-level and first-year graduate courses in engineering and computer science. It is well organized and covers basic concepts and algorithms in mathematical optimization methods, linear learning, and nonlinear learning techniques. The book is nicely illustrated in multiple colors and contains numerous examples and coding exercises using Python.

John G. Proakis, University of California, San Diego

Some machine learning books cover only programming aspects, often relying on outdated software tools; some focus exclusively on neural networks; others, solely on theoretical foundations; and yet more books detail advanced topics for the specialist. This fully revised and expanded text provides a broad and accessible introduction to machine learning for engineering and computer science students. The presentation builds on first principles and geometric intuition, while offering real-world examples, commented implementations in Python, and computational exercises. I expect this book to become a key resource for students and researchers.

Osvaldo Simeone, King's College, London

This book is great for getting started in machine learning. It builds up the tools of the trade from first principles, provides lots of examples, and explains one thing at a time at a steady pace. The level of detail and runnable code show what's really going when we run a learning algorithm.

David Duvenaud, University of Toronto

This book covers various essential machine learning methods (e.g., regression, classification, clustering, dimensionality reduction, and deep learning) from a unified mathematical perspective of seeking the optimal model parameters that minimize a cost function. Every method is explained in a comprehensive, intuitive way, and mathematical understanding is aided and enhanced with many geometric illustrations and elegant Python implementations.

Kimiaki Sihrahama, Kindai University, Japan

Books featuring machine learning are many, but those which are simple, intuitive, and yet theoretical are extraordinary 'outliers'. This book is a fantastic and easy way to launch yourself into the exciting world of machine learning, grasp its core concepts, and code them up in Python or Matlab. It was my inspiring guide in preparing my 'Machine Learning Blinks' on my BASIRA YouTube channel for both undergraduate and graduate levels.

Islem Rekik, Director of the Brain And SIgnal Research and Analysis (BASIRA) Laboratory

Software installation and dependencies

(Back to top)

Google Collab !

The majority of the notes and exercise wrappers in this repository can be run without the need to install anything locally - for free on Google Collab. Click the Collab sticker at the top of a notebook to open it in Collab.

Running locally

After cloning this repository and entering the directory we recommend one of three methods for successfully running the Jupyter notebooks contained therein.

Docker method (strongly recommended)

After installing docker and docker-compose on your machine
traverse to this repo at your terminal and type

docker-compose up -d

When running this command the first time an associated docker image is pulled from DockerHub.

Then in any web browser go to

localhost:8888

to view the repository contents - including jupyter notebooks.

Anaconda method

After installing Anaconda Python 3 distribution on your machine, cd into this repo's directory and follow these steps to create a conda virtual environment to view its contents and notebooks.

First, create the environment

conda create python=3.6 --name mlr2 --file requirements.txt

Then activate it

conda activate mlr2

Run jupyter via the command below

jupyter notebook --port=8888 --ip=0.0.0.0 --allow-root --NotebookApp.token=''

And finally, open any web browser and traverse to

localhost:8888

to view the repository contents - including jupyter notebooks.

pip method

Using Python3 and pip on your machine, cd into this repo's directory and follow these steps to install the required packages.

First install Python requirements

pip install -r requirements.txt

Run jupyter via the command below

jupyter notebook --port=8888 --ip=0.0.0.0 --allow-root --NotebookApp.token=''

And finally, open any web browser and traverse to

localhost:8888

to view the repository contents - including jupyter notebooks.

Contact

(Back to top)

This repository is in active development by Jeremy Watt and Reza Borhani. Please do not hesitate to reach out with comments, questions, typos, etc.

machine_learning_refined's Issues

Exercise 2.1 - Uniform sampling

I couldn't find any reference in the errata, hence may be it is not an error - but how does someone sample $K$ points from $[-1,1]^N$ when $K$ is not $N-th$ power? From reading the text and general knowledge, I thought that uniform sampling meant creating a $N$ dimensional grid of evenly spaced points - but this is obviously not possible when the dimension is not $1$ or $2$ when $N=100$. Does it mean something else? If it does - could it be specified explicitly in the errata?

solution to exercises?

Hello there! This is a really great book! However I am not able to find the solution to the mathematical exercises... Where are they?

Thanks!

Missing 2-4 chapters' exercise folders for 2nd edition

Hello!

I have just started to read the book and wanted to try out some exercises. But there is only information about execrises starting from chapter 5 in here. So, I just wanted to ask, if they were supposed to be there originally, or not.

Machine Learning 101

git lfs pull not working

When I try git lfs pull, I get:

batch response: This repository is over its data quota. Account responsible for LFS bandwidth should purchase more data packs to restore access.
error: failed to fetch some objects from 'https://github.com/jermwatt/machine_learning_refined.git/info/lfs'

ModuleNotFoundError: No module named 'autograd.misc'

I run this code at the terminal: pip install autograd
but when I run the the below cell:
import sys
sys.path.append('../../')
import matplotlib.pyplot as plt
from mlrefined_libraries import basics_library as baslib
from mlrefined_libraries import linear_algebra_library as linlib

Then I got the below error:

ModuleNotFoundError Traceback (most recent call last)
in ()
4 sys.path.append('../../')
5 import matplotlib.pyplot as plt
----> 6 from mlrefined_libraries import basics_library as baslib
7 from mlrefined_libraries import linear_algebra_library as linlib

/Volumes/DATA/projects/private/ml/mlrefined/mlrefined_libraries/init.py in ()
7 prefix=name+'.',
8 onerror=(lambda x: None)):
----> 9 import(modname)
10

/Volumes/DATA/projects/private/ml/mlrefined/mlrefined_libraries/convnets_library/convnet_lib/optimizers.py in ()
2 from autograd import value_and_grad
3 from autograd import hessian
----> 4 from autograd.misc.flatten import flatten_func
5 from IPython.display import clear_output
6 from timeit import default_timer as timer

ModuleNotFoundError: No module named 'autograd.misc'

I am using Anaconda with Python 3.6, I have no idea how to fix this issue. Can you check?

conda build failed with matplotlib pinned to v3.1.0 (why OLD version)? Will attempt to float all dependencies to latest...

the conda build method did not work using the requirements.txt file

conda create python=3.8 --name mlr2 --file requirements.txt

The latest version of matplotlib is 3.5.2.
v3.1.0 is obsolete has a dependency on some outdated package.

Is there some reason that matplotlib is pinned to 3.1.0 and all the other package dependencies are allowed to float to the latest versions?

Below is my conda output. Initially I tried it using the command on the githut package, but conda failed with a lot of incompatiblity conflicts.

Then I tried conda install for all packages EXCEPT the pinned matplotlib=3.1.0 and everything worked. Next, I tried to install matplotlib=3.1.0 and got the output below.

(mlr2) C:\Users\richadmin\Desktop\00_PythonWIP_2022\02-Borhani-ML-Refined-2021>conda install matplotlib==3.1.0
Collecting package metadata (current_repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.
Collecting package metadata (repodata.json): done
Solving environment: failed with initial frozen solve. Retrying with flexible solve.  
Solving environment: - Found conflicts! Looking for incompatible packages.
This can take several minutes.  Press CTRL-C to abort.
failed

UnsatisfiableError: The following specifications were found
to be incompatible with the existing python installation in your environment:

Specifications:

  - matplotlib==3.1.0 -> python[version='>=3.6,<3.7.0a0|>=3.7,<3.8.0a0']

Your python: python=3.8

If python is on the left-most side of the chain, that's the version you've asked for.  When python appears to the right, that indicates that the thing on the left is somehow not available for the python version you are constrained to. Note that conda will not change your python version to a different minor version unless you explicitly specify that.

I just installed into the conda environment with everything the latest version (Python 3.8 and all other packages in the requirements.txt file) and then matplotlib (floating to the LATEST VERSION) and it worked. I will try running the code examples and see if there are any bugs or errors due to the later version of matplotlib.

I will report back after I have gone through many of the notebooks.

Solutions to exercises on Cambridge Resources site missing

Dear authors,
Thanks for a very nice book, which we are adopting for a new class starting in Feb'21. It seems the Solutions at the Instructor Resources on the Cambridge Higher Education site are missing. There is a directory with zip files, but they contain the jpg images of the book (duplicate of the Figures directory).

Missing initialization in Exercise 6.5 MATLAB wrapper

In the MATLAB wrapper for Exercise 6.5, the parameters b, w, c, and V used in the tanh_softmax function are not initialized. The code will not successfully run then even if grad_b, grad_w, grad_c, and grad_V are written correctly.

Great work!

Your work build a bridge between machine learning and math, and provide a good intuition for machine learning. So when will your second book come out?

Chapter 2 exercises file is not rendering correctly

When opening the jupyter file ,it does not render in the github viewer correctly. It's very difficult to read.

Automatic Differentiation is not available

Automatic Differentiation Section (chapter 2) is not available on the below website anymore. In what order should the website be read now. https://jermwatt.github.io/mlrefined/index.html

how to get the solution?

pip install error-exist no requirements.txt: Mac Version 10.13.2

I followed the instructions from readme.md and tried to install required packages as follows:

  git clone https://github.com/jermwatt/mlrefined
  cd mlrefined
  pip install -r requirements.txt

and Error showed saying that

Could not open requirements file: [Errno 2] No such file or directory: 'requirements.txt'

And I couldn't find requirements.txt using search as well.

​Why don't it have code and source of the first version?​

Text errors/typos in chapter 6, 2nd edition

Some suspected errors/typos to report:

p.131. In example 6.3, "Here a of run normalized gradient descent ..." should be "Here a run of normalized gradient descent ...".
p.136. Equation 6.18 is not consistent with how tanh is usually defined. The consistent one should be "tanh(x) = 2 sigma(2x) - 1".
p.145. The inequality "exp(-yxCw) = exp(-C)exp(-yxw) < exp(-yxw)" is incorrect. It should be "exp(-yxCw) = (exp(-yxw))^C < exp(-yxw)"
p.158. The encoding in equation 6.62 does not make equation 6.66 and equation 6.67 equivalent to each other. Instead, the encoding should be done reversely: encode the label 1 into the vector [1 0]. (If consistency with section 7.5.4 is intended, then some other contents, instead of the encoding, need modification.)
p.168. "Section 6.24" is mentioned, but there is no such section.

I'm not confident that these are all errors/typos indeed. Hope they help.
Besides, this is a great machine learning book for beginners. I do understand what's going on in the text. Deep appreciation for the effort.

downloading the repository in the form of zip

Hello
After today's update, the folder "https://github.com/jermwatt/machine_learning_refined/tree/gh-pages/sample_chapters" files are not downloaded correctly by downloading the repository in the form of zip.

Supplementary videos

I have enjoyed working through the 2nd edition of the book, and I commend you for creating a machine learning book with such a good balance of intuition and rigor. I wondered if it would be viable down the road to include supplementary videos of the sections of the book? Something alongside the excellent videos by Caltech, which I will link below. Thanks again for the hard work!

https://home.work.caltech.edu/lectures.html

Chapter 3.0 page links to Chapter 6.0

Hey Professors,

The Chapter 3 Section 0 link on the index.html file leads to a page titled "6.0 Introduction". However, the contents of that page make sense for the introduction of Chapter 3. I think it could be that the title is incorrect whereas the contents are correct

Figure numbering skips 6 in section 14.2

https://jermwatt.github.io/mlrefined/blog_posts/14_Convolutional_networks/14_2_Edge_histogram_based_features.html

Looks like it jumps from Figure 5 right to Figure 7

Images on 3.10 don't display (despite src being correct)

The tags show instead:
For ex: <img src= '../../mlrefined_images/math_optimization_images/minibatch_functions.png' width="80%" height="auto"/>

https://jermwatt.github.io/mlrefined/blog_posts/3_First_order_methods/3_10_Minibatch.html

Not sure whether the text just needs to be explicitly referred to as an image. Seems a quick fix

broken image on Vital_Elements_of_Calculus/All_about_derivatives_part_2

Seeing a broken image here: https://jermwatt.github.io/mlrefined/blog_posts/Vital_Elements_of_Calculus/All_about_derivatives_part_2.html

Question about decision trees being universal function approximators.

Hi,

I hope this is the right place to post a question about the HTML materials! I was reviewing this and am having some trouble understanding why a decision tree or regression tree would be a universal function approximator. Would you be able to provide some clarification for why a regression tree would be a universal function approximator? If there is a pointer to a chapter within Machine Learning Refined that goes deeply into this, I would greatly appreciate a reference!

Thanks,

URL not working on Part 5: One-versus-All multi-class classification

The link to code at
https://jermwatt.github.io/mlrefined/blog_posts/Linear_Supervised_Learning/Part_5_One_versus_all.html
is note working.
Unable to open following link
https://github.com/jermwatt/mlrefined/blob/gh-pages/blog_posts/Linear_Supervised_Learning/Part_5_One_versus_all.ipynb

Which version of matplotlib was used?

Nice book and repo. I was running through some of the examples in the 2_5_Random.ipynb. Some of the plots render OK, but then others don't (e.g., static_plotter.two_input_surface_contour_plot) and for those I get a long series of the following errors/warnings (matplotlib: 3.5.1).

*c* argument looks like a single numeric RGB or RGBA sequence, which should be avoided as value-mapping will have precedence in case its length matches with *x* & *y*.  Please use the *color* keyword-argument or provide a 2D array with a single row if you intend to specify the same RGB or RGBA value for all points.

I was wondering whether you had a requirements.txt that specified the version of the libraries used for creating the notebooks.

It might be something else than what the message says. I would have thought that warnings wouldn’t stop the plot from being run. But a requirements.txt might help identify the culprit.

Example 3.3 Galileo and uniform acceleration

Hello sir,

i was trying to solve Exercice 3.4: Reproduce Galileo's Example, which meant reproducing the graph of Example 3.3, and got w=0.0196, which should be the earth gravity: g=9.81 m/s^2. Since i was able reproduce the requested graphs, i suppose that w is correct, but was wondering why it doesn't match g. What are the units of the input values?

Thank you.

Error in Example 3.5

There is an error in the number of runs to reach the minimum using the coordinate descent algorithm for this equation:

g(w1, w2) = 2w1² + 2w² + 2w1w2

The minimum is not reached in just two iterations.

The problem is that you call the algorithm with integer values ​​for the initial estimate of the w vector.

But if you use float values instead:

w = np.array([3.0,4.0])
max_its = 5
weight_history,cost_history = coordinate_descent_for_quadratic(g,w,max_its,a,b,C)

You get the correct values:

[array([3., 4.]), array([-2.,  4.]), array([-2.,  1.]), array([-0.5,  1. ]), array([-0.5 ,  0.25]), array([-0.125,  0.25 ]), array([-0.125 ,  0.0625]), array([-0.03125,  0.0625 ]), array([-0.03125 ,  0.015625]), array([-0.0078125,  0.015625 ]), array([-0.0078125 ,  0.00390625])]```

Problems with the electronic version on Amazon

Two major issues with the electronic version of the book on Amazon.

1. The electronic version is a PDF file, not MOBI. In other words, it has not been adapted to Amazon Kindle.
   In particular, links in the text don't work, from Kindle. And the contents menu is static, not hypertext.

2. Some of the materials on the HTML pages / notebooks just don't exist in the Amazon PDF version. The opposite of what I expected! (because each of your HTML pages / notebooks contains this insert: "The following is part of an early draft of the second edition of Machine Learning Refined. The published text (with revised material) is now available on Amazon"). For example, sections 6.2 and 6.3 in the HTML pages contain appendixes with proofs. These appendixes do not exist in the Amazon PDF version.

Can these problems be fixed? TIA