06 - GAUSSIAN PROCESSES - INTRODUCTION TO REGRESSION AND KERNEL METHODS

************************************************************************** BECOME ONE OF THE FIRST STUDENTS OF THE NEW STANDARD MACHINE LEARNING CURRICULUM! https://forms.gle/baxK3Cwz8q2xuJEt5 ************************************************************************** Timestamps for this lecture: 00:00 Recap: Multivariate Gaussian distribution 09:09 Recap: Some properties of the multivariate Gaussian distribution 14:59 Our model assumptions 16:13 Calculating the likelihood of observations given our model 19:07 Bayesian linear regression: Prior over our parameters w 20:15 Bayesian linear regression: The marginal likelihood 21:59 Bayesian linear regression: The posterior 27:39 Making predictions with our model: The standard way 32:47 Making predictions with our model: The "kernel trick" way 36:29 Kernels are back 41:05 Gaussian Process regression: Definition 52:55 Our Bayesian linear regression model is a Gaussian Process 47:05 Using kernels to construct a prior distribution 49:57 Conditioning on noise free data 52:34 Illustrating Gaussian Process draws from the prior vs. the posterior 55:13 Conditioning on noisy data 59:31 Varying the hyperparameters of our Gaussian process model Get the lecture slides: https://drive.google.com/file/d/1nIqISTpYxBYsl4ym2Hn95dKmVdXsX127/view?usp=sharing There is a shortage of well-presented learning resources for rigorous Machine Learning. In the FUNDAMENTALS OF MACHINE LEARNING program @ äon intelligence, laser-focused modules are provided for competent introductions, as well as in-depth discussions of advanced content. The INTRODUCTION TO REGRESSION AND KERNEL METHODS module provides the ideal introduction to the field of Machine Learning for anyone with a reasonable level of mathematical maturity. The contents of this module include: 01 - PREREQUISITES: Basic concepts of linear algebra and multivariable calculus are stated and practised with simple examples. Outlook on prerequisites of advanced content is given, including probability theory, eigenvalues and their eigenvectors, matrix decompositions, and special spaces in mathematics. 02 - FORMAL SETUP: The principle of empirical risk minimization is introduced along with famous error decompositions. Hypothesis spaces and their relation to the phenomenon of overfitting and underfitting are discussed. 03 - LINEAR REGRESSION: A vectorized hypothesis of linear relationship between features and lables is presented before formally introducing vector norms to measure the magnitude of error vectors. Via convex optimization of the mean squared error loss, we arrive at the normal equation, a closed form solution to the linear regression problem. Finally, polynomial regression as well as ridge regression are presented and solved. 04 - THE KERNEL TRICK: Using Mercer's theorem, kernels are introduced as a way to reduce the computational load of evaluating the Gram matrices present in polynomial ridge regression. It is demonstrated that the Gaussian kernel maps into an infinite dimensional feature space. Ridge regression is solved using the kernel formalism. 05 - REPRODUCING KERNEL HILBERT SPACES: Basic concepts of functional analysis are reviewed before providing a formally rigorous definition of kernels and the reproducing kernel Hilbert spaces (RKHS). Basic properties of functions in the RKHS are presented, including the representer theorem. 06 - GAUSSIAN PROCESSES: The formalism of Bayesian linear regression is introduced, leading to probability densities around our point estimates. Gaussian processes are defined and kernel matrices are used as covariance matrices. 07 - THE RELATIONSHIP BETWEEN KERNELS AND GAUSSIAN PROCESSES: Selected notions of linear algebra and measure theory are reviewed before examining the relationships between kernel methods as a frequentist idea and gaussian processes as a Bayesian idea. 08 - OUTRO: The resources used to compile the content are presented. Also, an outlook to the next module is given.