## Mathematical Foundations of Machine Learning [MA4801]

### Sommersemester 2018

### Prof. Dr. Michael M. Wolf

Lecturer: |
Prof. Dr. Michael M. Wolf | |

Assistant: |
Javier Cuesta | |

Lectures: |
Wednesday, 12:15-13:45, LMU Physik Hörsaal (Garching Forschungszentrum) | Anmeldung |

Exercises: |
Tuesday, 12:15 - 13:45, LMU Physik Hörsaal (Garching Forschungszentrum) Friday, 14:15-15:45, room MI 02.08.020 |
Anmeldung |

### News

- First exercise class on April 17. First lecture on April 11.
- There will be no Tutorial on May 1st and 4th nor Exercise sheet.

### Content

The course will provide an introduction to the mathematical foundations of supervised learning theory, neural networks, support vector machines and kernel methods.Date | Content | To read | Black board notes |
---|---|---|---|

April 11 | Intro, Statistical learning theory framework, error decomposition | Intro, 1.1 and 1.2 in lecture notes | |

April 18 | PAC bounds, growth function, VC-dimension and VC-dichotomy | 1.3, 1.5 and 1.6 in lecture notes | lec2 |

April 25 | VC-dimension of vector spaces, covering and packing numbers | 1.6 and 1.9 in lecture notes | lec3 |

May 2 | covering number PAC bound, pseudo and fat-shattering dimension | 1.9 and 1.10 in lecture notes | lec4 |

May 9 | pseudo and fat-shattering dimension, uniform stability, on-average stability, regularization | 1.10 and 1.11 in lecture notes |

### Prerequisites

Basic knowledge in linear algebra, analysis and probability theory is required as well as some elementary Hilbert space theory.### Lecture notes

A preliminary version of the lecture notes can be found here (version from May 13). They will be updated on a weekly basis (hence, think twice before printing).File | Date | Content | Comments | Solution |
---|---|---|---|---|

Exercise class 1 | Week 17 - 24 April | Basic probability review, Hoeffding's inequality | Basic Prob. Review | Solution 1 |

Exercise class 2 | Week 24 April - 1 May | PAC learning, VC dimension and VC-dichotomy | Wording in H2.2 changed | Solution 2 |

Exercise class 3 | Week 8 - 15 May | Covering numbers, pseudo dimension, and fat-shattering-dimension | Typo in H3.3b corrected | Solution 3 |

Exercise class 4 | Week 15 - 22 May | Algorithmic Stability and Uniform covering number | H4.3 a bit simplified and more hints added | Solution 4 |

Exercise class 5 | Week 22 - 29 May | Stability and generalized bounds |

### Literature

There are many good books on the topic. Recent examples with a focus on mathematical aspects are:- Foundations of Machine Learning, M. Mohri, A. Rostamizadeh, A. Talwalkar, MIT Press, 2012
- Understanding Machine Learning: From Theory to Algorithms, Shai Shalev-Shwartz, Shai Ben-David, Cambridge University Press, 2014

- Neural Network Learning: Theoretical Foundations, M. Anthony, P.L. Bartlett, Cambridge University Press, 1999
- Statistical Learning Theory, V.N. Vapnik, John Wiley & Sons, 1998