Mathematical Foundations of Machine Learning [MA4801]
Sommersemester 2016
Prof. Dr. Michael M. Wolf
| Dozent: | Prof. Dr. Michael M. Wolf | |
| Übungsleitung: | ||
| Mitwirkende: | ||
| Vorlesung: | Tuesday, 14:15-16:00, MI HS3 | Anmeldung |
| Übung: | Wednesday, 10:00-11:30, room 00.08.059 Thursday, 14:00-16:00, room 02.13.010 Friday, 12:00-14:00, room 03.06.011 |
Anmeldung |
News
Content
The course will provide an introduction into the mathematical foundations of learning theory, neural networks, support vector machines and kernel methods.Prerequisites
Basic knowledge in linear algebra, analysis and probability theory is required as well as some elementary Hilbert space theory.Notes
Preliminary lecture notes (ideally updated weekly, after every lecture) can be found here (last update: Nov 26th). This also contains some of the solutions to the exercises.
| File | Date | Content | Comments | Solution |
|---|---|---|---|---|
| Exercise class 1 | Week 13 - 20 April | ERM, error decomposition, Hoeffding's inequality | ||
| Exercise class 2 | Week 21 - 29 April | PAC learning | ||
| Exercise class 3 | Week 9-13 May | Growth function and VC-dimension | ||
| Exercise class 4 | Week 16-20 May | Concentration inequalities and Rademacher complexities | ||
| Exercise class 5 | Week 1-3 June | Adaboost, Neural Networks | ||
| Exercise class 6 | Week 8-10 June | VCdim and Rademacher complexities of neural networks | ||
| Exercise class 7 | Week 15-17 June | Neural networks - complexity and geometry | ||
| Exercise class 8 | Week 22-24 June | Recap | ||
| Exercise class 9 | Week 29-30 June | Rademacher complexity with margin, KKT and support vectors | ||
| Exercise class 10 | 1st week of July | Kernels | ||
| Exercise class 11 | 2nd week of July | Open discussion / preparation for the exam |
Literature
There are many good books on the topic. Recent examples with a focus on mathematical aspects are:- Foundations of Machine Learning, M. Mohri, A. Rostamizadeh, A. Talwalkar, MIT Press, 2012
- Understanding Machine Learning: From Theory to Algorithms, Shai Shalev-Shwartz, Shai Ben-David, Cambridge University Press, 2014
- Neural Network Learning: Theoretical Foundations, M. Anthony, P.L. Bartlett, Cambridge University Press, 1999
- Statistical Learning Theory, V.N. Vapnik, John Wiley & Sons, 1998
