Mathematical Foundations of Machine Learning [MA4801]
Sommersemester 2016
Prof. Dr. Michael M. Wolf
| Dozent: | Prof. Dr. Michael M. Wolf | |
| Übungsleitung: | ||
| Mitwirkende: | ||
| Vorlesung: | Tuesday, 14:15-16:00, MI HS3 | Anmeldung |
| Übung: | Wednesday, 10:00-11:30, room 00.08.059 Thursday, 14:00-16:00, room 02.13.010 Friday, 12:00-14:00, room 03.06.011 |
Anmeldung |
News
The repeat exam will be on September 26th, 11:30-12:30 in room 03.10.011. The (written) exam will be on July 28th, 11:00-12:00 in room MW0350. No written notes or electronic tools are allowed in the exam.Content
The course will provide an introduction into the mathematical foundations of learning theory, neural networks, support vector machines and kernel methods.Prerequisites
Basic knowledge in linear algebra, analysis and probability theory is required as well as some elementary Hilbert space theory.Notes
Preliminary lecture notes (ideally updated weekly, after every lecture) can be found here (last update: July 17th). This also contains some of the solutions to the exercises.
| File | Date | Content | Comments | Solution |
|---|---|---|---|---|
| Exercise class 1 | Week 13 - 20 April | ERM, error decomposition, Hoeffding's inequality | Solution 1 | |
| Exercise class 2 | Week 21 - 29 April | PAC learning | Solution 2 | |
| Exercise class 3 | Week 9-13 May | Growth function and VC-dimension | Solution 3 | |
| Exercise class 4 | Week 16-20 May | Concentration inequalities and Rademacher complexities | Solution 4 | |
| Exercise class 5 | Week 1-3 June | Adaboost, Neural Networks | see lecture notes | |
| Exercise class 6 | Week 8-10 June | VCdim and Rademacher complexities of neural networks | ||
| Exercise class 7 | Week 15-17 June | Neural networks - complexity and geometry | see lecture notes | |
| Exercise class 8 | Week 22-24 June | Recap | ||
| Exercise class 9 | Week 29-30 June | Rademacher complexity with margin, KKT and support vectors | ||
| Exercise class 10 | 1st week of July | Kernels | Solution 10 | |
| Exercise class 11 | 2nd week of July | Open discussion / preparation for the exam | Problems with solutions 11 |
Literature
There are many good books on the topic. Recent examples with a focus on mathematical aspects are:- Foundations of Machine Learning, M. Mohri, A. Rostamizadeh, A. Talwalkar, MIT Press, 2012
- Understanding Machine Learning: From Theory to Algorithms, Shai Shalev-Shwartz, Shai Ben-David, Cambridge University Press, 2014
- Neural Network Learning: Theoretical Foundations, M. Anthony, P.L. Bartlett, Cambridge University Press, 1999
- Statistical Learning Theory, V.N. Vapnik, John Wiley & Sons, 1998
