## Mathematical Foundations of Machine Learning [MA4801]

### Sommersemester 2016

### Prof. Dr. Michael M. Wolf

Dozent: |
Prof. Dr. Michael M. Wolf | |

Übungsleitung: |
||

Mitwirkende: |
||

Vorlesung: |
Tuesday, 14:15-16:00, MI HS3 | Anmeldung |

Übung: |
Wednesday, 10:00-11:30, room 00.08.059 Thursday, 14:00-16:00, room 02.13.010 Friday, 12:00-14:00, room 03.06.011 |
Anmeldung |

### News

**The repeat exam will be on September 26th, 11:30-12:30 in room 03.10.011.**The (written) exam will be on July 28th, 11:00-12:00 in room MW0350. No written notes or electronic tools are allowed in the exam.

### Content

The course will provide an introduction into the mathematical foundations of learning theory, neural networks, support vector machines and kernel methods.### Prerequisites

Basic knowledge in linear algebra, analysis and probability theory is required as well as some elementary Hilbert space theory.### Notes

Preliminary lecture notes (ideally updated weekly, after every lecture) can be found here (last update: July 17th). This also contains some of the solutions to the exercises.File | Date | Content | Comments | Solution |
---|---|---|---|---|

Exercise class 1 | Week 13 - 20 April | ERM, error decomposition, Hoeffding's inequality | Solution 1 | |

Exercise class 2 | Week 21 - 29 April | PAC learning | Solution 2 | |

Exercise class 3 | Week 9-13 May | Growth function and VC-dimension | Solution 3 | |

Exercise class 4 | Week 16-20 May | Concentration inequalities and Rademacher complexities | Solution 4 | |

Exercise class 5 | Week 1-3 June | Adaboost, Neural Networks | see lecture notes | |

Exercise class 6 | Week 8-10 June | VCdim and Rademacher complexities of neural networks | ||

Exercise class 7 | Week 15-17 June | Neural networks - complexity and geometry | see lecture notes | |

Exercise class 8 | Week 22-24 June | Recap | ||

Exercise class 9 | Week 29-30 June | Rademacher complexity with margin, KKT and support vectors | ||

Exercise class 10 | 1st week of July | Kernels | Solution 10 | |

Exercise class 11 | 2nd week of July | Open discussion / preparation for the exam | Problems with solutions 11 |

### Literature

There are many good books on the topic. Recent examples with a focus on mathematical aspects are:- Foundations of Machine Learning, M. Mohri, A. Rostamizadeh, A. Talwalkar, MIT Press, 2012
- Understanding Machine Learning: From Theory to Algorithms, Shai Shalev-Shwartz, Shai Ben-David, Cambridge University Press, 2014

- Neural Network Learning: Theoretical Foundations, M. Anthony, P.L. Bartlett, Cambridge University Press, 1999
- Statistical Learning Theory, V.N. Vapnik, John Wiley & Sons, 1998