Kies de Nederlandse taal
Course module: 202001574
Capita Selecta Statistics
Course info
Course module202001574
Credits (ECTS)5
Course typeCourse
Language of instructionEnglish
Contact personS. Langer
Contactperson for the course
S. Langer
S. Langer
Academic year2022
Starting block
Application procedureYou apply via OSIRIS Student
Registration using OSIRISYes
The general aim is that the student gains knowledge about the recent developments in the area of deep learning theory. At the end of the course, the student
  • is able to explain the state-of-the-art of deep learning theory, including recent developments and open questions
  • is able to read recent research papers and grasp the material in such a way that
    1. the student can present it to the fellow students and
    2. the student can indicate the most important aspects of the paper by formulating suitable questions/assignments (for the fellow students) and subsequently grading them appropriately.
Although Deep Learning has already reached the state of the art in many machine learning tasks, the underlying understanding of the methodology is still in its infancy. Most applications rely on intuition and trial and error. So far, we have only a limited understanding of
  • why we can reliably optimise non-convex objectives.
  • how expressive our architectures are, with respect to the class of hypotheses they describe.
  • why most complex models generalise to unseen examples when we use data sets orders of magnitude smaller than what classical statistical learning theory considers sufficient
In this course, we cover the latest in deep learning theory from different perspectives. We start by analyzing the expressive power of neural networks and discuss why depth helps here. We derive statistical risk bounds for different statistical settings, learn about their proof strategies and also their limitations. We discuss optimization related results, analyze the energy landscape, discuss the implicit bias in gradient descent and learn about different regularization methods.

After a first survey on neural network structures, the first few lectures cover the fundamental tools from optimization, statistics and information theory. These lectures are given by the lecturer. There is a weekly assignment for this part, the performance of which is included in the overall grade at the end. The second part of the lecture has the purpose to go over a number of recent publications. This is the active part of the lecture, where each student is responsible for presenting a topic assigned to him/her. The way the student designs the lecture is up to him/her. Interactive parts can be built in here to get the other students to participate. Of course, students will get support from the instructor in preparing their presentations for the course.


As the purpose of this class is to get students engaged with new research in the area of deep learning theory, the majority of credit will be given for the presentation. In particular, the final grade is composed of the quality of the students’ presentation (60%), the overall participation during the course (10%), and the average score for the assignments (30%).
Assumed previous knowledge
Students should have solid knowledge of statistics, including knowledge of probability, expectation, mean, variance, discrete and continuous distributions, confidence intervals, simple linear regression. For instance, the Probability Theory for AM and Mathematical Statistics courses cover this material. Completion of BSc-TCS Modules 04 and 06 is also acceptable.

Some knowledge about machine learning or applied statistical learning is helpful but not required.
Participating study
Master Applied Mathematics
Required materials
Recommended materials
Instructional modes
Presence dutyYes

Presence dutyYes

Assignment, Presentation

Kies de Nederlandse taal