Close Help Print
 Course module: 201700080
 201700080Information Theory and Statistics
 Course info
Course module201700080
Credits (ECTS)5
Course typeCourse
Language of instructionEnglish
Contact personM.C. de Jongh
E-mailm.c.dejongh@utwente.nl
Lecturer(s)
 Lecturer dr.ir. J. Goseling Examiner M.C. de Jongh Contactperson for the course M.C. de Jongh
Starting block
 2A
Application procedureYou apply via OSIRIS Student
Registration using OSIRISYes
 Aims
 body { font-size: 9pt; font-family: Arial } table { font-size: 9pt; font-family: Arial } After passing the assessment of this course a student: knows Shannon entropy and mutual information, is able to perform computations involving these information measures and is able to estimate these measures from a data set, knows the Huffman, Lempel-Ziv and CTW data compression algorithms, and understands the limits on data compression, understands the connection between machine learning and data compression and is able to quantify performance limits on machine learning algorithms, is able to quantify the optimal performance that can be expected from a classification system in terms of information measures.
 Content
 body { font-size: 9pt; font-family: Arial } table { font-size: 9pt; font-family: Arial } Information theory is a mathematical theory dealing with the fundamental principles of storing, processing, and transmitting data. The first half of this course covers the core concepts of information theory, including entropy and mutual information. These are then used to derive fundamental limits on data compression and communication. The second half of this course focuses on applications of information theory to statistics and machine learning. In particular, information theory will be used to develop performance limits on machine learning algorithms. Assessment 1) Written exam (60 %, needs >= 5.5) 2) homework (20 %) 3) report, based on project or reading assignment (20 %)
Assumed previous knowledge
 Basic probability theory, for instance from:• B-AM M4,• B-EE M8,• B-CS/BIT M4 or• 202001177 Probability Theory and Statistics (premaster M-CS).
 Participating study
 Master Computer Science
 Participating study
 Master Applied Mathematics
 Participating study
 Master Electrical Engineering
Required materials
Handouts
 Lecture notes
Recommended materials
-
Instructional modes
 Lecture Tutorial
Tests
 Test
 Close Help Print