Kies de Nederlandse taal
Course module: 202000425
Stochastic Models
Course info
Course module202000425
Credits (ECTS)5
Course typeStudy Unit
Language of instructionEnglish
Contact L.L.M. van der Wegen
PreviousNext 5
dr. A. Asadi
Lecturer A. Braaksma
dr. S.M. Meisel
dr. S. Rachuba
Lecturer W.R.W. Scheinhardt
Academic year2022
Starting block
Application procedureYou apply via OSIRIS Student
Registration using OSIRISYes
After successful completion of this module component, the student is able to:
  1. formulate a Markov chain model for a given problem description and solve this model;
  2. formulate a Stochastic Dynamic Programming (SDP) model for a given problem description and solve this model;
  3. formulate a Markov Decision Process (MDP) model for a given problem description and solve this model;
  4. interpret the outcomes of an SDP and MDP model in order to construct an optimal strategy, which is applicable in a given practical situation;
  5. select an appropriate queueing model (M/M/1, M/M/c, etc.) for a given problem description and solve this model;
  6. interpret the implications of a queueing system’s performance on (given) performance indicators and formulate practical recommendations for system improvement.
Students first learn the basics of Markov chains (in discrete and continuous time, and Poisson processes). Using that knowledge, they next learn about stochastic dynamic programming and about queueing theory. Stochastic dynamic programming can be used to solve sequential multistage decision problems under uncertainty, e.g., transportation planning for multiple time periods with uncertain order arrivals and transportation times. Queueing theory can be used to analyse queueing problems that occur, e.g., in production and logistics, service industries, and communication networks.


Passing grade is 5.5 (average of two subtests on SDP and Queueing) and a score of at least 40% on all subtests (SDP and Queueing). In case of passing one of the subtests (SDP or Queuing) the score remains valid for the same year. But, in case of failing the Stochastic models, both parts needs redo for the upcoming year.
Max of Exam 1 and Exam 2 counts.

Assumed previous knowledge
1. Probability theory (e.g., random variables, cumulative distribution functions, probability density functions, conditional distributions and measures of a distribution).
2. Operations research (e.g., basic knowledge of queueing models and understands the idea behind dynamic programming).
Module 8
Participating study
Bachelor Industrial Engineering and Management
Required materials
WL Winston – Operations Research: Applications and Algorithms, 4th edition ISBN: 9780357337769 (ebook)
Recommended materials
Instructional modes


Self study without assistance


Stochastic Models Exam

Kies de Nederlandse taal