Aims - This module aims to provide a continuation of the study of random processes, but with the emphasis now on operational research applications and including queueing theory, renewal and semi-Markov processes, and reliability theory. It is primarily intended for third and fourth year undergraduate students and taught postgraduate students registered on the degree programmes offered by the Department of Statistical Science, or jointly with other departments. The academic prerequisite for such students is STAT0007, or in the case of taught postgraduate students, an equivalent undergraduate introductory module in applied probability.

Objectives - On successful completion of the module, a student should understand such concepts for stochastic processes as the Markov property, stationarity and reversibility and be able to determine whether such properties apply in straightforward examples; recognise and apply appropriately a range of models in a variety of applied situations so as to determine properties relevant to the particular application.

Applications - Stochastic systems arise in many areas of application. They play a fundamental role in Operational Research which addresses real-world problems through the use of mathematics, probability and statistics; topics such as queueing theory and reliability are important examples. Stochastic processes are also vital to applications in finance and insurance, and have many applications in biology and medicine, and in the social sciences. Stochastic process theory underpins modern simulation methods like Markov-chain Monte-Carlo (MCMC).

Content - Markov processes: revision of general concepts, reversibility and detailed balance equations. Renewal theory and reliability: regenerative events and renewal processes, alternating renewal processes, renewal reward processes. Queues: the general single server queue, Markov queueing models (M/M/k), limited waiting room, more general queues (M/G/1, G/M/1), queueing networks. Semi-Markov processes: properties and simple examples. Reliability: single repairable units, simple systems of units. Basics of spatial point processes. Programming: basic coding in R or Python.

Key texts - Available from ReadingLists@UCL.