Module overview
The module will introduce the basic ideas in modelling, solving and simulating stochastic processes.
Aims and Objectives
Learning Outcomes
Learning Outcomes
Having successfully completed this module you will be able to:
- State the Kolmogorov equations for a Markov process where the transition intensities depend not only on age/time, but also on the duration of stay in one or more states
- Calculate the distribution of a Markov chain at a given time
- Determine the stationary and equilibrium distributions of a Markov chain
- Describe a time-inhomogeneous Markov chain and its simple applications
- Understand, in general terms, the principles of stochastic modelling
- Demonstrate how a Markov jump process can be simulated
- Derive the Kolmogorov equations for a Markov process with time independent and time/age dependent transition intensities
- Describe a Markov chain and its transition matrix
- Understand survival, sickness and marriage models in terms of Markov processes
- Recall the definition and derive some basic properties of a Poisson process
- Understand the definition of a stochastic process and in particular a Markov process, a counting process and a random walk
- Solve the Kolmogorov equations in simple cases
- Classify a stochastic process according to whether it operates in continuous or discrete time and whether it has a continuous or a discrete state space, and give examples of each type of process
- Define and explain the basic properties of Brownian motion, demonstrate an understanding of stochastic differential equations and then to integrate, and apply the Ito formula
- Classify the states of a Markov chain as transient, null, recurrent, positive recurrent, periodic, aperiodic and Ergodic
- Demonstrate how a Markov chain can be simulated
Syllabus
Markov Chain
Definition and basic properties
Classification of states and decomposition of state space
The long term probability distribution of a Markov chain
Modelling using Markov chains
Time-homogeneous Markov jump process
Poisson process and its basic properties
Birth and death processes
Kolmogorov differential equations
Structure of a Markov jump process
Time-inhomogeneous Markov jump process
Definition and basics
A survival model
A sickness and death model
A marriage model
Sickness and death with duration dependence
Basic principles of stochastic modelling
Classification of stochastic modelling
Postulating, estimating and validating a model
Simulation of a stochastic model and its applications
Brownian motion: Definition and basic properties. Stochastic differential equations, the Ito integral and Ito formula. Diffusion and mean testing processes. Solution of the stochastic differential equation for the geometric Brownian motion and Ohrnstein-Uhlenbeck process.
Learning and Teaching
Teaching and learning methods
Lectures, problem classes, coursework, surgeries and private study
Type | Hours |
---|---|
Teaching | 48 |
Independent Study | 102 |
Total study time | 150 |
Resources & Reading list
Journal Articles
Hickman JC (1997). Introduction to Actuarial Modelling. North American Actuarial Journal, 1(3), pp. pg. 1-5.
Textbooks
Grimmett G (1992). Probability and Random Processes : Problems and Solutions. Oxford University Press.
Karlin S & Taylor A (1975). A First Module in Stochastic Process. Academic Press.
Brzezniak Z & Zastawniak T (1998). Basic Stochastic Processes : A Module Through Exercises. Springer.
Kulkarni VG (1999). Modelling, Analysis, Design and Control of Stochastic Systems. Springer.
Grimmett G & Stirzaker D (2001). Probability and Random Processes. Oxford University Press.
Assessment
Summative
This is how we’ll formally assess what you have learned in this module.
Method | Percentage contribution |
---|---|
Assignment | 20% |
Class Test | 10% |
Exam | 70% |
Referral
This is how we’ll assess you if you don’t meet the criteria to pass this module.
Method | Percentage contribution |
---|---|
Exam | 100% |
Repeat
An internal repeat is where you take all of your modules again, including any you passed. An external repeat is where you only re-take the modules you failed.
Method | Percentage contribution |
---|---|
Exam | 100% |
Repeat Information
Repeat type: Internal & External