Reinforcement and Online Learning

Learning Outcomes

Subject Specific Intellectual and Research Skills

Having successfully completed this module you will be able to:

• Critical appraisal of recent scientific literature in reinforcement and online learning
• Critically appraise the merits and shortcomings of model architectures on specific problems

Subject Specific Practical Skills

Having successfully completed this module you will be able to:

• Gain facility in working with reinforcement and on-line learning algorithms in order to create and evaluate their performance and applicability in different application domains
• Apply existing reinforcement and on-line learning models to real applications

Knowledge and Understanding

Having successfully completed this module, you will be able to demonstrate knowledge and understanding of:

• The key factors that have made reinforcement and on-line learning successful for various applications
• Underlying mathematical and algorithmic principles of reinforcement and online learning

Classical Reinforcement

• TD learning
• Q learning
• State Space Models
• Example: TD-Gammon

On-line Learning

• Regret minimisation
• Stochastic vs. adversarial
• Full information, semi-bandit, and bandit feedback

Monte Carlo Tree Search (MCTS)

Applications

• AlphaZero: combining MCTS, reinforcement learning and deep learning
• Hyper-parameter search in deep learning with bandit theory
• Playing no-limit poker with counterfactual regret minimisation

Teaching and learning methods

Lectures and labs

Resources & Reading list

Textbooks

Richard Sutton and Andrew Barto (2017). Reinforcement Learning: An Introduction.

Csaba Szepesvari (2010). Algorithms for Reinforcement Learning.

Summative

This is how we’ll formally assess what you have learned in this module.

Referral

This is how we’ll assess you if you don’t meet the criteria to pass this module.

Repeat

An internal repeat is where you take all of your modules again, including any you passed. An external repeat is where you only re-take the modules you failed.

Repeat Information

Repeat type: Internal & External