Module overview
Deep learning and differentiable programming has revolutionised numerous fields in recent years. We've witnessed improvements in everything from computer vision through speech analysis to natural language processing as a result of the advent of cheap GPGPU compute coupled with large datasets and some neat algorithms. This module will look at how deep learning works, from the theoretical foundations of the concepts of differentiable programming right through to practical implementation.
Linked modules
Prerequisites: COMP3223 or COMP6245
Aims and Objectives
Learning Outcomes
Subject Specific Intellectual and Research Skills
Having successfully completed this module you will be able to:
- Critically appraise recent scientific literature in deep learning
- Critically appraise the merits and shortcomings of model architectures on specific problems
Subject Specific Practical Skills
Having successfully completed this module you will be able to:
- Apply existing deep learning models to real datasets
- Gain facility in working with differentiable programming and deep learning libraries in order to create and evaluate differentiable programmes and deep networks
Knowledge and Understanding
Having successfully completed this module, you will be able to demonstrate knowledge and understanding of:
- Underlying mathematical and algorithmic principles of deep learning
- The key factors that have made deep learning successful for various applications
Syllabus
Historical Developments
- MLPs
- CNNs, LeNet and the ImageNet competition
- RNNs
- Attention mechanisms and transformers
Mechanics
- Backpropagation
- Automatic differentiation
Learning Algorithms
- Initialisation
- SGD, Momentum, etc.
Auto-encoders
- variational
- denoising
CNNs
- Architectures
- Region Propositions
- Semantic Segmentation
Sequence Modelling
- Linear Embeddings
- RNNs
∗ LSTMs
∗ GRUs
∗ back-prop through time
- Transformers
Deep Learning Applications
- Computer Vision
- Natural Language Processing & Generation
- Speech
Differentiable Relaxations
- Straight-through operator
- Sampling
Current research challenges
Learning and Teaching
Teaching and learning methods
Lectures, labs and guided self-study
| Type | Hours |
|---|---|
| Specialist Laboratory | 20 |
| Lecture | 24 |
| Wider reading or practice | 46 |
| Completion of assessment task | 60 |
| Total study time | 150 |
Resources & Reading list
Textbooks
Ian Goodfellow, Yoshua Bengio and Aaron Courville (2016). Deep Learning.
Assessment
Summative
This is how we’ll formally assess what you have learned in this module.
| Method | Percentage contribution |
|---|---|
| Group project report | 40% |
| Blackboard quizzes | 20% |
| Laboratory Report | 40% |
Referral
This is how we’ll assess you if you don’t meet the criteria to pass this module.
| Method | Percentage contribution |
|---|---|
| Set Task | 100% |
Repeat
An internal repeat is where you take all of your modules again, including any you passed. An external repeat is where you only re-take the modules you failed.
| Method | Percentage contribution |
|---|---|
| Set Task | 100% |
Repeat Information
Repeat type: Internal & External