Module finder

This module is about the fundamentals of algorithms solving continuous optimisation problems, which involve minimising functions of multiple real-valued variables, possibly subject to restrictions, constraints, and nondifferentiable regularisations on the values that the variables may take. We focus (not exclusively) on convex optimisation, where the choice of topics is motivated by relevance to machine learning and data science.

The module has a two-part syllabus.

Part 1 covers the theoretical foundation of optimisation: convex analysis. Topics include the notion of convexity, subdifferential, optimality conditions and properties of various formulations of continuous optimisation problems.

Part 2 focuses on methods for solving optimisation problems. Topics include various gradient descent methods, higher-order methods, coordinate descent, randomisation, and heuristics.


[Module focus] This module is on structural continuous nonlinear optimisation in the real Euclidean space. This module is not about linear programming, combinatorial optimisation nor PDE-constrained optimisation.


[Prerequisites] A good knowledge of linear algebra and (differential) calculus is required for this module. Exposure to numerical analysis and vector calculus is helpful but not required; the applications will be kept basic and simple. Students will write scripts in MATLAB/Python, so familiarity with programming is required.


[Who should enrol] This module is expected to be beneficial to anyone who uses or will uses optimisation in machine learning and related work. More specifically, people from the following fields: machine learning, signal and image processing, communications, bioinformatics, control, robotics, computer graphics, computer vision, operation research, scientific computing, computational mathematics, and finance.

This module is about the fundamentals of algorithms solving continuous optimisation problems, which involve minimising functions of multiple real-valued variables, possibly subject to restrictions, constraints, and nondifferentiable regularisations on the values that the variables may take. We focus (not exclusively) on convex optimisation, where the choice of topics is motivated by relevance to machine learning and data science.

The module has a two-part syllabus.

Part 1 covers the theoretical foundation of optimisation: convex analysis. Topics include the notion of convexity, subdifferential, optimality conditions and properties of various formulations of continuous optimisation problems.

Part 2 focuses on methods for solving optimisation problems. Topics include various gradient descent methods, higher-order methods, coordinate descent, randomisation, and heuristics.


[Module focus] This module is on structural continuous nonlinear optimisation in the real Euclidean space. This module is not about linear programming, combinatorial optimisation nor PDE-constrained optimisation.


[Prerequisites] A good knowledge of linear algebra and (differential) calculus is required for this module. Exposure to numerical analysis and vector calculus is helpful but not required; the applications will be kept basic and simple. Students will write scripts in MATLAB/Python, so familiarity with programming is required.


[Who should enrol] This module is expected to be beneficial to anyone who uses or will uses optimisation in machine learning and related work. More specifically, people from the following fields: machine learning, signal and image processing, communications, bioinformatics, control, robotics, computer graphics, computer vision, operation research, scientific computing, computational mathematics, and finance.

This module is about the fundamentals of algorithms solving continuous optimisation problems, which involve minimising functions of multiple real-valued variables, possibly subject to restrictions, constraints, and nondifferentiable regularisations on the values that the variables may take. We focus (not exclusively) on convex optimisation, where the choice of topics is motivated by relevance to machine learning and data science.

The module has a two-part syllabus.

Part 1 covers the theoretical foundation of optimisation: convex analysis. Topics include the notion of convexity, subdifferential, optimality conditions and properties of various formulations of continuous optimisation problems.

Part 2 focuses on methods for solving optimisation problems. Topics include various gradient descent methods, higher-order methods, coordinate descent, randomisation, and heuristics.


[Module focus] This module is on structural continuous nonlinear optimisation in the real Euclidean space. This module is not about linear programming, combinatorial optimisation nor PDE-constrained optimisation.


[Prerequisites] A good knowledge of linear algebra and (differential) calculus is required for this module. Exposure to numerical analysis and vector calculus is helpful but not required; the applications will be kept basic and simple. Students will write scripts in MATLAB/Python, so familiarity with programming is required.


[Who should enrol] This module is expected to be beneficial to anyone who uses or will uses optimisation in machine learning and related work. More specifically, people from the following fields: machine learning, signal and image processing, communications, bioinformatics, control, robotics, computer graphics, computer vision, operation research, scientific computing, computational mathematics, and finance.

This module is about the fundamentals of algorithms solving continuous optimisation problems, which involve minimising functions of multiple real-valued variables, possibly subject to restrictions, constraints, and nondifferentiable regularisations on the values that the variables may take. We focus (not exclusively) on convex optimisation, where the choice of topics is motivated by relevance to machine learning and data science.

The module has a two-part syllabus.

Part 1 covers the theoretical foundation of optimisation: convex analysis. Topics include the notion of convexity, subdifferential, optimality conditions and properties of various formulations of continuous optimisation problems.

Part 2 focuses on methods for solving optimisation problems. Topics include various gradient descent methods, higher-order methods, coordinate descent, randomisation, and heuristics.


[Module focus] This module is on structural continuous nonlinear optimisation in the real Euclidean space. This module is not about linear programming, combinatorial optimisation nor PDE-constrained optimisation.


[Prerequisites] A good knowledge of linear algebra and (differential) calculus is required for this module. Exposure to numerical analysis and vector calculus is helpful but not required; the applications will be kept basic and simple. Students will write scripts in MATLAB/Python, so familiarity with programming is required.


[Who should enrol] This module is expected to be beneficial to anyone who uses or will uses optimisation in machine learning and related work. More specifically, people from the following fields: machine learning, signal and image processing, communications, bioinformatics, control, robotics, computer graphics, computer vision, operation research, scientific computing, computational mathematics, and finance.

Module Contents: This module discusses continuous optimization problems where either the objective function or constraint functions or both are nonlinear. It explains optimality conditions, that is, which conditions an optimal solution must satisfy. It introduces the most popular numerical methods such as line search methods, Newton’s method and quasi-Newton’s methods and conjugate gradient methods for solving unconstrained optimization problems, and penalty function method and sequential quadratic programming methods for solving constrained optimization problems. Further, it explores
the theoretical background behind these powerful optimization methods by looking into how a specific method was motivated, developed, implemented and applied.

Characters: This module considers nonlinear continuous optimisation problems. This differentiates it from linear programming where the objective and constaint functions are affine. The continuity means that the decision variable may take continuous values instead of discrete values as in integer
programming. The module does not discuss mathematical modelling of practical problems; instead it focuses on numerical methods for solving standard optimization models of practical problems

Module Contents: This module discusses continuous optimization problems where either the objective function or constraint functions or both are nonlinear. It explains optimality conditions, that is, which conditions an optimal solution must satisfy. It introduces the most popular numerical methods such as line search methods, Newton’s method and quasi-Newton’s methods and conjugate gradient methods for solving unconstrained optimization problems, and penalty function method and sequential quadratic programming methods for solving constrained optimization problems. Further, it explores
the theoretical background behind these powerful optimization methods by looking into how a specific method was motivated, developed, implemented and applied.

Characters: This module considers nonlinear continuous optimisation problems. This differentiates it from linear programming where the objective and constaint functions are affine. The continuity means that the decision variable may take continuous values instead of discrete values as in integer
programming. The module does not discuss mathematical modelling of practical problems; instead it focuses on numerical methods for solving standard optimization models of practical problems

This module introduces key operational research (OR) techniques. It is split into two sections focusing on deterministic and stochastic OR techniques respectively. The methods taught on the module are useful tools for solving practical management and logistics problems.

The deterministic OR section focuses on linear and integer programming. The process of modelling problems of a practical nature as a linear or integer program will be developed. Following an explanation of a standard version of the simplex method, some of its variants will be introduced. The main ideas of linear programming duality will also be explained. A computer workshop session trains students in the use of commercial linear programming software. The branch and bound approach for solving integer programming problems will also be developed.

In the stochastic OR section, the main topics are decision theory and simulation modelling. Models and examples are given to demonstrate applications of the topics. Discrete event simulation is taught via lectures and computer workshops while Decision Theory is taught in lectures.

This module introduces key operational research (OR) techniques. It is split into two sections focusing on deterministic and stochastic OR techniques respectively. The methods taught on the module are useful tools for solving practical management and logistics problems.

The deterministic OR section focuses on linear and integer programming. The process of modelling problems of a practical nature as a linear or integer program will be developed. Following an explanation of a standard version of the simplex method, some of its variants will be introduced. The main ideas of linear programming duality will also be explained. A computer workshop session trains students in the use of commercial linear programming software. The branch and bound approach for solving integer programming problems will also be developed.

In the stochastic OR section, the main topics are decision theory and simulation modelling. Models and examples are given to demonstrate applications of the topics. Discrete event simulation is taught via lectures and computer workshops while Decision Theory is taught in lectures.

This module introduces key operational research (OR) techniques. It is split into two sections focusing on deterministic and stochastic OR techniques respectively. The methods taught on the module are useful tools for solving practical management and logistics problems.

The deterministic OR section focuses on linear and integer programming. The process of modelling problems of a practical nature as a linear or integer program will be developed. Following an explanation of a standard version of the simplex method, some of its variants will be introduced. The main ideas of linear programming duality will also be explained. A computer workshop session trains students in the use of commercial linear programming software. The branch and bound approach for solving integer programming problems will also be developed.

In the stochastic OR section, the main topics are decision theory and simulation modelling. Models and examples are given to demonstrate applications of the topics. Discrete event simulation is taught via lectures and computer workshops while Decision Theory is taught in lectures.

This module is based partially on lectures and partially on the study of existing scores, together with listening tasks. It is understood that practical experience of hearing your arrangements is the best teacher of all: therefore there is an expectation that you will write for existing and active ensembles. A prior knowledge of notation software e.g. Sibelius or Finale, is also important.