## Important notice

Due to unforeseen circumstances, the **Bayesian Networks** course will not be given in Spring semester 2017-2018. It is replaced by the course **Reinforcement Learning**. All students who have registered for the Bayesian Networks course are advised to follow the Reinforcement Learning course.

## Admission requirements

Some basic prerequisites are:

Basic calculus and linear algebra: You should be comfortable taking derivatives and understanding matrix vector operations and notation.

Basic probability and statistics: You should know basics of probabilities, Gaussian distributions, mean, standard deviation, etc.

## Description

Machine learning becomes a very hot topic in the last decade. Reinforcement learning, a type of machine learning, proves to be a powerful technique to help system autonomously make a decision. The basic idea of reinforcement learning is not to tell a learner which action to take, but to let a system analyze which action to take so as to maximize a reward signal.

In this course, we will discuss what is reinforcement learning, what are the key features that distinguish reinforcement learning from other approaches in machine learning, how to formulate a practical problem into a reinforcement learning problem formally, and how to analyze the performance of a reinforcement learning algorithm. The core techniques and approaches in reinforcement learning we are going to cover include the basic reinforcement learning; multi-armed bandits (which are a good problem to strengthen our understanding of reinforcement learning); Markov decision processes; dynamic programming; etc.

This course is designed to be a general introduction to the reinforcement learning theory and will provide students with a good starting point in the wide field of machine learning. Moreover, several guest lectures are provided in this course to show some cases of practical applications using reinforcement learning techniques, for example AlphaGo and Q learning for game.

## Course objectives

After completing the reinforcement learning course, the students should be able to:

Understand the key features and components of reinforcement learning;

Knowledge of theoretical foundations on basic and advanced reinforcement learning techniques;

Understand the scientific state-of-the-art in the field of reinforcement learning;

Define an application problem as a reinforcement learning problem;

Understand how to apply leaned reinforcement learning techniques on a new problem.

## Timetable

The schedule can be found on the student's website.

Detailed table of contents can be found in Blackboard.

## Mode of instruction

Lectures.

## Course Load

Total load for the course is 6 EC.

## Assessment method

The grade will be determined by the final exam grade.

## Blackboard

Yes.

## Reading list

Mandatory:

- R. Sutton and A. Barto, Reinforcement Learning, MIT Press, 1998.

Recommended but not mandatory:

D.P. Bertsekas, Dynamic Programming and Optimal Control, 2 Vols., Athena Scientific Press, 2005

M. Wiering and M.V. Otterlo, Reinforcement Learning: State-of-the-art, Volume 12, Springer Science & Business Media 2012.

## Registration

You have to sign up for classes and examinations (including resits) in uSis. Check this link for more information and activity codes.

## Contact

Lecturer: dr. Kaifeng Yang.

## Remarks

None.