Admission requirements
Some basic prerequisites are:
Basic calculus and linear algebra: You should be comfortable taking derivatives and understanding matrix vector operations and notation.
Basic probability and statistics: You should know basics of probabilities, Gaussian distributions, mean, standard deviation, etc.
Description
Machine learning becomes a very hot topic in the last decade. Reinforcement learning, a type of machine learning, proves to be a powerful technique to help system autonomously make a decision. The basic idea of reinforcement learning is not to tell a learner which action to take, but to let a system analyze which action to take so as to maximize a reward signal.
In this course, we will discuss what is reinforcement learning, what are the key features that distinguish reinforcement learning from other approaches in machine learning, how to formulate a practical problem into a reinforcement learning problem formally, and how to analyze the performance of a reinforcement learning algorithm. The core techniques and approaches in reinforcement learning we are going to cover include the basic reinforcement learning; multi-armed bandits (which are a good problem to strengthen our understanding of reinforcement learning); Markov decision processes; dynamic programming; etc.
This course is designed to be a general introduction to the reinforcement learning theory and will provide students with a good starting point in the wide field of machine learning. Moreover, several guest lectures are provided in this course to show some cases of practical applications using reinforcement learning techniques, for example AlphaGo and Q learning for game.
Course objectives
After completing the reinforcement learning course, the students should be able to:
Understand the key features and components of reinforcement learning;
Knowledge of theoretical foundations on basic and advanced reinforcement learning techniques;
Understand the scientific state-of-the-art in the field of reinforcement learning;
Define an application problem as a reinforcement learning problem;
Understand how to apply leaned reinforcement learning techniques on a new problem.
Timetable
The schedule can be found on the student's website.
Detailed table of contents can be found in Blackboard.
Mode of instruction
Lectures
Computer lab
Course load
Hours of study: 168 hrs (= 6 EC)
Lectures: 26:00 hrs
Computer lab: 26:00 hrs
Examination: 03:00 hrs
Other: 113:00 hrs
Assessment method
The final grade is a combination of grades for: (1) the written exam (60%) and (2) the report about the practical assignment(s) (40%).
Reading list
Mandatory:
- R. Sutton and A. Barto, Reinforcement Learning, MIT Press, 1998.
Recommended but not mandatory:
D.P. Bertsekas, Dynamic Programming and Optimal Control, 2 Vols., Athena Scientific Press, 2005
M. Wiering and M.V. Otterlo, Reinforcement Learning: State-of-the-art, Volume 12, Springer Science & Business Media 2012.
Registration
You have to sign up for courses and exams (including retakes) in uSis. Check this link for information about how to register for courses.
Please also register for the course in Blackboard as soon as the lecturer has made it available.
There is limited space for students who are not enrolled in the Computer Science programme or one of the Data Science specialisations (Data Science: Computer Science and Astronomy and Data Science). Please contact the programme coordinator/study advisor (mailto:mastercs@liacs.leideuniv.nl) if you are an external student.
Contact
Lecturer: dr. Kaifeng Yang.
Website: Blackboard