Admission requirements
Students should have passed statistics and probability or "Inleiding Kansrekening" and "Inleiding Mathematische Statistiek".
Students should have at least basic programming skills in R.
Description
Have you ever wondered how we can answer the question: should I shop at supermarket A or supermarket B? Classical statistical learning assumes we have a dataset ready to use from which we can learn to answer such a question, but in many real-world applications we don’t have any data readily available which we can use. For example, if I want to know which supermarket I prefer, I have to collect data. However, while I am collecting data I might suffer from buying my groceries at the suboptimal supermarket. So the question becomes, how can I simultaneously collect data while getting the best groceries possible. This is precisely one of the main topics of interactive machine learning: how can we gain information while simultaneously making good predictions, i.e. trade-off exploration and exploitation.
One of the fundamentally unique perspectives we obtain from studying interactive machine learning is that every time we make a prediction, as long as we get feedback on our prediction, we can learn from how the world reacts to our prediction. In this way, we can continuously learn and improve our models. We will study several types of feedback to our predictions: full information feedback, stochastic bandit feedback, and nonstochastic bandit feedback.
With full information feedback we get feedback on all possible predictions. In this feedback model we will learn about three foundational algorithms: the exponential weights algorithm, the online gradient descent algorithm, and the perceptron algorithm. Strikingly, we will see that even without any statistical assumptions we can predict almost as well as the best possible prediction.
With stochastic bandit feedback we only get feedback about the prediction that we actually made, with the additional assumption that the rewards for our predictions come from some distribution. In this feedback model we will learn about the UCB algorithm, which is used throughout the internet. We will also learn how to use as little data as possible to decide which option we should use in simple decision problems.
With nonstochastic bandit feedback we only get feedback about the prediction that we actually made, without any additional assumptions on the rewards for our predictions. Because we make no additional assumptions we can adequately handle malicious users of our deployed machine learning models. In this feedback model, we will study the EXP3 algorithm and the Banditron algorithm, which are extensions of the exponential weights algorithm and the perceptron algorithm, respectively, designed to learn from bandit feedback.
Course Objectives
Learn how to:
collect data while simultaneously issuing predictions
learn from feedback to improve your models
trick statistical learning algorithms into making mistakes
deal with malicious users
exploit your knowledge of statistics in machine learning
Timetable
In MyTimetable, you can find all course and programme schedules, allowing you to create your personal timetable. Activities for which you have enrolled via MyStudyMap will automatically appear in your timetable.
Additionally, you can easily link MyTimetable to a calendar app on your phone, and schedule changes will be automatically updated in your calendar. You can also choose to receive email notifications about schedule changes. You can enable notifications in Settings after logging in.
Questions? Watch the video, read the instructions, or contact the ISSC helpdesk.
Note: Joint Degree students from Leiden/Delft need to combine information from both the Leiden and Delft MyTimetables to see a complete schedule. This video explains how to do it.
Mode of Instruction
Lectures, exercise classes, and homework
Assessment method
The assessment consists of homework (10%) and a written (re)exam (90%). The homework counts as a practical exercise, for which no resit is possible. In order to pass the course, the grade for the (re)exam must be at least 5, and the (unrounded) final grade must be at least 5.5. There is no minimum grade for the homework required to be admitted to the (re)exam.
Reading list
Lectures notes will be made available on Brightspace.
Registration
As a student, you are responsible for enrolling on time through MyStudyMap.
In this short video, you can see step-by-step how to enrol for courses in MyStudyMap.
Extensive information about the operation of MyStudyMap can be found here.
There are two enrolment periods per year:
Enrolment for the fall opens in July
Enrolment for the spring opens in December
See this page for more information about deadlines and enrolling for courses and exams.
Note:
It is mandatory to enrol for all activities of a course that you are going to follow.
Your enrolment is only complete when you submit your course planning in the ‘Ready for enrolment’ tab by clicking ‘Send’.
Not being enrolled for an exam/resit means that you are not allowed to participate in the exam/resit.
Contact
Dirk van der Hoeven - d.van.der.hoeven@math.leidenuniv.nl
Remarks
Software
Starting from the 2024/2025 academic year, the Faculty of Science will use the software distribution platform Academic Software. Through this platform, you can access the software needed for specific courses in your studies. For some software, your laptop must meet certain system requirements, which will be specified with the software. It is important to install the software before the start of the course. More information about the laptop requirements can be found on the student website.