Admission requirements
The prerequisites include basic linear algebra (e.g., vector space and matrix operations), calculus (mainly differentiation), and Python programming. No background of machine learning or artificial intelligence is required.
Description
Neural computation and neural networks are biologically inspired computational models that have been extensively used to solve numerous challenging problems, such as image classification, machine translation, and time-series forecasting. Rooted in computational neuroscience, neural networks (NNs) accomplish these tasks by mimicking the way information is processed in the human brain.
This course aims to provide a concise yet rigorous overview of the entire field, focusing on explaining the fundamental principles that empower NNs. It will also cover widely applied NN models, such as the well-known multi-layer feedforward network and long short-term memory. On the practical side, various examples and coding exercises will be provided to the students.
- Introduction and Biological Background
- Hopfield model
- Multi-layer Perceptron
- Back-propagation
- Recurrent Neural Networks
- Convolutional Neural Networks
- Generative Models
Course objectives
The course offers a comprehensive overview of the field through a series of lectures and programming exercises. Once students have gained some experience with the exercises, we will assign a practical project on implementing and training neural networks. This will help deepen their understanding of the subject.
Upon successful completion of the course,
You can explain the main working principles of the biological neuron and how it ispirated the McCulloch-Pitts model.
You are able to explain the working principles of the Hopfield Network and provide examples using the following concepts: network topology, stable patterns, basin of attraction, capacity, energy function.
You are able to state and prove the main theoretical statements about Hopfield networks and single-layer perceptrons: attractor weight definition, stability condition, central property of the energy function.
You are able to state the difference between classification and regression and to provide examples for both cases.
You can explain the fundamentals of multi-layer perceptrons and functioning mechanism blocks: input/hidden/output layers, weights, bias, activation function, etc.
You can reproduce the logical flow of the back-propagation algorithm and are able to apply it on practical examples.
You recognize the difference between tabular data and time series and the type of neural networks that are useful in the second scenario. You can explain how Recurrent Neural Network work and define topology, functioning, unfolding, LSTMs, extension of previous concepts to this case.
You can discuss how Convolutional Neural Networks are useful and powerful. You are also able to illustrate how they work and why they allow for computational savings with respect to multilayer perceptron and why.
You are able to implement all seen neural networks using python libraries and exploit them for prediction tasks.
You can empirically prove the accuracy of implemented prediction models.
Timetable
You will find the timetables for all courses and degree programmes of Leiden University in the tool MyTimetable (login). Any teaching activities that you have sucessfully registered for in MyStudymap will automatically be displayed in MyTimetable. Any timetables that you add manually, will be saved and automatically displayed the next time you sign in.
MyTimetable allows you to integrate your timetable with your calendar apps such as Outlook, Google Calendar, Apple Calendar and other calendar apps on your smartphone. Any timetable changes will be automatically synced with your calendar. If you wish, you can also receive an email notification of the change. You can turn notifications on in ‘Settings’ (after login).
For more information, watch the video or go the the 'help-page' in MyTimetable. Pleas note: Joint Degree students Leiden/Delft have to merge their two different timetables into one. This video explains how to do this.
Mode of instruction
Lectures and practical sessions.
Hours of Study: 114 (= 6 EC)
Lectures: 14
Programming exercises: 3
Practical work: 48
Exam and preparation: 30
Assessment method
The final grade is a combination of grades for (1) the written exam (70%) and (2) the practical assignment (30%).
Both the written exam and the practical assignment are mandatory. The final grade for the course is determined by calculating the weighted average. However, the exam grade must be above the passing mark of 5.5. Students have the opportunity to retake the exam but not the assignment.
Reading List
The following titles are recommended but not mandatory for the course:
Hertz, John, Anders Krogh, and Richard G. Palmer. Introduction to the theory of neural computation. CRC Press, 2018.
Bishop, Christopher M, and Hugh Bishop. Deep Learning, Foundations and Concepts. Springer, 2023. https://www.bishopbook.com
Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016. https://www.deeplearningbook.org/
MacKay, David JC, and David JC Mac Kay. Information theory, inference and learning algorithms. Cambridge university press, 2003. https://www.inference.org.uk/itprnn/book.pdf (Only Chapter 5 is relevant to the course. This is very good in the learning theory, which contains various advanced and insightful discussions)
Registration
From the academic year 2022-2023 on, every student has to register for courses with the new enrollment tool MyStudymap. There are two registration periods per year: registration for the fall semester opens in July and registration for the spring semester opens in December. Please see this page for more information.
Please note that it is compulsory to register for every exam and retake. Not being registered for a course means that you are not allowed to participate in the final exam of the course.
Extensive FAQ on MyStudymap can be found here.
Contact
Lecturers: Dr. E. Raponi
Website: See course page on Brightspace.
Remarks
Not applicable.