Admission requirements
The prerequisites include basic linear algebra (e.g., vector space and matrix operations), calculus (mainly differentiation), and Python programming. No background of machine learning or artificial intelligence is required.
Description
Neural computation and neural networks are biologically inspired computational models that have been extensively used to solve numerous challenging problems, such as image classification, machine translation, and time-series forecasting. Rooted in computational neuroscience, neural networks (NNs) accomplish these tasks by mimicking the way information is processed in the human brain.
This course aims to provide a concise yet rigorous overview of the entire field, focusing on explaining the fundamental principles that empower NNs. It will also cover widely applied NN models, such as the well-known multi-layer feedforward network and long short-term memory. On the practical side, various examples and coding exercises will be provided to the students.
- Introduction and Biological Background
- Hopfield model
- Multi-layer Perceptron
- Back-propagation
- Recurrent Neural Networks
- Convolutional Neural Networks
- Generative Models
Course objectives
The course offers a comprehensive overview of the field through a series of lectures and programming exercises. Once students have gained some experience with the exercises, we will assign a practical project on implementing and training neural networks. This will help deepen their understanding of the subject.
Upon successful completion of the course,
You can explain the main working principles of the biological neuron and how it ispirated the McCulloch-Pitts model.
You are able to explain the working principles of the Hopfield Network and provide examples using the following concepts: network topology, stable patterns, basin of attraction, capacity, energy function.
You are able to state and prove the main theoretical statements about Hopfield networks and single-layer perceptrons: attractor weight definition, stability condition, central property of the energy function.
You are able to state the difference between classification and regression and to provide examples for both cases.
You can explain the fundamentals of multi-layer perceptrons and functioning mechanism blocks: input/hidden/output layers, weights, bias, activation function, etc.
You can reproduce the logical flow of the back-propagation algorithm and are able to apply it on practical examples.
You recognize the difference between tabular data and time series and the type of neural networks that are useful in the second scenario. You can explain how Recurrent Neural Network work and define topology, functioning, unfolding, LSTMs, extension of previous concepts to this case.
You can discuss how Convolutional Neural Networks are useful and powerful. You are also able to illustrate how they work and why they allow for computational savings with respect to multilayer perceptron and why.
You are able to implement all seen neural networks using python libraries and exploit them for prediction tasks.
You can empirically prove the accuracy of implemented prediction models.
Timetable
In MyTimetable, you can find all course and programme schedules, allowing you to create your personal timetable. Activities for which you have enrolled via MyStudyMap will automatically appear in your timetable.
Additionally, you can easily link MyTimetable to a calendar app on your phone, and schedule changes will be automatically updated in your calendar. You can also choose to receive email notifications about schedule changes. You can enable notifications in Settings after logging in.
Questions? Watch the video, read the instructions, or contact the ISSC helpdesk.
Note: Joint Degree students from Leiden/Delft need to combine information from both the Leiden and Delft MyTimetables to see a complete schedule. This video explains how to do it.
Mode of instruction
Lectures and practical sessions.
Hours of Study: 114 (= 6 EC)
Lectures: 14
Programming exercises: 3
Practical work: 48
Exam and preparation: 30
Assessment method
The final grade is a combination of grades for (1) the written exam (70%) and (2) the practical assignment (30%).
Both the written exam and the practical assignment are mandatory. The final grade for the course is determined by calculating the weighted average. However, the exam grade must be above the passing mark of 5.5. Students have the opportunity to retake the exam but not the assignment.
Reading List
The following titles are recommended but not mandatory for the course:
Hertz, John, Anders Krogh, and Richard G. Palmer. Introduction to the theory of neural computation. CRC Press, 2018.
Bishop, Christopher M, and Hugh Bishop. Deep Learning, Foundations and Concepts. Springer, 2023. https://www.bishopbook.com
Goodfellow, Ian, Yoshua Bengio, and Aaron Courville. Deep learning. MIT press, 2016. https://www.deeplearningbook.org/
MacKay, David JC, and David JC Mac Kay. Information theory, inference and learning algorithms. Cambridge university press, 2003. https://www.inference.org.uk/itprnn/book.pdf (Only Chapter 5 is relevant to the course. This is very good in the learning theory, which contains various advanced and insightful discussions)
Registration
As a student, you are responsible for enrolling on time through MyStudyMap.
In this short video, you can see step-by-step how to enrol for courses in MyStudyMap.
Extensive information about the operation of MyStudyMap can be found here.
There are two enrolment periods per year:
Enrolment for the fall opens in July
Enrolment for the spring opens in December
See this page for more information about deadlines and enrolling for courses and exams.
Note:
It is mandatory to enrol for all activities of a course that you are going to follow.
Your enrolment is only complete when you submit your course planning in the ‘Ready for enrolment’ tab by clicking ‘Send’.
Not being enrolled for an exam/resit means that you are not allowed to participate in the exam/resit.
Contact
Lecturers: Dr. E. Raponi
Website: See course page on Brightspace.
Remarks
Software
Starting from the 2024/2025 academic year, the Faculty of Science will use the software distribution platform Academic Software. Through this platform, you can access the software needed for specific courses in your studies. For some software, your laptop must meet certain system requirements, which will be specified with the software. It is important to install the software before the start of the course. More information about the laptop requirements can be found on the student website.