Admission requirements
None.
Description
The modern successes of machine learning are to a large extent based on earlier well developed physics insights (See Physics Nobel Prize 2024). This course explores this physics foundation of in machine learning up to recent advances int the field. We discuss machine learning as a generalization of the thermodynamics of spin systems and optimization as the search for a physical ground state. The interplay between physics and machine learning becomes clear in analyzing optimal Deep Neural Networks as those that sit on an edge of chaos phase transition; how the optimal state in Deep Neural Networks saturates the maximal possible information bottleneck bound, and how formally Deep Neural Networks are equivalent to disordered quantum field theories. Alternatively, using machine learning insights for physics, Restricted Boltzmann Machines are shown to be highly efficient variational wavefunctions for densely entangled quantum groundstates. The recent Large Language Models are shown to be equivalent to self-re-inforcing spin systems.
Topics:
Entropy and Information
Learning and deep Neural Networks as generalized Ising models
Optimization MCMC and Restricted Boltzmann Machines
Entropy, Information and the Information Bottleneck
The phase transition of Double Descent
The QFT/Neural Network correspondence and the Edge of Chaos.
Course objectives
The course will consist of two parts:
The first part of the course will consist of a set of conventional lectures and (graded) assignments;
The second part of the course will consist of a set of independent research projects with a written report and presentations by the students;
Schedule
The timetables are available through My Timetable (see the button in the upper right corner).
Teaching method
See Brightspace
Assesment method
The final grade will be determined as follows:
Graded assignments (20%)
Project Presentation (30%)
Project report (50%)
Resit, review & feedback
Examinations are held twice during the academic year for each component offered in that academic year. Midterm tests cannot be retaken. The Board of Examiners determines the manner of resit for practical assignments.
For review and feedback, see Brightspace.
Reading list
All materials (this syllabus, problem assignments, background material) can be found on the Leiden University Brightspace site.
Registration
Enrolment through MyStudyMap (button in upper right corner) is mandatory. General information about course and exam enrolment is available on the website.
Contact
For substantive questions, contact the lecturer(s) (listed in the right information bar).
Remarks
Software
Starting from the 2024/2025 academic year, the Faculty of Science will use the software distribution platform Academic Software. Through this platform, you can access the software needed for specific courses in your studies. For some software, your laptop must meet certain system requirements, which will be specified with the software. It is important to install the software before the start of the course. More information about the laptop requirements can be found on the student website.