Admission requirements
Admission requirements and any restrictions. The course is open to MA students with an active interest in deep learning-based approaches to natural language. Exposure to AI (esp. neural networks), statistics and computational linguistics is desired but not critical.
Description
Brief introductory description of the course. Please include course subject and teaching materials used. The course will provide a historical perspective on deep learning for natural language processing (NLP), and will address recent topics such as Transformers (BERT), attention-based models and recent models for dialogue. In addition, we will discuss language acquisition, the cognitive plausibility of AI models, and the extraction of semantic structure from raw text. We will take a look at the current revival of linguistic structure in the deep learning community, either through explicit modeling on the basis of annotated data or through the analysis of attention patterns in Transformers (according to which linguistic structure is a 'by-product' of neural attention).
We will go through a bit of theory in the first hour of every lecture, and proceed with a discussion of
recent literature in the second hour, with an active role for students (in pairs if attendance numbers
allow, otherwise individually) will introduce a paper on the collective reading list, which will be up for
discussion by the group. In the case of insufficient student numbers, students will have repeated
presentation turns. The course concludes with an assignment for writing a critical review paper (8 pages) on a chosen topic. These papers will be presented at the end of the course in class by the students with a (short) Powerpoint presentation. All paper contributions will be combined by the students into a coherent, joint (overview) paper (max. 30 pages): students will integrate the gist of their separate papers and collaborate on textual integration.
Course objectives
Concise description of the course objectives formulated in terms of knowledge, insight and skills students will have acquired at the end of the course. The relationship between these objectives and achievement levels for the programme should be evident.
Students will obtain knowledge about the historical and current trends in deep learning-based NLP. They will be able to take a critical look at current literature, and will have a basic understanding of the challenges, opportunities and pitfalls of deep learning applied to language. Students will have obtained knowledge about the connections of artificial cognitive models and natural cognition upon completion of the course. A secondary skill that students will obtain is the ability to summarize and present complex literature, orally to an audience and in the form of papers. The joint class paper builds a skill for collaboration and joint paper composition.
Timetable
Visit MyTimetable.
Mode of instruction
Seminar
Assessment method
Papers and presentation
Assessment
The final grade will depend on a review of the submitted papers (the personal paper, and the contribution
to/integration in the group paper) plus the in-class presentation. Clarity of presentation, the ability to put
a topic in context, the ability to provide a critical review, and the quality of textual integration of the
separate paper in the group paper are evaluation criteria.
Weighing
The separate papers have a weight factor of 3; the oral paper presentation has a weight factor of 2; the
integration of the separate paper in the group paper has a weight factor of 1.
Resit
The resit consists of resubmitting the separate paper, and updating the group paper with the next text.
Inspection and feedback
How and when an exam review will take place will be disclosed together with the publication of the exam results at the latest. If a student requests a review within 30 days after publication of the exam results, an exam review will have to be organized.
Reading list
The booktitles and / or syllabi to be used in the course, where it can be purchased and how this literature should be studied beforehand.
A literature list will be made available in thru Brightspace.
Registration
Enrolment through uSis is mandatory.
General information about uSis is available on the
[website](https://www.student.universiteitleiden.nl/en/administrative-matters/registration--de-
registration/course-and-exam-enrolment/course-and-exam-enrolment/humanities#tab-2).
Contact
Contact information Prof.dr. S.A. Raaijmakers Education Administration Office Reuvensplaats
E-mail address Education Administration Office Reuvensplaats: osz-oa-reuvensplaats@hum.leidenuniv.nl