nl en

Evaluating an Empirical Study


Important Note

  • All Semester II bachelor and master psychology courses and examinations (2020-2021) will be offered in an on-line format.

  • If it is safe and possible to do so, supplementary course meetings may be planned on-campus. However, attendance at these meetings will not be required to successfully complete Semester II courses.

  • All obligatory work groups and examinations will be offered on-line during Central European Time, which is local time in the Netherlands.

  • Information on the mode of instruction and the assessment method per course will be offered in Brightspace, considering the possibilities that are available at that moment. The information in Brightspace is leading during the Corona crisis, even if this does not match the information in the Prospectus.

Entry requirements

Open to MSc Psychology (research) students


The aim of this course is to teach students in the research master programme – regardless of their area of specialisation – about a variety of approaches and methods that can be used in evaluating and designing psychological research. This is intended to broaden students’ views about the possibilities to address different kinds of research questions. The course meetings and assignments are intended to make students more aware of the added value of creatively combining different research traditions, and to enhance their ability to consider, combine, and apply multiple approaches within a single research design.

Course objectives

The course aims to:

  • Provide an overview of the different types of experimental methods used in psychological research, including their strengths and weaknesses;

  • Increase students’ knowledge and skills necessary for properly evaluating the design of a published empirical study;

  • Teach students to provide recommendations for improving a published research design.


For the timetables of your lectures, work group sessions, and exams, see the timetables page of your study programme. You will also find the enrolment codes here. Psychology timetables



Students need to enroll for the lectures. Master’s course registration

Mode of instruction

This course consists of:

  • 8 2-hour lectures (four modules of two sessions each)

  • 1 plenary meeting.

Attendance is mandatory.

Assessment method

Students will evaluate a paper of their own choice and suggest improvements regarding design and method.

The Institute of Psychology follows the policy of the Faculty of Social and Behavioural Sciences to systematically check student papers for plagiarism with the help of software. Disciplinary measures will be taken when fraud is detected. Students are expected to be familiar with and understand the implications of this fraud policy.

Reading list

The literature will consist of 2-3 articles per module: one article to introduce the method and one or two to describe it in more detail.
Each of the four modules consists of two meetings. The first meeting will be an introduction into the topic(s) to be discussed and the second meeting will focus on practising the application of the main topic(s).

Module 1: Hypothesis generation and levels of explanation
The aim of this module is to increase the sensitivity of the students with respect to two important issues. The first concerns the process of hypothesis generation, something that is often overlooked in methodological courses. How does one generate a good idea for an empirical study? Even though psychological research is often said to aim for testing hypotheses about human mind and behavior, hypotheses are actually often derived after the fact—that is, after the relevant data have been collected. We will discuss the logical and ethical implications of this practice. The second issue relates to the fact that the human mind and behavior can be explained at different levels: at a phenomenological, functional, or physiological level. They all provide potentially useful insights into psychological processes but, unfortunately, many researchers confuse these levels and consider some as “causing” others. We will discuss examples for, and the implications of this classical logical error. This module aims to stimulate critical thinking and controversial discussion.
Teacher: Prof. Dr. S. Nieuwenhuis

Key readings:

  • Kerr, N. L. (1998). HARKing (Hypothesizing After the Results are Known). Personality and Social Psychology Review, 2, 196-217.

  • Weisberg, D.S., Keil, F.C., Goodstein, J., Rawson, E., & Gray, J. (2008). The seductive allure of neuroscience explanations. Journal of Cognitive Neuroscience, 20, 470-477.

Module 2: Exploring the brain
Functional neuroimaging can be a useful tool for testing psychological theories. This module will cover the methodological aspects of designing and analyzing an experimental task for the use of functional magnetic resonance imaging (fMRI). The module will focus on the following questions: 1) Which theories can be tested with fMRI? 2) What are the experimental requirements for testing these theories? 3) How can we evaluate fMRI research? These questions will be illustrated with data examples from various research areas.
Teacher: Dr. A.C.K. van Duijvenvoorde

Key readings:

  • Poldrack, R. A. (2012). The future of fMRI in cognitive neuroscience. Neuroimage, 62, 1216-1220.

Module 3: Evaluating interventions
In this module we will discuss the strengths and weaknesses of different designs to evaluate interventions, ranging from case reports to randomized controlled trials. Examples will be drawn from the clinical and medical research areas, but the principles are applicable to other interventions as well. The standards for conducting and reporting intervention research will also be addressed.
Teacher: Dr. V. Ly

Key readings:

  • Boutron, I. et al. (2008). Methods and processes of the CONSORT group: Example of an extension for trials assessing nonpharmacologic treatments. Annals of Internal Medicine, 148, 295-309.

  • Vandenbroucke, J.P. (2008). Observational research, randomized trials, and two views of medical science. PLoS Medicine, 5: e67.

Module 4: Observation and simulation
In this module, we will focus on understanding rich social situations using standardized and controlled studies. We will address the question of how theoretically meaningful aspects of social situations can be simulated or re-created in the lab (experimental simulations and games), to observe their effects under highly controlled circumstances. We will use specific research examples to practice evaluating and designing empirical studies. Moreover, we will discuss differences between fundamental (process-focused) and applied (problem-focused) research to learn how standardized and controlled studies relate to reality.
Teacher: Dr. M.K. Noordewier

Key readings:

  • Wilson, T. D., & Aronson, E., & Carlsmith, K. (2010). The art of laboratory experimentation. In S. Fiske, D. Gilbert, & G. Lindzey (Eds.), The handbook of social psychology (5th ed., vol. 1, pp. 49-79). New York: Wiley.

  • Reis, H.T., & Gosling, S.D. (2010). Social psychological methods outside the laboratory. In S. Fiske, D. Gilbert, & G. Lindzey, (Eds.), The handbook of social psychology (5th ed., vol. 1, pp. 82-114). New York: Wiley.

  • An empirical article (to be announced) that we will evaluate and re-design in the second meeting.

Contact information

Dr. Michiel van Elk