About
Constructed-response (‘open-ended’) items constitute an integral part of the PISA assessment: they allow assessing higher-order cognitive skills in some domains, present a change from closed-response (e.g. multiple-choice) items, allow for partial grading, prevent students from guessing right, and permit to gather information that would not be possible using a closed-response format. With the exception of some short, numeric answers, the resulting text responses require human coding before the data can be analysed. The process of coding is, however, prone to error, time consuming, and expensive.
This project aims to introduce a system for automatically coding open-text responses into the operational procedures in PISA. It will also evaluate the feasibility of using artificial intelligence (AI), and in particular natural-language processing (NLP) methods for coding longer text responses.
Contributors
- Gisele Alves (Instituto Ayrton Senna, Brazil)
- Nico Andersen (DIPF, Germany)
- Roger Beaty (Pennsylvania State University, United States)
- Mathias Benedek (Universität Graz, Austria)
- Denis Dumas (University of Georgia, United States)
- Peter Organisciak (University of Denver, United States)
- John D Patterson (Pennsylvania State University, United States)
- Ricardo Primi (Universidade São Francisco, Brazil)
- Ricelli Silva (University of São Paulo, Brazil)
- Fabian Zehner (DIPF, Germany)