|
After taking this course students:
- demonstrate ability in carrying carry out classical item-test analyses and in interpreting the meaning of difficulty parameters (value), discrimination (item-total correlation)
- demonstrate ability in carrying out item-test analyses based on item-response theory, in interpreting the meaning of parameters, including item difficulty (beta), discrimination (a-parameter), item and test information parameters, using dedicated software
- demonstrate ability in in carrying out analyses on test data aimed at detecting sources of bias or lack of accessibility in items (such as DIF-analysis)
- demonstrate ability in carrying out analyses using Shiny Item analysis and R-studio, under conditions of strong teacher support
- demonstrate insight into the principles of systematic design of assessment of and for learning, which in turn aid valid decisions about the test-taker
- demonstrate ability to design items in a subject domain, using the principles taken from the ECD sub models.
- demonstrate ability to judge the accessibility and construct coverage of items in a subject domain, using scientific principles form literature and take consequential measures in item improvement
- will gain insight in large-scale assessments methodology and analysis (such as PISA and PIRLS)
- will gain insight in recent developments of valid game-based assessment and simulations
- will gain insight in the most important methods of standard setting, both classical methods for multiple-choice tests and methods for complex performance-based assessments, as applied on existing test data.
- demonstrate ability to reflect critically on the quality of own constructed educational assessments and educational assessment of others, using the ECD-model and test accessibility principles as frames of reference.
Relationship with EST labour market (Intended Learning Outcomes [ILO's] as described in the EST programme specific appendix of the EER):
- This course contributes (strongly) to: Domain expertise, Design competency
- To some extent, this course contributes to: Research competence, Academic reflection,
- To a limited extent this course contributes to Advice competency,
This courses assesses the following ILO's in some way (can include formal/informal, formative/summative, peer/expert): Domain expertise, Research competence.
|
 |
|
The course aims to develop hands-on skills in both construction and evaluation of assessments, that serve different user goals and on individual, group and system level. In order to create a firm theoretical basis for assessment construction and item-assessment analysis a collection of journal articles, book chapters and self-produced study texts are offered, related to the topics of the workshops meetings.
Throughout the course the perspective of Evidence-centred assessment design is used, which can be considered as state-of-the art both in assessment theory and practice. Arriving at valid inferences about (groups of) test-takers, that are informative for further actions is a key focus of this perspective.
Students will get to know the professional field of educational measurement including the contemporary major interest and innovations. During the course both known experts and practitioners in the field of Educational Measurement will provide guest workshops. For example from companies like the educational measurement institute Cito and Explain. For each of these topics, students are expected to study selected scientific papers.
Students are expected to draw from the theoretical and practical insights gained through self-study and participation in the workshops when they carry out two intermediate assignments and a long-term item construction assignment.
Students get support and feedback in small group meeting with the course teacher, scheduled by the student groups. Also, students are encouraged to give peer-feedback by serving as one another’s pilot test-takers and colleague experts during the course and the final mini-symposium on their item construction.
Assessment:
Students carry out two short intermediate assignments directed at the application of psychometric methods used to determine item and assessment quality. Each of these assignments account for 15% of the final grade. In addition, students carry out a long term group assignment in which they construct a set of 20 original test items for the subject domain chosen in this course, reading comprehension. As in practice they write a reflective report in which they provide an overview of the test content and a justification of construction decisions made in the perspective of evidence-centred design for the particular subject domain. This long term assignment accounts for 60% of the final grade. The presentation of the work done during the final meeting to peers accounts for 10% of the grade. Each of the assignments mentioned needs to be completed at a sufficient level (minimum grade 5,5). The final grade for this course is based on the weighted mean of the four assignment grades.
Relationship with technology:
- Item construction: TAO-testing – Open source software;
- Creating item objects (stimuli, response objects, texts): any drawing or animation software inside Adobe Suite and Microsoft Office Suite;
- Psychometric Item-test analyses: R-based application Shiny Item analysis and R-studio;
- Collection and analysis of item meta data: Microsoft Excel and Microsoft Access
|
|
| |
|
|
Enrolment in this course is subject to the following requirements:
The student has successfully completed the UT’s pre-M programme Educational Science and Technology (pre-M EST) (or equivalent/related BSc programme, to be decided by the M-EST admission committee)
In case you, as a student from another master’s programme, want to participate in this course, please contact the EST staff: y.c.h.dethouars@utwente.nl or m.e.tijhuis@utwente.nl |
| Master Educational Science and Technology |
| | Required materialsCourse material| Selected course readings (via Canvas) |
| Course material| Selected test content and related documents (via Canvas |
|
|
Recommended materials-Instructional modesLecture| Presence duty | | Yes |
| Self study without assistance
| Tutorial| Presence duty | | Yes |
|
|
TestsIntermediate assignment and long term
|
|
| |