| Summary: | Part of a special issue on computer-based performance assessment of problem solving. A number of field trials were conducted to explore test design and fairness issues related to the development of a computer-based performance assessment of problem solving in an architectural context. Examinees in an architectural licensure examination were given design problems consisting of a set of specifications and code requirements, and they were required to construct solutions on the computer using a set of provided computer tools. The fairness issue of whether examinees with relatively little computer experience are disadvantaged on the computer-delivered exam did not appear to be a big problem. A more complex fairness issue that emerged is the need to create tasks that are sufficiently comparable so that it does not matter which set of tasks examinees receive. In addition, tasks intended to measure the same skills need to be of comparable difficulty and highly correlated. A number of inconsistencies emerged in the field trials in relation these issues, and determining reasons for this was crucial in setting the final test specifications for the operational examination.
|
|---|