A Primer on Computer-Adaptive Testing
What are the key components of computer adaptive testing (CAT)? What differentiates this method from conventional testing? A conventional, or fixed-form, test administers the same items across students and can be taken using either a paper-and-pencil form or on a computer. An adaptive test adjusts the difficulty based on student performance, administering different test questions of varying difficulty to test takers.
This primer highlights the key components of CAT and differentiates CAT from traditional testing methods. The final section focuses on additional considerations for CAT as it relates to educational testing.
Overview of CAT
One of the foundations of adaptive testing is that the testing experience be dynamic and sensitive to the performance of the test taker. The idea of adaptive or adjustable testing is not new and was first implemented by Binet back in 1912 with his intelligence test. Binet’s method made each successive question reflect past performance and allowed his IQ test to be both efficient and able to match the skill of the respondent.
The availability of computers made it possible to improve on Binet’s method by use of Item Response Theory models (IRT) that improved the sensitivity of the test to student performance. The key advance was to automate the scoring process and remove the need for a human proctor as was the case with Binet’s IQ test. The IRT models in combination with the personal computer made CAT a method that could be used in the classroom or on secured mobile devices.
Advantages of CAT over Conventional Methods
As more schools have switched to computer-based testing it is important to distinguish a computer-based fixed-form test from a CAT. In a computer-based fixed-form test, students will complete a fixed number of test questions. Each student may or may not see the same questions and the item order can be randomized to discourage cheating. These computer-based tests share some benefits with CAT including immediate scoring and paper resource savings.
Adaptive testing offers additional benefits compared to the computer-based fixed-form test. The following four elements of a CAT provide context to the discussion of the advantages of CAT over other testing methods.
- Item selection that adapts to performance. With a CAT, it is possible to select new items using feedback from previous responses. This means that the CAT will become easier if the student is struggling and more difficult if the student is performing well. The goal is to select items that are appropriate for test takers thus reducing examinee frustration or boredom. A conventional test must balance measuring some students well or every student with less precision.
- Improved efficiency. Adaptive testing makes it possible to administer questions near a students’ ability level and provides more information about student performance. This improves the precision of measurement and reduces the number of questions a student must answer. Students need less time in front of computer to complete the test and teachers have more time for instruction. The end result is a test that is both shorter and more accurate than a standard conventional test.
- Test security and item bank maintenance. The matching of items to students results in different item content being seen by different students and reduces the risk of cheating in the classroom. The reduced test length for CAT also reduces item exposure and the risk of students being coached on frequently seen items.
- Results delivered immediately. Another advantage of computer-based testing is the immediate calculation of ability level and feedback for both students and teachers. Additionally, when CAT is used for formative assessment, a student’s past performance can be used as the starting point for subsequent exams to measure growth and target instruction to areas of weakness.
Characteristics of Computer Adaptive Testing
Adaptive testing provides numerous benefits relative to conventional tests. Characteristics of a CAT that should be considered before implementation include the following:
- Students cannot skip questions or perform a posttest review. One difference between conventional and adaptive testing is that adaptive testing does not allow students to skip and return to questions later. Conventional tests allow a student to return to a question later after completing other items while CAT requires the test questions to be answered as they appear. Unlike conventional testing methods, student performance on an adaptive test does not rely on the number-correct score for determining student performance. As described above, each question given to the student depends on the answer to the previous question and therefore the entire response pattern is taken into consideration when determining the student’s overall performance.
- Item bank size limits application of CAT. The number of test questions required to cover all the skills taught in school over a range of difficulty renders CAT a method limited to companies with substantial resources. A large effort is required to maintain a CAT item bank and keep the item content fresh and up-to-date as well as secure. However, the cost of developing and maintaining a CAT item bank is worth the effort as it improves student measurement and satisfaction with the testing process.