During COVID lockdown, many university instructors struggled with the shift to online teaching and testing. The tools they had available, such as the testing engines inside their learning management systems (LMSs) and the traditional in-class paper testing they’d been relying on for years, did not meet the needs of an all-online environment. It was a scary new world for many.
Adding to the difficulties, most universities did not have available funding for a more effective testing solution. However, relief is in sight: federal funding is available to colleges and universities through the Higher Education Emergency Relief Fund. This fund provides grants and other options to help improve online instruction, including assessment of student learning.
With funding available, what should you look for when choosing a more robust, instructor-led assessment solution?
1. Get More from Your Item Types
Look for a wide variety of item types beyond multiple choice, true/false, or fill-in-the-blank (FITB). While these types have definitely stood the test of time, they may not be sufficient to assess student mastery at the higher levels of cognitive complexity required in college courses (Webb’s DOK 3, Bloom’s Analysis and above). By carefully designing traditional items and expanding your use of more complex item types, you can create a cheat-resistant, open-book test.
2. Identify Academic Dishonesty
Cheating can be a serious problem for most college instructors. Although online proctoring can help, you need robust reporting that enables you to use cascading details to identify cheaters and avoid false positives. The diagram below shows the kind of analysis you can perform:
If you answer “yes” to all of these questions for one or more students, it’s time for a deeper investigation by Academic Senate or Student Ethics committees. If the test engine does not provide these details, you won’t have the evidence.
3. Make Sure Cumulative Item Statistics Are Available
Updating and maintaining items to measure students more accurately and to capture changes in instruction can be a challenge for faculty. You want to keep what’s working and be able to easily identify what needs improvement.
A testing solution that provides both cumulative item statistics (stats that reflect each item’s use in all tests) and test item statistics (stats that reflect the item’s use in a single test) can help you determine whether an item simply didn’t work for the most recent course session or whether the item truly needs to be revised.
4. Rescore Individual Tests for Accuracy
No matter how carefully you develop your test items, automated scoring issues may slip through. You need a system where it is easy to see the entire suite of student responses so you can identify and correct these issues.
5. Link Individual Test Items to Standards and Outcomes
In most LMSs, it’s possible to link assignments and tests to department-defined student outcomes and accreditation standards. However, many do not provide a method to do so for individual test items. As higher education increases its focus on measuring achievement on learning outcomes and satisfying accreditation standards, being able to show directly how each item on your test aligns becomes more crucial.
While a better testing engine may not solve all online assessment issues (for example, cheating remains a significant issue), better testing combined with an increased understanding of effective online test development can make testing in college classes better for students and instructors.