Navigating the Evaluation Landscape:
Understanding Interest Groups and Their Evaluation Needs
Many players participate in course evaluations. Each of these players has a particular focus and set of needs that an evaluation system must meet to be successful.
Through close cooperation with a number of universities and researchers in higher education, we have identified some of the requirements inherent to an automated system for evaluating instruction.
An institution’s executive leadership wants to provide:
- Continuous quality improvement
- Support for “learning from the best”
- Support for accreditation measures
- Open procedures
- An ability to compare faculty results and make organizational development more effective
These leaders want to minimize additional work for faculty members who support a service-oriented concept for implementing the evaluation. They would like an automated process that enables surveys to be carried out on a regular basis so they can find trends and tendencies, strengths and weaknesses between instructors and across quarters, semesters, or even years. They then want to use this information to determine necessary action. And last but not least, administrators want a modern quality management system, giving them an edge over their competitors.
Deans, Deans of Studies, or Department Managers
Department heads would naturally like to have an overview of the data collected for their department (such as comparative reference data for course subject and course type). They would like to retain subject-matter relevance, even when surveys are conducted across departments. In addition, they would like survey results that allow them to see what action(s) may need to be taken to improve their courses.
Faculty members are interested in improving their teaching—they want prompt and specific feedback immediately after a course is over. However, they are also concerned that the survey instrument is fair and that it provides data security. Many instructors would like to be able include their own questions in the survey and would like to be able to make anonymous comparisons to their colleagues.
Students want to be taken seriously and to see that they are part of the change process. They are concerned that their honest and, at times, critical opinions might be used against them, so anonymity and security of responses are crucial. Like instructors, they, too, would like to see what has been done with their responses in a survey. And they would like to be spared unnecessary questions or surveys—otherwise they will stop providing course feedback.
Computer Centers/Institutional Research Groups
Computer centers and institutional research groups generally have little spare capacity and would like a procedure that makes as few demands on their resources as possible. In this case, a browser-based solution available institution-wide as a web service is the best solution. The head of the computer center is happy when there is as little administrative paperwork (e.g., setting up user accounts) as possible. They want to set up data security and data access security once, and then have them run automatically. Institutional research groups are happy because they data they need to analyze and recommend improvements is available in a single, easily-accessed location.
The administration needs a system that conforms to data security regulations as well as university evaluation policy and regulations. They may want some of their services—over and above teaching—to be evaluated as well.
From this discussion, we can see that a functional and effective evaluation system must include a variety of questionnaires and support a variety of needs. Further, we can see that comprehensive and continual evaluation requires a set of data to model the organization, including all of its departments, courses, and instructors. This structure forms an important foundation for creating data reference norms and Quality Measurement processes, as well as allowing for convenient re-use of previously developed materials. To guarantee as much flexibility as possible, consider making mobile, online, and paper surveys available for deployment. The responsibility for implementing surveys, their costs, and the access rights to the resulting data must be finely layered and adaptable to the needs and contexts in your organization.
How Can Scantron Help?
Scantron is a long-time leader in assessment solutions and services. We’ve served the education market from kindergarten to professional career development for more than 40 years. Our course evaluation solution, Class Climate, provides the best in course and instructor evaluations, and delivers broad-based survey functionality as well.
Class Climate provides world-class support for course evaluations. It’s built-in structure mimic department and course structure for most institutions—and can be customized to fit as necessary. Libraries make it easy to design and re-use questions year-over-year for longitudinal tracking. Core questionnaires that allow for instructor customization ensure that the needs of both administration and individual faculty or departments are met. Automated reminders and connectors to many common learning management systems make it easy to increase and track responses. Quality dashboards provide immediate, at-a-glance quality tracking and comparison.
Class Climate provides the greatest possible degree of automation for comprehensive course evaluation. The potential value of this solution to you can be seen in these critical areas:
- Digital document capture
“Scanning, not typing” means a drastic cost reduction in capturing data while minimizing errors. Within a few hours you can accurately analyze tens of thousands of survey questionnaires.
- Instant feedback for optimum self-reflection
Instant feedback means a new quality in evaluation for instructors. They can reflect on the feedback and make changes while they are still involved in the on-going course. Norm reference values and comparative profile lines motivate instructors to optimize their teaching.
- Batch processing
To systematically categorize hundreds or thousands of surveys, Class Climate imports existing data from course catalogs and faculty registers. The support provided by this metadata framework enables you to create reports, set up norm reference value data and make comparisons across subsets (programs of studies, departments, instructors, courses, etc.).
- Organizational overview and central process control
Class Climate provides a kind of “process control center” for the central evaluation coordinator. This perspective can give university executive officers, as well as department heads valuable insights. Optional QM views provide a high degree of openness. “Learning from the best” is supported, as is the identification of quality weaknesses.
Class Climate provides both decentral, instructor-oriented surveys using active survey accounts as well as central evaluations. Both procedures can be combined, yielding interesting possibilities such as giving university service organizations their own survey accounts (e.g. cafeteria, administration, library). And Class Climate can resolve the issue of ensuring relevance of centrally conducted course evaluations to a particular discipline. Faculty members are systematically involved and instructors with active accounts using the VividForms questionnaire designer can evaluate any aspect of the courses they are interested in—internet access is all it takes.
Class Climate provides specific advantages for everyone in the organization involved in evaluation:
- Students can expect that their judgments are analyzed and forwarded to their instructors quickly. Class Climate also takes security issues for students seriously. Class Climate promotes the dialog between teacher and student.
- Instructors spend a minimum amount of time and receive optimum feedback for self-reflection. Class Climate enables them to supplement the survey with their own individual questions. They receive reports that enable them to compare themselves with other reference groups.
- University executive officers establish an efficient service infrastructure and emphasize the service aspect of the central evaluation. University executives can also show that they take the needs, and anxieties, that often accompany an evaluation seriously.
- The Computer Center appreciates the web-based approach, which means that a client installation is not needed. As a result, the university-wide roll-out is smooth and trouble-free. Security and data protection aspects are high priorities for Class Climate. In addition, we offer hosting services for institutions who prefer to leave the IT work to us.
- Third parties: Student organizations, the administration, thesis students, research project members, PhD students, library, cafeteria—Class Climate is a web service that can be used anywhere in the organization where survey data needs to be gathered. Departments of social science especially appreciate this sophisticated mechanism for survey data capturing.
Class Climate users have been involved in its development from the very beginning. The requirements originated in practice and were decisive in developing the features that Class Climate now provides. Users of the system provide feedback on features, requests, and ideas for improvement are discussed intensively and have led to the development of standard software for course evaluation. This can also be seen in the detailed product documentation, training materials, and coherent support concept.
Scantron’s Professional Services team is here to help!
- Professional development enables you and your staff to navigate the software and interpret the results more effectively. Scantron Corporation has a fully staffed corporate development department through which we offer an array of product training and staff development classes and seminars for Class Climate. The team is dedicated to building capacity for change within organizations and institutions through a series of training and consulting activities.
- Get your first evaluation or survey project up and running faster with Scantron Implementation Services. We provide dedicated experts with extensive experience in course evaluation management to help you implement your system and begin processing your surveys quickly and efficiently for a more rapid return on investment. The Scantron implementation team works with you to create a manageable schedule customized to ensure your success in the evaluation process. Depending on your needs, consultations can be performed onsite or via the phone or web.
- Scantron offers a wide array of Professional Services to customize, enhance, and support your Class Climate capabilities. Save yourself time by letting Scantron do the work for you. We listen to your needs and then work with you to create the applications you require.
- Scantron’s world-class Print Services are available to print your custom forms and fulfill your large-volume needs with mailing and data processing services.
Whatever the evaluation or survey assistance you need, Scantron has the products, tools, services, and expertise to help you ensure that you have the right program for your students. Our web-based software, combined with our length and breadth of experience, help you get the most out of your evaluations and results. Contact us today and see how Scantron can meet you where you are and help you get to where you want to be.
Back to Article Series Home
© 2015 Scantron Corporation. All rights reserved.