Predicting College & Career Readiness by Analyzing Assessment Results
In today’s educational environment, it’s more important than ever to demonstrate that high school graduates are ready for the world they face. Educators must ensure that students graduate with the knowledge they need to enter postsecondary education or the modern workforce—and succeed.
Unfortunately, a recent study by Achieve, Inc. shows that instructors and employers do not feel that recent high school graduates are prepared for college or the workforce.
"Significant majorities of both college instructors and employers report that recent high school graduates arrive at college or the workplace with gaps in their preparation:
- 96% of instructors at two-year colleges reported at least some gaps in their students’ preparation (including 46% who reported large gaps in preparation).
- 88% of instructors at four-year colleges reported at least some gaps in their students’ preparation (including 34% who reported large gaps in preparation).
- 82% of employers reported at least some gaps in recent high school graduates’ preparation for typical jobs in their companies (including 48% who reported large gaps in preparation)."
The study goes on to discuss the need for setting high expectations for students. “It’s critical that schools clearly communicate the expectations of colleges and employers early in a student’s high school experience and help them to understand the coursework they will need to complete,” said Michael Cohen, president of Achieve.
How can schools help close the readiness gap? We believe the first step is identifying the specific areas where students are falling short by using a modern, robust analytics tool so you can dynamically explore the various factors that contribute to college and career readiness. Combine this data and ongoing analysis with your instructional adjustments and expertise to ensure your students are ready for their journey to college and career.
What is College and Career Readiness (CCR)?
“Simply put, ‘college and career readiness’ is the umbrella under which many education and workforce policies, programs, and initiatives thrive. From high-quality early education and strong, foundational standards in elementary school to rigorous career and technical education programs and college completion goals, college and career readiness is the unifying agenda across the P-20 education pipeline.”
With all of the discussion and focus—dare we say controversy—surrounding the Common Core State Standards and initiatives such as PARCC and SBAC, it’s easy to lose sight of what drives these programs: a sincere effort to improve student achievement so that students are ready for the next chapter in their lives. Yet we’re falling short of achieving that goal:
- Four out of 10 new college students take remedial courses.
- Employers consistently comment on high schools graduates not being ready to enter the workforce.
Educators need a set of articulated goals that define what college ready or career ready looks like. What knowledge must graduating students have to succeed in the world they’re entering?
Different organizations define this in different ways:
- PARCC defines CCR as the level of proficiency exhibited in the areas of English Language Arts (ELA) and Mathematics. Overall readiness is determined through a series of objective, academic performance assessments (PARCC does not assess non-academic factors such as persistence, motivation, time management, and so on). This consortium identifies five performance levels for ELA and Math: minimal command, partial command, moderate command, strong command, and distinguished command.
- The Educational Policy Improvement Center (EPIC) defines CCR as follows:
“College and career readiness refers to the content knowledge, skills, and habits that students must possess to be successful in postsecondary education or training that leads to a sustaining career. A student who is ready for college and career can qualify for and succeed in entry-level, credit-bearing college courses without the need for remedial or developmental coursework.”
EPIC goes on to specify prerequisite skills, including proficiency in reading a variety of texts, fluency in writing, quantitative literacy (mathematics), comprehension of the sciences, awareness of social systems, and more.
Various state and city departments of education are also starting to create their own definition of CCR:
- New York City Department of Education sets four key benchmarks:
- Common Core Learning Standards: Academic mastery that students demonstrate at every grade level as identified by city and state standards.
- Academic & Personal Behaviors: Set of learning habits and skills that support academic readiness. This includes non-cognitive, socio-emotional indicators, specifically: work habits and organizational skills, collaboration and communication skills, persistence, self-regulation, and social engagement.
- Academic Programming: Choices regarding the level of rigor and subjects that students complete, without requiring remedial academic courses after high school.
- College and Career Access: Learning about postsecondary pathways and careers to develop meaningful personal aspirations. This domain includes exploration; financial knowledge; effective use of summer months; and direct support for access, enrolment, and transition activities.
- Texas was the first state to integrate their own CCR standards in 2008. The Texas Education Agency worked with the Texas Higher Education Coordinating Board and the Texas Workforce Commission to boost college and career readiness. Standards are drawn from Texas Essential Knowledge and Skills (TEKS) standards in ELA, Math, Science, and Social Science and underlying Cross‑Disciplinary studies.
As you can see, most definitions identify similar factors for what it means to be college or career ready. These definitions are based on a strong foundation of knowledge and intellectual skills, including academic achievement. Most studies find significant commonalities between being college ready and career ready, so foundational measures of readiness will serve equally well for students heading either to college or to the modern workforce.
How Can We Measure College and Career Readiness?
Another area of fierce discussion surrounds how to measure student understanding of the core subjects necessary to succeed in college or a career. Assessments are a commonly used (some would say overused) measurement instrument, but do we all mean the same thing by “assessment”? While some educators and many parents hear “assessment” and think, “Oh, no, not another test!”, at Scantron, we believe that assessments should be thought of more broadly.
We believe any type of student measurement is a form of assessment. Homework assignments assess students. Classroom quizzes assess students. Teacher observations during class participation exercises and group or individual projects assess students. And finally, yes, state grade-level high‑stakes tests and college readiness exams assess students. In short, educators are already extensively assessing students every day.
The problem is the results from all these forms of assessment are stored in different systems: classroom gradebooks, student information systems, assessment solutions, even simple spreadsheets or printed reports. It’s hard to prove student understanding when it can take weeks to create a comprehensive report that brings together all these different forms of assessment.
Historically, there’s been no easy way to unify these disparate sources of data and make them readily available for analysis, prediction, and data-driven remediation. And the stakes for measuring student understanding (or lack thereof) are high.
Programs like NCLB, Race to the Top, School Improvement Fund, and others can inject much-needed investment capital into the education system. However, they require proof of student growth and achievement using commonly understood measures. Even more, they require being able to remedy any issues with student understanding before those issues create bigger problems in later courses. Educators need to be able to predict as early as possible whether students are on the right track to be ready for college or a career.
How Can We Predict College and Career Readiness?
The first step to predicting readiness is to define exactly what indicators your district uses to identify readiness. The College & Career Readiness & Success Center suggest the following easily measurable indicators: 
- Course performance
- Course completion
- Performance on summative assessments
- Behavior and conduct
Once you’ve identified your indicators, you can perform (or commission) a predictive validity study to compare, for example, performance on a benchmark or interim test to performance on a summative assessment (e.g., a high-stakes state test or college entrance exam). With that study in hand, you can then use the results of your benchmark assessment to identify students who are most in need of intervention, targeting them with instruction that is aimed at gaps in their current understanding.
You don’t have to wait until students are in high school. You can perform predictive validity studies on any grade level and incorporate the results into your ongoing analysis of student needs. Some assessment solutions, such as Performance Series® from Scantron, enable you to build the results of a predictive validity study into your reports.
Alternatively, an advanced analytics solution such as Scantron Analytics can show a prediction of student summative performance based on earlier benchmark scores, as shown below, with drill-down available to student-level detail.
How can Scantron help?
Scantron has an extensive track record providing assessment and analytics solutions and services to help thousands of customers measure and accelerate student growth.
Scantron’s Assessment Development and Psychometric Services group performs predictive validity studies using customers’ assessment data. Typically, these studies use results from Performance Series or Achievement Series® assessments to predict performance on the state’s summative tests.
For example, we worked with some Alabama districts to establish a strong correlation between Performance Series scores and ACT® Aspire™ scores. The correlation enabled Alabama educators to administer benchmark Performance Series assessments in Math and Reading and to use those scores to predict student performance on the end-of-year Aspire tests. As a result, educators had the confidence to use their Performance Series results for diagnostic purposes, identifying specific areas where students may need more instruction.
To help schools get the most out of their data, Scantron offers Scantron Analytics, powered by the world-class analytics engine QlikView®. Scantron Analytics can incorporate the results of a Scantron psychometric study into highly visual dashboards that help predict how students will perform on summative benchmark tests in addition to analyzing other data you are already gathering. By storing all the information in memory, Scantron Analytics delivers powerful analytics without the need for a separate data warehouse. Using information you’re already collecting sourced from a wide variety of educational systems, Scantron Analytics displays easy-to-read, graphical dashboards and data visualizations. Important trends and previously hidden connections jump out, so you can spend your time developing creative solutions instead of trying to make sense of rows and columns of numbers.
Scantron Analytics is fully interactive. Want to see details about a student cohort? Simply click a chart to drill down farther—all the way down to individual students. Aggregating and disaggregating data by demographics such as gender, income, program participation, or any other characteristic you track is just as easy. New discoveries and insights are only a click away as you explore your data to answer your questions, your way, in real time.
Predefined dashboards and data connectors make it easy to get up and running quickly. Need to access data that’s not predefined? Just ask and we’ll work with you to make sure your dashboards and data visualizations contain exactly what you want to see.
Whatever the assessment or analytics assistance you need, Scantron has the products, tools, services, and expertise to help you ensure that you have the right program for your students. Our software combined with our comprehensive suite of assessment services help you get the most out of your assessments and your data.
Back to Scantron Analytics
© 2015 Scantron Corporation. All rights reserved.