Achieve Helps Close Skills Gaps Among Less Academically Prepared Students, Suggests Evidence from a 38 college, 2,000+ Student Study

New York, January 13, 2020 -- Macmillan Learning, a privately-held, family-owned education publishing company, published research today that demonstrated a positive relationship between use of Achieve, the company’s new digital learning platform, and final exam scores. Research from the ACT, College Board and others have found that a student’s performance on college entrance exams and highschool GPA can predict their future success in two- and four-year institutions. Using these predictors of post-secondary success, the findings from this efficacy study suggest that use of Achieve can help bridge the performance gap often seen among students entering college. That is, the more that less academically prepared students use Achieve, the closer they can come to meeting the performance of their more academically prepared peers.

Less academically prepared students (as determined by ACT or SAT scores and high school grade point average) who engaged in at least 80% of assigned activities in Achieve elevated their final exam grade nearly a full letter grade and closed the gap in their average performance and the performance of their more academically prepared peers by about half. The findings build on recently announced data presented at the Academic Evaluation Association (AEA) annual meeting in November demonstrating that students performed better on final exams when they used Achieve’s pre-class activities, regardless of their motivation or academic preparedness levels coming into the class. 

“The benefits of a post-secondary certificate or degree are well known, yet we still see students entering college less prepared to succeed. The instructors we partner with are always exploring new ways to fill the skills gaps of less academically prepared students while keeping their more prepared students challenged. While there’s no silver bullet, we believe in providing instructors with insights into the differential efficacy of our solutions, and are pleased that we can offer them evidence that Achieve will support their efforts to help all their students succeed,” said Kara McWilliams PhD., VP of Impact Research, Macmillan Learning, and the study’s Primary Investigator.

The company partnered with instructors at 38 two- and four-year institutions to research the efficacy of Achieve, evaluating 2,251 students during the spring 2019 semester. The results revealed today represent one finding from a multi-year efficacy study, with other results being published following data collection and peer-review. Macmillan Learning’s Achieve platform launches this month and will be widely available in Fall.

Key Findings

• Use of Achieve is positively related to student’s final exam scores, regardless of their level of academic preparedness coming into college p<.0001. For every ten percent increase in a student’s engagement in assigned activities, they can expect a 5.7 percentage point increase on their final exam score

• The magnitude of Achieve’s influence on final exam scores was greater for students less academically prepared to succeed r(657) = .58, p<.0001 but was significantly positive for their more academically prepared peers also r(961) = .42, p<.0001.

• Among students taking STEM courses, the magnitude of the relationship was greatest among less academically prepared females (r(205) = .57, p<.0001, but was also significant among their male counterparts r(237) = .41, p<.0001 and more academically prepared female r(416) = .33, p<.0001 and male r(281) = .39, p<.0001 STEM students.

• Students who complete at least 80% of their assigned activities in Achieve, earn nearly a letter grade higher, on average, than students who complete less than 80%.

• The gap in performance on final exams closes by about half between less and more academically prepared students, when students less prepared to succeed complete at least 80% of their assigned activities in Achieve. 

About The Research

This study investigated the differential efficacy of Achieve among students less and more academically prepared to succeed. Macmillan Learning collected students’ self-reported highschool GPA as well as ACT or SAT scores, if one or both had been taken. Using those scores, students were then categorized as less or more academically prepared, and outcomes were evaluated among each group. Achieve efficacy research began while the tool was in beta testing so that feedback from instructors and students as well as student outcomes could be used to evolve the product, and to provide instructors with a transparent body of timely evidence of effectiveness.

This study complied with the American Psychological Association ethical standards for research.  It was approved by a third-party Institutional Review Board (IRB) and then approved at individual institutions where required.  IRB approval enabled researchers to compare course results with detailed information about each student, including their academic background. It also enabled the collection of student records including final exam scores and course grades. Forty instructors elected to participate in the study and their students were offered the opportunity to opt-in, and 74% of students chose to participate. 

This research has been reviewed by Macmillan Learning’s Impact Research Advisory Council (IRAC), which is comprised of experts in educational technology, methodology, and psychometrics. 

The findings released today are part of a larger body of research conducted by Macmillan Learning’s Learning Science team that began in 2017 and helped guide development of the Achieve platform.  The team has been studying its efficacy overall, and among subgroups of students to evaluate whether the tool supports all students - like those more and less academically prepared to succeed, more and less motivated to succeed, first generation students, and those with competing demands like jobs and families, among other important cohorts. 

Academic preparedness was measured by both college readiness status as determined by performance on the ACT or SAT, and high school grade point average.

Find the full “Achieving Student Success” research report here.

About Achieve

Achieve is an evolutionary digital learning platform that includes learning materials with a comprehensive set of interconnected teaching and assessment tools. It offers the best features of each of Macmillan Learning’s digital solutions in one platform that is intuitive to use and is flexible for students and instructors. Achieve was developed using published foundations of learning science and in partnership with students and instructors with the goal of supporting students of all levels motivation and preparedness and helping to engage students in and out of class so that they have better outcomes. To that end, instructors facilitate can learn in the way that best suits their class, whether it’s traditional, online, blended, or fully “flipped” classrooms. 

About Macmillan Learning
Macmillan Learning is a privately-held, family owned company that improves lives through learning. By linking research to learning practice, we develop pioneering products and learning materials for students that are highly effective and drive improved outcomes. Our engaging content is developed in partnership with the world's best researchers, educators, administrators, and developers. To learn more, please visit macmillanlearning.com or see us on Facebook, Twitter, LinkedIn or join our Macmillan Community.