Can digital courseware improve education?

As the Postsecondary Success strategy enters its fifth year of personalized learning investments, it is a good time to take stock of what has been learned and to draw implications for future investments. A new report by SRI International (SRI), commissioned by the Bill & Melinda Gates Foundation, provides lessons learned from the foundation’s investments in digital courseware over the last five years.

Based on the investments reviewed for this report, it’s clear the foundation strives to produce insights that can shape future investments and uses of technology to enhance student success and higher education affordability. Although there was an overall positive impact of foundation-funded courseware projects on course completion rates, the challenge of reliably achieving positive outcomes at scale remains a major issue. The Carnegie Foundation for the Advancement of Teaching’s Pathways Project was the only project that had a positive effect of consequential magnitude. The Pathways project provides insights into what it takes to achieve consistently positive outcomes, but further work is needed to find ways to do so at lower cost to enable faster scaling.

It is important to keep in mind that this review reflects a window in time. Technology advances rapidly, and product features and approaches that are commonplace today were either just entering or even unheard of in 2009 when the first of the foundation’s grants were awarded.

Drawing on the lessons learned and knowledge gaps identified through this report, SRI has identified 10 recommendations for future courseware investments.

• A strong rationale remains for investing in high-quality courseware for introductory courses; however, courseware should be designed with scaling in mind, providing enough flexibility to support adaptations for different institutional settings.
• Courseware design and adaptation should be informed by an understanding of the institutions where the courseware will be implemented, including their goal, student characteristics, and organizational constraints.
• The maturity of instructional courseware and its prior evidence of effectiveness should be considered when making investments.
• More emphasis on third party, independent evaluations of impact should be applied to courseware investments.
• Philanthropy can play an important role in promoting iterative design cycles and standards for measuring the effectiveness of innovative instructional approaches incorporating digital learning.
• Funders should take a phased approach to supporting courseware innovations, with later stages of funding dependent on demonstrated capacity to collect data that can inform improvement.
• Funding decisions and evaluation activities should be tightly coupled.
• Funders should consider alternative strategies for increasing the supply and visibility of effective courseware, such as funding and other incentives linked to successful R&D outcomes (e.g. prize competitions and payment for success).
• Funders might want to consider R&D grant programs that identify and test courseware design and implementation principles explicitly.
• A portion of the investment in evaluating courseware and related technology tools should be devoted to examining longer-term impacts on degree completion.

The foundation’s emphasis on improving college success for underrepresented minorities, low-income students, and first-generation college goers was reflected in the context for which their grantees designed and implemented instructional software.

Read the Lessons from Five Years of Funding Digital Courseware Report executive summary here.

This article is published in collaboration with The Bill & Melinda Gates Foundation. Publication does not imply endorsement of views by the World Economic Forum.

To keep up with the Agenda subscribe to our weekly newsletter.

Author: Barbara Means, Ph.D., is director of the Center for Technology in Learning at SRI International. Vanessa Peters, Ph.D., is an education researcher at SRI International’s Center for Technology in Learning. 

Image: Students take notes from their iPads at the Steve Jobs school in Sneek. REUTERS/Michael Kooren.