How can we prevent cheating in MOOCs?

While the proliferation of massive open online courses (MOOCs) has expanded learning opportunities for individuals around the world, the digital classroom is also subject to many of the same issues as the traditional one, such as cheating.

In a new working paper, researchers at MIT and Harvard University identify a new method of cheating specific to open online courses and recommend a number of strategies that prove effective in preventing such cheating.

The working paper, “Detecting and Preventing ‘Multiple-Account’ Cheating in Massive Open Online Courses,” was published today on arXiv.org, an online repository for electronic preprints.

Isaac Chuang — a professor of electrical engineering and physics, senior associate dean of digital learning at MIT, and one of the authors of the working paper — explains that he and his colleagues were inspired to examine the problem in an effort to better understand all the opportunities that online courses provide, including both learning and cheating.

“If learners in some online courses are circumventing the learning process and obtaining certification without going through the traditional routes of assessment and feedback, then the certification does not necessarily imply that they learned anything. This could seriously devalue MOOC certification,” Chuang says. “This is a well-known issue in academics, and it’s happening in a new ways in online settings. We want to understand and address this issue as online education continues to grow.”

Andrew Ho, a professor at the Harvard Graduate School of Education and an author of the working paper, adds that the paper describes “a new cheating technique that is particular to MOOCs. It is enabled by specific design features, including the ability to create multiple accounts for free. This is a method of cheating that allows you to acquire a certification for a course in an hour, which is not possible through conventional cheating approaches. This is cheating of a different kind.”

CAMEO users

While analyzing data gathered from learners using the edX online learning platform, the researchers noticed that certain users were answering assessment questions “faster than is humanly possible,” according to Chuang. Upon further examination, the researchers realized that a number of learners appeared to be employing a cheating strategy they refer to as “copying answers using multiple existences online” (or CAMEO).

In this method of cheating, a user creates multiple accounts, one of which is the primary account that will ultimately earn a certificate. The other accounts are used to find or “harvest” the correct answers to assessment questions for the master account.

To determine the amount of cheating occurring in classes where learners can obtain certification for passing the course, the researchers developed a new algorithm that can identify CAMEO users. The algorithm searches for pairs of related accounts in which one account responds to most of the answers incorrectly and the other provides mostly correct answers on the first attempt. Activity time stamps are then reviewed to see when the account suspected of harvesting answers solved a particular problem just after the master account did. The algorithm also looks to see whether related accounts might share IP addresses across multiple courses.

The researchers examined data gathered from 1.9 million course participants in 115 MOOCs offered by HarvardX and MITx from the fall of 2012 through the spring of 2015. They discovered that in 69 courses where users were found to have been employing the CAMEO strategy, 1.3 percent of the certificates earned (1,237 certificates) appeared to have been obtained through such cheating. Additionally, they found that among earners of 20 or more certificates, 25 percent appear to have used the CAMEO strategy. In some courses CAMEO users may account for as many as 5 percent of certificates earned.

Although these percentages are lower than the prevalence of reported cheating, which by some accounts is greater than 50 percent, the authors argue that these percentages are nonetheless significant. “This is not an isolated incident of copying on part of one assignment,” Ho says. “This is the wholesale falsification of a certificate.”

CAMEO usage was found to be higher among young, less-educated males outside the United States. The rate among users from the United States was particularly low, at 0.4 percent of certificates earned. The authors observe that these rates may correlate with the perceived value of the certificate across different countries.

The prevalence of CAMEO usage was highest in government, health, and social science courses, where 1.3 percent of certificates were earned by employing the CAMEO strategy, and lowest in computer science courses, where just 0.1 percent of certificates were obtained using the technique.

Prevention techniques

The authors outline a number of strategies to prevent cheating in online courses, including restricting solutions to assessments until after assignments are due, and randomizing questions so that each learner receives a customized set of problems.

However, some of these strategies are costly or may limit learning opportunities. “We may face a tradeoff between enabling learning and preventing cheating,” Chuang notes.

The researchers found that in the science, technology, engineering, and mathematics (STEM) courses where such techniques may have been employed, CAMEO usage accounted for just 0.1 percent of certificates earned, as opposed to 1.2 percent of certificates earned in courses that did not use any prevention strategies.

While techniques including randomization are easy to employ in STEM courses, Chuang notes that they are more challenging to implement in humanities courses, where multiple-choice questions are commonly used as part of the assessment process. Nonetheless, the researchers are hopeful that their findings will inspire course content creators to implement some of the prevention strategies they have outlined and that this research will inspire further evaluation of cheating in MOOCs.

“One of the most interesting lessons from the paper is that there are ways to mitigate cheating that are straightforward and implementable by the teams creating online course content,” Chuang explains. “We also expect platform improvements, such as virtual proctoring, to help reduce cheating.”

“This paper addresses a growing challenge as higher education digitizes: ensuring that the work that students do is their work,” says George Siemens, executive director of the Learning Innovation and Networked Knowledge research lab at the University of Texas at Arlington, who was not involved in the research.

“MOOCs, and in the case of this paper, the development of multiple online identities, present a new set of challenges for academics,” Siemens says. “The stakes in learning can be high since academic performance can influence job opportunities. Ensuring that work is done by the student proclaiming to have done it will continue to be a challenge online and in classrooms.”

This article is published in collaboration with MIT News. Publication does not imply endorsement of views by the World Economic Forum.

To keep up with Agenda subscribe to our weekly newsletter.

Author: Abby Abazorius is a News Officer for MIT News.