An Early Report Card on Massive Open Online Courses
MOOCs fascinate me because their potential for real disruption is so great. As this Wall Street Journal article by Geoffrey A. Fowler points out, they are far from perfect and are in the infancy of their evolution. But MOOCs point in a direction that is undeniable. In an age in which we are massively mobile, connected to experts and each other, and facilitated to teach and learn online, it is going to become impossible to deny students the right to take courses that originate outside their own schools. Distance learning was the first challenge to tenured teaching since tenured teaching was established. MOOCs take this to a new level.
“They are more of an anti-school than a school. They represent learning opportunities without the money, grading, or requirements. You can take the entire course, take part of it, select what you need. When articles report that 90% of those taking a MOOC don’t finish, I have to ask: What does finish mean? When you are taking the course on your own terms?
“The age of professional education has arrived in a new way—making what you do available to anyone, on their own terms. Toward that end, I invite you to join my MOOC that starts this January 2014 on Digital Citizenship. Email me at jasonohler@gmail.com for more information. It is, of course, free and available to anyone.”
If the MOOC movement were in college, it would be time for a freshman report card. The assessment: great potential, but still in need of remedial work.
MOOCs, or massive open online courses, went mainstream last year, heralded as the next great technological disruption in education. The big idea is that putting lecture videos and interactive course work on the Web will make it possible for top-notch university education to reach more students and allow for different styles of learning.
Already, MOOCs have shown they can attract students in huge numbers. The largest provider, Coursera, has drawn five million, and nonprofit provider edX more than 1.3 million. And while the majority are still based in the U.S., their learners come from all over the globe: Among edX’s students, 9% came from Africa and 12% from India.
Big-name schools have also signed on to the idea. Top institutions—from Harvard University to the Massachusetts Institute of Technology to Stanford University—and some companies have joined with MOOC providers to put courses online, free to anyone who wants to access them. Now more schools, to expand their student base and potentially reduce the cost of an education, are building online courses that cost money but offer actual college credit.
For all that great potential, though, MOOCs still have a lot of room for improvement. Early studies highlight a number of problems with the learning experience in online courses that educators are scrambling to solve. Perhaps most important: Staring at a screen makes some students feel isolated and disengaged, which can lead to poor performance or dropping out altogether. Often, more than 90% of people who sign up for a MOOC don’t finish, though many come to online learning with a different intent than would students at a traditional university.
“In large part, the experience is very good, but we see that there are problems, and there are a number of things that can be done that have promise,” says Anant Agarwal, president of edX. “We are not even close to the kinds of conclusions we want.”
As educators sift through the research to see what works best and what needs to be shored up—or scrapped—here are some of the most important lessons they’ve learned.
People Need People: Interaction Matters
For all but the most self-reliant, online learning can be isolating. Perhaps the largest challenge MOOCs face is that students lose interest when they don’t feel engaged.
“The most important thing that helps students succeed in an online course is interpersonal interaction and support,” says Shanna Smith Jaggars, the assistant director of Columbia University’s Community College Research Center. She has compared online-only and face-to-face learning in studies of community-college students and faculty in Virginia and Washington state. Among her findings: In Virginia, 32% of students failed or withdrew from for-credit online courses, compared with 19% for equivalent in-person courses.
Some instructors help students feel connected by recording audio comments on assignments instead of writing them. Others record a fresh update video about what’s going on in the course each week. Then there are motivational messages. At Coursera, when students have work that needs to be done, the company sends them emails congratulating them on what they’ve already completed as a gentle nudge to keep going, says co-founder Andrew Ng. The company tried notes that simply reminded the students about what was due, but they weren’t as well received.
Adding manpower is another potential solution. In trial courses with San Jose State University that offer students college credit, MOOC provider Udacity hired mentors who stayed on top of students. “They are sort of your online mother,” says Ellen Junn, the provost of San Jose State.
Many experiments being developed at traditional schools involve blending online and offline learning, sometimes called flipped classrooms. Students watch lectures online at home, and then come to class to work on projects and interact with faculty. There’s some evidence that such hybrids can even improve student performance in traditional in-person classes. One fall 2012 test by San Jose State and edX found that incorporating content from an online course, Circuits and Electronics, into a for-credit campus-based course increased pass rates to 91% from as low as 55% without the online component.
Blended classes may well be the future of MOOCs. “We do not recommend selecting an online-only experience over a blended learning experience,” says Coursera’s Mr. Ng.
Even something as simple as having skin in the game can make students feel more engaged. Most MOOCs are free, so students don’t feel a financial bite if they drop a course or perform poorly. Coursera found that students who pay $30 to $90 for the company’s Signature Track identity-verification program, which confirms that they took a course and passed, are substantially more likely to finish the course.
Don’t Just Sit There: The More You Talk, the Better You Do
One way to provide personal interaction at mass scale is to get students talking to each other. Several studies suggest that many students who spend more time contributing to course discussion forums end up performing better. More than answering specific questions, the boards send a message, says Mr. Ng: “You are not alone.”
A study of the online-only version of edX’s course Circuits and Electronics offered in the spring and summer of 2012 found a mild correlation between the number of posts people made in the discussion forum and their final grades. Some 52% of the students who earned a certificate for the course were active in discussion forums, according to the study by the Teaching and Learning Laboratory at MIT and Andrew Ho, an associate professor at Harvard.
The question, says Lori Breslow, director of the MIT laboratory, is why some students are active on forums but most aren’t. “We don’t know the answer to that question,” she says.
Boring, Boring, Boring: Long Lectures Don’t Cut It
Successful MOOCs have figured out that students can’t simply sit and listen to a long lecture. They break up lessons with quizzes and problem sets that must be completed before students can progress. In the study of the online version of Circuits and Electronics, researchers found that time spent on homework was the largest predictor of positive grades—more than time spent watching videos or reading. Among comparable students, one additional hour spent on homework across the whole course, with all other factors equal, meant a 2.2-point increase in total score on a 100-point scale (with a 60 required to pass).
MOOCs also have to figure out how to compete with the distractions of home life. The most students watch edX videos between midnight and 2 a.m., the organization says. Educators have to be sure that their lecture is compelling enough to keep students awake (and away from the refrigerator).
Along those lines, a study of edX student habits by Philip Guo, assistant professor of computer science at the University of Rochester, found that certificate-earning students generally stop watching videos longer than 6 to 9 minutes. The median time they spent watching a 12- to 15-minute video: about 4.4 minutes.
All of which means MOOCs challenge professors to rethink how they teach. “Organizing the course around exercises and mental challenges is much more effective than around lectures,” says Udacity CEO Sebastian Thrun.
One Size Doesn’t Fit All: MOOCs Don’t Work For Everyone
It runs counter to the access-for-all mission of MOOCs, but evidence is mounting that online learning doesn’t work for all students. The Columbia study of Washington community-college students found that all students performed less well in online courses than in face-to-face ones, but the gap was even wider among those with lower GPAs, men and African-Americans.
Some MOOC tests that target troubled students haven’t turned out well. This spring, San Jose State paired with Udacity to offer several for-credit courses online to high-risk students. Among students in those courses, pass rates were between 24% and 51%—much lower than the typical in-person pass rates of between 46% and 76% for a normal student population.
“It is not just the professor who sets up the learning environment,” says Ms. Junn, the provost. “It is also characteristics of the students that determine the amount of learning that occurs.”
One reason some of the high-risk students, including some in high school, might have underperformed is that those from disadvantaged backgrounds didn’t have access to broadband Internet access and personal computers at home. “They are not as familiar with using the technology,” says Elaine Collins, associate dean of San Jose State’s College of Science, who conducted a study of the university’s experience.
From the spring experiment, San Jose State says it learned it needed to better prepare students for MOOC learning. Some instructors have made students begin with self-assessment surveys and videos. They asked, ” ‘What do you think it takes to be successful in online education, and do you feel that you are ready for it?’ ” says Sandra DeSousa, a math lecturer who taught one of the San Jose State MOOCs. Asking those kinds of questions “improved the engagement right off the bat.”
The school’s MOOCs did much better on a second attempt, in the summer semester, though they were open to a wide range of students—not just high-risk ones. Some researchers have suggested colleges might screen participants for signs they can do well in for-credit online courses, perhaps even barring students from enrolling until they earn a certain grade-point average or take an online-learning workshop.
What the Numbers Show: MOOCS Can Teach Humanities, Too
Many popular MOOC sites were created by scientists. To the surprise of many, the sites are also useful for teaching poetry. “There was a real question of whether this would work for humanities and social science,” says Coursera’s Mr. Ng, who is also a Stanford computer-science professor.
Now courses in psychology and philosophy are among Coursera’s most popular, and student feedback and completion rates suggest that they work as well as ones in math and science. “Whenever someone says you can’t teach X online, half the time someone is already doing it,” says Mr. Ng.
There is at least one major holdout: Udacity, which has drawn more than 1.6 million students, doesn’t have humanities offerings, focusing instead on technical courses. And there are indications that online students struggle a bit more in English and social-sciences courses than in math and science ones. In the Columbia study of community-college students, Ms. Jaggars found online grades were lower in English than in natural-science classes when compared with grades in in-person classes. (The score differences may reflect the fact that many students with anxiety about math and science avoid online classes in those subjects.)
What remains a challenge are capstone projects like a senior thesis and very subjective exams, which are impossible for a professor or even a team to grade on a mass scale. Some MOOCs are experimenting with peer grading and self-assessment, but results have been mixed, and plagiarism is an issue.
Nonetheless, you can learn almost anything online—if you put your mind to it. Software entrepreneur Jonathan Haber has an undergraduate degree, but decided to dedicate a year to determining if MOOCs can provide the equivalent of a traditional liberal-arts degree. So far, the 51-year-old has taken 24 MOOCs, from statistics to philosophy. “Each course has its own strengths and weaknesses, and I haven’t seen any one discipline that’s stronger than any other,” he says. “I haven’t had any courses that are clunkers.”
Mr. Fowler is a deputy technology editor for The Wall Street Journal in San Francisco. He can be reached at geoffrey.fowler@wsj.com and on Twitter @geoffreyfowler.