Big Data is Finally Coming to Education Here’s What We’ve Learned So Far

Big data has been the story of the past year. Nearly every industry sector has been hit by the urge to quantify, measure, and analyze, and they are using that information to improve their products and services, which in turn is improving our lives on both the micro and macro scales. (Can you even remember what shopping was like before personalized product recommendations?)

For a variety of reasons, education has been slow to the party. There have been difficulties surrounding what data to collect, how to collect those data, and what they might mean.As well, there have been concerns about privacy, especially as it involves students in elementary and secondary schools. But, big data is finally coming to education, thanks in part to educational technology companies like Knewton, which uses analytics to personalize educational experiences for individual learners, and to massive open online courses (MOOCs), which are free courses available from many of the top schools in the world. MOOCs have been able to supply huge amounts of data because they have attracted a huge following: more than 10 million students globally have enrolled in the courses.

The question, of course, has been what to do with all of the data. Over the past few months, results of various studies have been released, from MOOCs and other sources, providing insights into how people learn. Some of the findings reinforce things we already knew, while others are quite surprising and are already transforming approaches to education both online and in the classroom.

Here are some of the main things we’ve learned so far.

Long lectures don’t work.

The hour-long lecture that has been the staple of higher education for decades has taken a beating from big data. Research from MOOC provider edX has identified the ideal length of a video lecture to be between 6 and 9 minutes, after which learner engagement falls off sharply. This finding is having major implications in both college and corporate classrooms.

The best predictor of future course behavior is past course behavior.

In 2010, President Obama set a goal of 60% college completion in the United States by 2020. To meet this goal, many colleges and universities now offer courses and even full degree programs online. However, while online learning offers many advantages over traditional formats, one problem that has arisen is that students are more likely to drop out of online courses. One study of more than 50,000 students in community colleges found that the completion rate for online courses was 8% lower than for traditional courses. Data from MOOCs suggest that one way to boost completion rates is to increase engagement early in the course. The researchers found that students who were more engaged at the beginning of a course were more likely to finish, regardless of their stated intentions or motivations.

Even in online courses, offline support is essential.

The rise of online education has generated something of a heated debate over how effectively students can learn from a computer. Proponents generally focus on the convenience and cost savings associated with online learning, while opponents argue that an online learning environment can never replace a face-to-face one. Studies of the effectiveness of online education have shown mostly positive, but still mixed, results. Data from MOOCs suggests that the difference may have to do with the level of offline support online learners receive: after analyzing 230 million interactions from a single course, researchers from MIT and Harvard found that the single most important predictor of student success was whether students worked offline with someone else. It didn’t matter if that “someone else” was an expert in the subject or just another student in the class—the important thing was getting help offline.

Not all big data comes from massive courses. In education, big data can even be derived from analyzing everything from institutional variables to the learning activities of individual students. The potential here includes helping teachers identify when students are having problems, personalizing lessons so that each student gets the information he or she needs and is ready for, and tracking learners’ journeys through courses in order to better guide those journeys.

Even in individual schools and classrooms, the results are significant. The implementation of a data analytics system at Arizona State University resulted in a 10% increase in passing rates and a 50% decrease in dropout rates. In Delaware, using big data analytics to personalize education for individual students has led to 10% increases in both reading and math proficiency.

Big data is still very new to education, and there remains a lot to be revealed about how people learn, which will lead to improved education at all levels and in all formats. But there is no doubt that education is in the middle of a major transformation, with technology and big data leading the way.