The 12 unmissable big data stories of the past year

2014 was an eventful year for those of us watching for the impact that big data is having on society.

While 2013 brought an explosion of awareness around big data, 2014 showed the market for big data services beginning to mature. We also saw how “datafication” is starting to impact the lives of more and more people around the world.

The year started with a bang in January when market researchers at IDG revealed that 70% of businesses had either launched a big data strategy, or were planning one for the near future. The BBC also reported that big data was being put to use to fight bullying by a 13-year-old schoolboy from Virginia, US.

January also marked the point where mobile internet use overtook desk-bound access for the first time.

In February, the UK Government announced that £73 million was to be spent on 55 projects designed to boost big data business, which is anticipated to boost the country’s economy by £215 billion and create 58,000 new jobs by 2017. One of the first initiatives is to be the country’s first Urban Big Data Research Centre, based at the University of Glasgow.

March brought the announcement of the formation of the world’s biggest corporate big data partnership as IBM, GE, AT&T, Intel and Cisco joined forces to launch the Industrial Internet Consortium – pledging to work together to provide “improved integration of the physical and digital worlds.”

In April, pharmaceutical giant Merck revealed how it had used its data lake and Hadoop-based technology to optimize vaccine production processes, allowing for more efficient manufacturing and ultimately lower cost-per-unit medicines.

IBM also announced that it was to open a Big Data and Analytics Centre of Competence in Athens, Greece.

In May, real time analysis specialists ExtraHop, based in Seattle, US, announced that they had raised $41 million in venture funding to develop its cloud-based on-the-fly analytics services, capable of analyzing 20Gb of data per second (or 400,000 transactions). Businesses including bet365.com and Alaskan Airways rely on the company’s systems to give them insights into customer behaviour as it happens.

June saw the start of the 2014 Fifa World Cup. Winners Germany credited their use of SAP’s Match Insights technology in giving them a competitive edge. One insight prompted the winning team to cut down average possession time from 3.4 seconds to 1.1 seconds – evident in the style of play during the game in which they beat hosts Brazil 7 – 1.

In July, Google rolled out Pigeon – its local search algorithm designed to help users find information, products or services based on their real-world location. It works on the principle that businesses in close proximity to the user will be more relevant, ranking sites based on this criteria. It uses data submitted by businesses as well as aggregated from hundreds of directory-style services such as Yelp and FourSquare.

August saw social blogging and sharing service Tumblr announce that it has partnered with image recognition specialists Ditto to create an innovative (and some might say invasive) service which scans uploaded pictures for branded content. Information about when, where and by whom their products are being consumed will be sold back to businesses such as Coca Cola and Kraft. It is even optimized to analyze the faces of people in the images to detect whether they appear to be happy or sad. This month also saw the ALS Ice Bucket Challenge reach its peak, with 2.4 million tagged videos  uploaded to social media, leading to the charity receiving $100 million in donations this month alone (compared to $20 million during the whole of the previous year).

In September the Chinese Academy of Sciences announced that it had developed facial recognition software reliable enough to work as a payment authorization system. Working with researchers who have built what could be the world’s biggest “face database”, containing 50 million images, the technology, already used at border controls, will be rolled out to commerce in 2015.

October saw New Scientist report on Chicago’s civic leaders’ attempts to harness big data to solve its “urban ills”. The city council unveiled plans which included attempting to highlight which children were most at risk of lead poisoning from the buildings they live in, and analyzes garbage collection data to determine where rodent infestations are likely to break out.

US mid term elections took place in November and were of course more heavily analyzed than any previous general election. Ahead of the election, pundits predicted that victory would go to the party which made best use of analytics during their campaigning. Big data and new technology also played its part in the fight against the Ebola outbreak, with mobile phone tracking used to draw maps of population movement, and spikes in calls to international helplines used to predict “hot zones”.

In December, Sony cancelled the planned release of their upcoming film The Interview when thieves made off with huge amounts of data – around 100 terabytes, it is claimed – after hacking their internal systems. The issue threatened to become an international incident after US intelligence officials announced that their primary suspects were hackers working for the government of North Korea. Private staff emails as well as unreleased films were uploaded to the internet by a group calling themselves the Guardians of Peace.

I can’t wait for all the big data stories of the next 12 months, but until then here are my predictions of what 2015 will bring for Big Data.

This article is published in collaboration with SmartData Collective. Publication does not imply endorsement of views by the World Economic Forum.

To keep up with Forum:Agenda subscribe to our weekly newsletter.

Author: Bernard Marr is a globally recognized big data and analytics expert.

Image: An internet cable is seen at a server room in this picture illustration taken in Warsaw January 24, 2012. REUTERS.