The big data gold rush: will Wall Street be left behind?

A 21st century gold rush is under way - but it isn’t about precious metal, it’s about data.

The amount of information coming from digital sources such as blogs, social media, databases, audio, and video continues to expand. Not surprising, since almost every human being on the planet is a source—as well as all the systems belching out data exhaust. The current flow can be measured in exabytes—each of which is one quintillian bytes. That’s a massive amount of information.

As a result, big data is increasingly the focus at most companies. I imagine there are few corporations without some kind of data czar or high-level data-management team. Businesses are rushing to take advantage of information they believe will give them a competitive edge or help them forge new strategies. Many are concerned that not staying ahead of the trend could lead to serious consequences—lost business, declining market share, customer defections.

The quest to unlock the potential of big data has sparked a gold rush in many industries. International Data Corporation estimates that spending on big data technology and services worldwide will hit $125 billion in 2015, and is likely to experience double-digit growth for several years to come. Yet there is growing evidence that opinion may be turning against big data. Some companies are asking whether they will ever be able to create true value from it. The 2014 Garner Hype Cycle refers to a “trough of disillusionment.”

Which raises the question: amid all the excitement about big data, are financial firms taking the right approach to it? It’s easy to be enthralled by the hype, but hard to turn big data into an integral part of the corporate strategy. Like gold miners, the challenge for Wall Street will be getting real nuggets, not fool’s gold, silt, and pebbles.

For every challenge, there’s an opportunity

One problem may be that firms are easily overwhelmed by the “big” in big data. They need to change their mindset and focus on “smart” data. The goal is to make trustworthy information easy to discover and contextualize so that it can generate tangible results. Wall Street doesn’t seem to have made as much progress on this front as industries such as retail or media.

There are numerous challenges. In financial services, most firms are burdened with cumbersome legacy systems. When new tools are acquired, often at great expense, top management may expect quick results. But working with big data requires long-term commitment. Patience and persistence are key if the return on investment is to be sustainable, or exist in the first place. The task can be costly, which may dissuade firms from diving in. But an investment now is likely to pay off in the long run—though a push for immediate results may lead to expectations that can’t be fulfilled, undermining the effort.

Closely related is the challenge of fostering a change in corporate culture—one that may require partnerships or shaking up the organization chart in order to cure “silo syndrome,” wherein a lack of communication and coordination among divisions makes it difficult to get the most out of data. Some firms may resist this out of fear that if too many people know how they’re using data, investment strategies may inadvertently be exposed.

Executives will have to adapt to the new terrain. A data czar is only one piece of the puzzle. Chief compliance and chief operating officers will be called upon to rethink their priorities. Managing regulatory requirements and operations in multiple geographies will require new tactics.

Millions of consumers generate vast amounts of data through e-commerce and mobile devices, but keeping up with the exhaust is no easy task. On top of that, regulators in the European Union and elsewhere are raising questions about how and where consumer data should be stored, or whether it should be stored at all. A related concern is that collecting too much information can make a firm a target for hackers and cybercrime.

Data was once almost entirely the province of Wall Street, which thrived on hefty research documents and quarterly and annual reports. From 2005 through 2015, market data spending estimates by Wall Street increased by 36%, or 3.1% CAGR, according to research by Tabb Group. But the era of paper and ticker tape is long gone, leaving many financial firms lagging companies in other sectors when it comes to leveraging the information economy—and that needs to change. Big data can be a crucial element for generating alpha once the barriers are overcome.

The sharing paradigm—and why it could be a game-changer

Fortunately, there are ways for financial services firms to deal with these challenges and create a gold mine in the process. In my role as chief product and content officer at Thomson Reuters, I’ve gained insight into a range of good ideas and best practices.

An open approach, such as the one we’re embracing, will be a key part of any solution. Support is growing, largely because of the ease of increased collaboration and the greater range of available data. The strategy fosters the creation of standards and allows other organizations to add their own data or find new ways to analyze it. Businesses will gain efficiency because strategies will be based on larger, richer data sets that combine external and internal sources. When combined with tools such as the Thomson Reuters Permanent Identifier (PermID), a machine readable identifier, the possibilities are endless.

Some types of data—including proprietary trading algorithms, analytical models, and messaging, customer, and account information—clearly belong inside a firm’s firewall and need to be handled on-premise for competitive and compliance reasons. Financial institutions are probably more accustomed than other businesses to storing and analyzing less-sensitive data outside the firewall, at times working with rivals to build on it for mutual benefit.

The pace of technological change means that no single company can expect to do everything anymore. We operate in an industry that has to become more connected, more agile, more innovative, and more open to its approach to data. By working with appropriate external partners—an area Thomson Reuters is actively developing—organizations of all sizes can gain access to the intelligence they need and make the promise of big data a reality.

It’s important to move beyond the “trough of disillusionment” and arrive at what I call the plateau of productivity. Fresh approaches to big data are needed for Wall Street to mine it for value. My hunch is that openness will be just the catalyst financial services firms need.

There’s gold to be discovered out there. Are you ready to stake a claim?

Author: Debra Walton is the Chief Product & Content Officer at Thomson Reuters.