According to CB insights, the biggest chunk of last year’s 19.6 billion in total VC funding went to startups in the “business intelligence, analytics and performance management” space. There are more and more investments in data companies, companies are investing more in their data, and well, there’s just more of the stuff.
For some companies, this uptick in data volume has led to real breakthroughs.
Netflix has used many millions of hours of viewing data to better assess what people want to watch. This has led to both to better recommendations and to better original programming from the streaming giant.
This is the promise of big data, and when an insight like this is reached, it can have a serious impact how a company does business and its bottom line. For many companies though, the promise of big data is not being fulfilled.
"Every single company I've worked at and talked to has the same problem without a single exception so far—poor data quality."
Serious data challenges
A panel at the Social Shakeup called “Data is Real-time and Ubiquitous” looked at both the opportunities and the difficulties of data usage in large companies: People like Ned Kumar of Fedex and Linda Brunner of Siemens Healthcare discussed the role of data in their organizations, and the data challenges that they face in using it to improve how their businesses function.
These sentiments are reflected in a recent report on the state of data use in enterprises, issued by IDG Connect. They found that the biggest hurdles facing companies in terms of data were poor data quality and excessive data. The most significant group of challenges was around the difficulty of extracting insights from data: for many, identifying trends proved difficult, and a lot data was not perceived as actionable.
This makes some sense: While huge quantities of data from a range of sources can yield new understanding, reaching the point where they can make those numbers useful is a big leap for a lot of companies. Many are still in earlier stages, creating processes for collecting, organizing and storing data from across different sources. Ensuring that that data is accurate, and normalizing it so that it will be comparable across sources is a major project, even for a business without a giant, multinational footprint.
Let’s say we’re just talking about audience size and engagement data: how many people are your ads touching? How many are your social posts being seen by? How many site visitors do you have, and how long are people watching your videos for?
If your end goal for those communications is extremely specific, say, driving purchases of a single item, perfect alignment between different datasets might not be as critical as just being able to track who interacted with your content and then purchased the product.
If, however, you are trying to impact a higher-level metric, like brand awareness, being able to collect, organize, understand, and compare all that data is necessary both for measuring the overall impact of your campaigns and for understanding the relative effectiveness of different channels.
Data, generally speaking, is hard: Ruslan Belkin, VP of Engineering at Salesforce and DJ Patil, Chief Data Scientist of the United States recently spoke about data products at the First Round CTO Summit. Belkin said: “Every single company I’ve worked at and talked to has the same problem without a single exception so far—poor data quality.”
It’s also possible to be too focused on data to the exclusion of anything else: Tesco, the 5th largest retailer in the world, has been consistently focused on leveraging data over the last decade. They’re often cited as an example of data leadership and yet they recently reached an 11-year low in market value. According to an article in the Harvard Business Review ”in less than a decade, the driver and determinant of Tesco’s success (its data capabilities) has devolved into an analytic albatross.”
So how do you do data better?
Smart performance measurement
Using data better is often a question of determining exactly what you’re trying to do with it.
According to the IDG report, to create real value from data, companies need to ”determine what information is most relevant to the strategic direction of their business.”
If we look at performance data—that is, measuring how well your efforts are working—there needs to be a clear purpose for tracking every metric you set out to measure. The data you collect and analyze, and any data and results that get presented within teams and across an organization, should be determined by your overall business goals.
"you should have as little data as possible"
Of course, few will say that they’re tracking data just for the sake of it—most often performance metrics are tied back to business goals in some way. But with the volume of data out there, unless it’s clear in the minds of more or less everyone on a team how a metric fits into the larger business strategy, it risks becoming a point of confusion rather than a driver of better results.
Detail can add clarity, but it can also obscure. At that Social Shakeup panel, William Flanagan of Audenti said “you should have as little data as possible.” In terms of performance data, you could translate that to mean that extra detail should only be included when it adds insight and doesn’t distract from the big picture.
In an ideal world, key performance metrics, tied to specific business goals, should be determined on a company-wide level. Reporting across teams should be centered around how efforts impact these key metrics. Individual teams may have KPIs that are specific to them, but they should tie into the larger picture as much as possible.
In addition to harmonizing activities across a business, aligning on core metrics also tamps down the tendency to focus solely on the data that makes a department or a team look good, rather than the ones that matter most.
Better data campaigns
Beyond performance measurement, data is driving more initiatives, in marketing and elsewhere. As with metrics, some people are pointing to a mismatch between the expectations and the results of these efforts.
IDG’s report suggested that gathering more data is not necessarily the best way to become data-driven. It states that “in order to drive data transformation, organisations might be best placed focusing more on their long-term data strategy and the delivery of business impact, as opposed to data sources and trying to drive insight from large datasets.”
It’s probably best to start small: DJ Patil, the chief data scientist of the US said: “If you try to build crazy ambitious things like machine learning, it’s going to fail on you. Get the pipelines and other stuff correct, then build on top of that.”
Aligning data initiatives company-wide and simplifying data infrastructure as much as possible can help drive data innovation, rather than stifle it. And, it can’t hurt to remember this line from DJ Patil: “when it comes to data products, clever beats smart nine times out of ten.”