Industry commentators love gushing about how Big Data changes everything, and how it is “worth its weight in gold”. But is all data created equal? Where exactly does data get its value from? 

Big Data spending is expected to hit $48.6 billion in 2019, so people clearly see potential. But you’ll find that most Big Data discussions center around the 3Vs – Volume, Velocity and Variety. Notice how none of these characteristics tells us why Big Data is a good thing. Here’s the story we’re told – first collect all the data you possibly can, then store it in a data lake. Throw in a dash of data munging, and it magically morphs into a treasure of tremendous profit. Hmmm, where else have we heard that before?

How Does Data Add Value?

Regardless of your organizational structure, your data adds value only when it improves your business operations, or provides your customers with a great product experience.

What does that look like in practice? For example, let’s say you have an online store:

• You cold start your personalized product recommendation engine for new users by scraping their interests from social media sites.
• You reduce false positives in your fraud detection system by mining hard to fake metrics about your users from their social graph.
• You use Instagram photo tags to find microtrends in customer demand. Combine that with your revenue management system to forecast demand at a granular level, and tame inventory inefficiencies.

But how do you know that you can do any of these things?

This is where exploratory data analysis comes in. Once you’ve framed the right questions, you can use a wide variety of tools to visualize and drill into relationships in your data. Tools such as:

• Apache Zeppelin with Spark.
• Splunk for server logs.
• pandas with matplotlib.
• ggplot2 and R.
• The much underappreciated workhorse, Microsoft Excel.

But what kind of patterns and insights should you look for? Simple. Ones that are actionable, or tell you something that you didn’t already know before.

Actionable Insights

Let’s say you’ve released a shiny new mobile app for your online store. What are some high impact metrics you could track? Many will obsess over off the shelf metrics like daily active users (DAU). Don’t do that. App analytics firm Amplitude does a great job explaining why DAU is nothing more than a vanity metric. Can you think of specific product improvements you can make looking at your DAU numbers? I can’t.

Instead, how about a metric that compares shopping cart abandonment rates between your mobile app and your website? That could clue you in on problems your customers are facing. If you see an uptick in mobile abandonment, perhaps those users were having a bad internet connection. Or maybe they did not have their credit card handy. Either way, you now know what to do next. You build a mobile notification feature that nudges your customers toward all the awesomeness still waiting for them to complete their purchase. Even if you recover a small fraction of abandoned carts, the impact on your bottom line could be huge.

Novel Insights

You already know a lot about your business – you’re the domain expert. Your data should tell you something new.

Can you estimate the power of a nuclear device by looking at a photograph? Impossible, you say. But British physicist Geoffrey Taylor did exactly that. He correctly estimated the explosive yield of the Trinity nuclear tests in 1947, before President Truman announced the results. All he used was a delightfully simple but profound technique from the world of physics called dimensional analysis. As Alex Schultz, VP of Growth at Facebook points out, if you can make useful estimates for something as complex as a nuclear detonation, you can certainly do the same for most business scenarios.

Experts might try to convince you otherwise – “You don’t know what you don’t know! You need to have a Big Data system in place first. Then you can tell how much it will benefit you.”

Nonsense.

You already know a lot about your business – you’re the domain expert! Use back of the envelope calculations to set the bar for how good your new data analytics system needs to be. If it isn’t meaningfully increasing your accuracy, or telling you something new, it is worthless.

Takeaway

You’ve seen how easy it is to get lost in all the Big Data hype – no wonder only 27% of C-level executives think that their companies are getting any value out of their data. Stay above the fray and make sure you intuitively understand how data insights will boost your business. Only then can you make an informed decision about what data and analytics systems you need.