Big Data has been highly hyped. Big Data was touted as an all knowing source, where N=all. This would mean that the number of pieces of information we have is so huge, that there is enough data to know just about everything. That sounds wonderful, but we don’t yet know all that.
Granted, Big Data knows a lot—enough to dismay those concerned about privacy. But, Big Data still doesn’t know all. So, we don’t yet have the ability for computer algorithms and data analytics to automatically spit out all the answers. And, to get the most from Big Data, the technical/statistical skills of data scientists need to be combined with judgment and subject matter expertise such as knowledge of the dynamics of the business, specialized expertise in the emotional areas of marketing, or specialized expertise in any area.
Rather than hyping Big Data, we’re finally seeing the media put more emphasis on its limitations. For example, the September 24, 2015 issue of Bloomberg Businessweek ran the article “How Much of Your Audience Is Fake” by Ben Elgin, Michael Riley, David Kocieniewski, and Joshua Brustein. It reports that, in today’s era where metrics about clicks are valued, many of the so called clicks included in web traffic counts are actually just automated bots, not humans who actually view the web page.
As the article points out, the expected superior performance of digital advertising’s more precise targeting failed to materialize. The article cites an example where a company’s return on investment for digital advertising was merely two to one, compared with six to one for TV. According to the article, digital’s weak performance is largely due to the fact that so much click traffic is automated bots, not humans.
Yet, the problem goes beyond bots, and also lies in the fact that algorithms, such as those used for targeting, cannot yet automatically provide all the answers. For some time now, I’ve been pointing out the important role of judgment and subject matter expertise in Big Data, especially in the earlier stages of applying the data. My previous posts on this include “Warning: Big Data Can be Biased and even ‘Troublingly Wrong’—Just Like Data Years Ago” and “Success with Big Data Requires More than Just Technical Skills”.
In fact, these kinds of problems with Big Data are getting more media attention. For example, the July/August 2015 issue of Marketing Insights, a publication of the American Marketing Association, contained several articles about the problems associated with Big Data and about the importance of paying attention to traditional marketing principles when making use of data. A quote from just one of those articles “Data by Device” by Susan Griffin, makes the point well: “Despite the presumption that we know so much about the consumer ‘segment of one’ and the plethora of data that we have about what individual consumers do, Big Data is not reliable or even revelatory of the emotional whys behind consumer behavior.” This article, as well as other articles in Marketing Insights’ July/August issue, advocate relying on the principles of traditional marketing in addition to using Big Data.
The July/August Marketing Insights’ cover story is an article about clickbait by freelance writer Marguerite Mc Neal. It discusses the problems associated with too much emphasis on click metrics and page views, but not enough on the marketing and the emotions that motivate people to buy. The publication also featured an article titled “Mining for Insights” by Susan Scarlet and Sarah Tarraf, which provides an excellent discussion of tapping expertise from traditional marketing and market research when using Big Data.
I, myself, am especially inclined to see value in combining the technical side of Big Data with subject matter expertise because of my background. In my early career, my role was very much like the work of what we call data scientists today. After seeing the limitations of data analytics, I concentrated on understanding the issues that drive business success, which are the “Winning Moves” that I focus on today.
So, in conclusion, it’s good to see increased recognition of Big Data’s limitations. Understanding those limitations and augmenting Big Data with judgment and subject matter expertise can lead to far better outcomes.
