Those of you who have followed my writing for the last several years know that I have long been an advocate of combining human understanding of what the information means with the technical skills of data science. And, we see more and more examples of the value of the human side of Big Data, examples such as facial recognition going terribly wrong, as well as examples in baseball, where successful teams are combining human elements with data science. Thus, as progress is made in applying Big Data, we increasingly see the value of combining human input with the technical.
A recent Wall Street Journal article featured an excellent brief description of why human input is so essential for making sense of Big Data and for using artificial intelligence (AI) techniques such as machine learning. These kinds of techniques have been very successful in certain situations, such as in the highly publicized wins where computers beat masters at chess and at the game of Go. But, there are still major challenges in trying to get computers to do everything a human can. And, that Wall Street Journal article provided a quote that is an excellent brief description of AI’s challenges.
The quote appeared in the Wall Street Journal’s May 18, 2018 issue in the article “AI Can’t Reason Why“ by Judea Pearl and Dana Mackenzie, co-authors of “The Book of Why: The Science of Cause and Effect”. The quote, as stated in the article text, says that “today’s machine learning programs can’t tell whether a crowing rooster makes the sun rise, or the other way around.”
In my view, that quote is a really good, succinct description of the challenges involved with AI. Nonetheless, it is still said that eventually, as machine learning is based on more and more data, the computers, not a programmer or data scientist, will create the algorithms that put the data to use. But, getting computers to do that correctly can be a major challenge. And, the above quote succinctly points out why it can often be so difficult to get the machines to come up with appropriate algorithms on their own. Computers can know that two items, like rooster crowing and sunrise, commonly occur together. In other words, computers can identify that these two items are correlated. But, computers are not capable of explaining the relationship between the two. That’s why human input is still so important in today’s world of Big Data.