Why So Serious? It’s Just Data!
Data -> Information -> Knowledge -> Wisdom
Many people were up in arms about the continuing revelations from the NSA as well as the ongoing exposure and coverage of the substantial amount of data that’s being captured in all sorts of ways in today’s society. Laying politics and trust in governmental institutions aside for the rest of this piece … when we think about these massive sets of data, the one thing to remember is this: it’s only data.
It is data that needs to be transformed into at the very least information in order to make it begin to have real value. This is not a trivial task regardless of how easy it is to summon the paranoia of what “might” or “could” be possible. Then the level I’d put above information is knowledge, and the one above that might best be defined as wisdom. Machines can do a great deal of the work to translate data into information, but machines have a really hard time creating knowledge and an even harder challenge with imparting wisdom.
We can clearly crunch through billions of bits of information at an ever faster pace – but this is only useful if you have an analytic construct for what you’re trying to understand. It’s a lot harder and requires a bit more ‘heavy-lifting’ than many people seem to be aware of. If all one is attempting to do is speed up the volume of transaction processing that can be accomplished (see High Speed Trading as an example) the wizards can clearly make things happen at mind numbing speeds – however those are based on simple (or quite complex) rules with known actions to be taken.
In the world of seeking to understand what all this data MEANS .. and then use it to the benefit of business decision making and effective personalization of marketing in a ony-by-one world well things are not quite as simple. Clearly for news organizations operating on 30 second sound bites the actual effort behind the headlines is dull and boring – although many of us know it’s actually fascinating work requiring a fairly broad set of skills. Hence, we get the fear machines churning massive concerns about privacy and dire predictions about potential ills – I am not minimizing the reality that bad actors could wreak havoc on lives – in reality it’s mostly still just data. (By the way, these skill sets I mention are thought to be truly lacking in the workforce today creating a huge gap in jobs vs. people to fill them. Just in so-called “cloud” skills there are literally millions of open jobs )
Defining the Questions that Lead to Understanding
In the retail world, data has been collected for many decades yet it’s only a fairly recent phenomena that we are beginning to utilize these data streams, at scale, at the individual level for useful information (or even knowledge) driven commercial applications. Before I get those “cards and letters” YES, as an industry we have been extracting certain kinds of value from these data for a very long time. What I am talking about is the arena that our media punditry echo chambers are obsessed with – that is INDIVIDUALLY identified behaviors and their potential use in future personalization.
Part of the ways the world continues to change has to do with Moore’s law, as interpreted by Intel where processing speed continues to double every ~18 months, and the ever-decreasing costs of data storage and processing horsepower. But extraction of actual marketing value still comes down to figuring out the answers to these rather basic questions:
What are we trying to learn from this massive amount of data?
How can we leverage that data from a marketing standpoint?
Where we can we find the most value in the data?
There is no easy way to find nuggets of information amongst all of the data, let alone obtain actual knowledge. Yes, there are all sorts of amazing software tools being created to harness and manipulate and ‘learn’ from the data and commercial applications are coming on line too fast to keep up with. The potential value and usefulness of these new data applications and information creation takes real time to filter through the societal consciousness – as we as marketers make peoples lives easier, saving them time or ‘hassle’ but clever tools and products driven by data slowly but surely some of the fear will subside.
Don’t Be Frightened
There’s a constant arms race in terms of privacy and encryption. Here in the United States the reality is that we do live in an open society. Information is part of the commerce we enter into every day. Data is behind every interaction and transaction we make. Do we need protections, common sense laws and agreements among people, governments and business – absolutely. Will we get there tomorrow? Probably not …
But here is plea, I truly hope that we don’t let fear shut off these spigots of data, which could, in fact, truly make our lives better. The reality is that our technology advances outstirrping our human capacity to understand ramifications and avoid mistakes has always been present – today the rate of change has accelerated to such a degree that my biggest worry is more about forgetting the sage wisdom of …“just because we can, does not mean we should” – the data are the least of my worries.