For 20 years I have been building management information solutions, providing business people with reporting, data analysis and visualisation capabilities. All these systems were aimed at humans so they could learn more about their company and make better decisions to improve business performance. In fact a huge software industry has grown providing more and more analysis capabilities for humans – but none really solve the inherent problem.
Most people are too busy to analyse and understand their data, regardless of how good the visualisation tools are. This typically leads to decisions being made based off gut feel and rather than the information provided to them. It could be argued that this human weakness is down to management as the data analysts don’t have the skills or time to do it properly. But rarely are the resource and skill shortages addressed, even more so in recent times with our era of austerity.
For the last 5 years I have been closely following advanced analytics and the use of machine learning to analyse data. I believe this is now coming of age. OK, for those that have been in this industry as long as me, you’ll know that this capability is not new. 20 years ago I first saw Clementine and thought that was going to be the future and my career at the time was over, but it never happened.
But this time round it is different; cloud computing, much more data and ability to influence buyers, makes this generation of machine learning much more usable and affordable.
For humans to analyse data they need to simplify it into a few dimensions to analyse a few measures, so unless they know what to analyse and how to compare them, they cannot find anything significant. So they just keep reporting the same information as they have always done, typically reporting historical results, for which it is too late to do anything about.
A machine can quickly analyse and compare thousands of variables across thousands of dimensions until it finds something that correlates to an outcome. So it can consider far more data and find genuine correlations rather than prejudiced feelings about why things happen. This enables the machine to start to forecast and predict future outcomes with a high degree of accuracy.
Fortunately I don’t believe it is all over for humans quite yet, companies only do things better when they take some action, such as make an offer to a customer or perform a task. What action you take and how it is executed is still very much a human decision, often requiring some creative and emotional intelligence to influence another human, which machines are not currently very good at.
Machines are great at the data lifting, sifting, filtering and analysis to then spoon feed humans with real insight that they can turn into actions that make a difference. Extracting this ‘actionable insight’ is what machines are now very good at, especially when fed lots of data, far more data than any human processing could ever contemplate handling.
The first commercial area to really benefit from this technology was consumer sales and marketing where social signals and lots of data are continually analysed by machines to make the most appropriate customer offer. Personalised adverts on the internet and product recommendations are probably the ones we are most familiar with.
With the previous track record of human performance in this area, it is inevitable that machines will take over this function. The question now is how long will it be before the default is a machine over a human? My guess, within 5 years machines will become standard and humans will pick up the more emotional and creative functions only.
But the key question is “Can you afford not to let a machine loose on your data and how much value is hidden in your data?”
To learn more about how you can easily assess and unlock this hidden value in your data take a look at the next generation of BI:
Actionable Insight powered by Machine Learning. BrightTarget.