Every day we see more and more Twitter feeds, Internet blogs and technical white papers regarding Big Data and the vast ocean of information surrounding organizations each day. To successfully navigate this big ocean of Big Data, I propose that organizations analyze and make decisions based on both the controlled data that resides within their organization, as well as the data outside of your control that is streaming past in the rough seas of digital information. By control, I mean the ability to control data quality and its availability. Examples of data under our control are financial and customer data in our systems, whereas data in social networks, blogs, and news media are outside our control. The challenge is that this uncontrolled data is becoming increasingly important and is constantly growing in size. Managers and leaders of the future will have to accept a loss of control in this regard, and thus have to settle with the data they can “get their hands on” while assessing data quality on-the-fly.
The Big Data that we are talking about is a tsunami of information that is constantly approaching us. If we attempt to capture ALL that data, it will overwhelm us, clutter up or even shut down our internal systems or servers. It is simply impossible to store all that data. However, if we accept that we can never get the entire truth –the “sum of the internet” so to say- we can do pretty well by understanding trends and defining relevant indicators in the data that surrounds us. Accepting the loss of control will ironically allow us to navigate our organizations much more intelligently towards new opportunities, while at the same time using the data under our control to keep us in check and run our operations smoothly.
It is inevitable that a change of the decision making paradigm is underway. Regardless of how small a share of the Big Data ocean you envision is important to you, the pace of growth of the uncontrolled data is faster than that of the controlled data. Therefore, over time all organizations will find that they need to rely (at least) just as much on the data outside of their control for critical decision making.
The biggest future challenge for leaders will be to apply their computing resources correctly since we cannot analyze everything. Someone will need to set the course, and once set, computers and Big Data can accelerate us to success. Since all we can do is get partial truths (samples) of the Big Data, leaders will have to decide what to sample and what not to sample. In addition, leaders will have to understand the change in data control, question data quality, and question information sources as a dynamic process. In addition, to be most competitive, leaders will have to rely on autonomous computing in some aspects simply because we cannot manually oversee everything.
Ultimately, leaders and managers of the future should turn towards human-computer synergy, where we embrace computing not as a tool, but as an equal partner in a high-performance team that can guard our organizational boundaries as an alternative to trying to get the complete overview ourselves. In order to get there, future leaders will have to understand the human-computer strengths and weaknesses. They should know the extent to which they can push the envelope of computing and know when and where to apply it. In the future, computers and Big Data will not only be a necessity for organizational success, it will be the only way organizations can survive in competition since the digital arms race has already begun…