Click to learn more about author Matthew Gierc.
Self-service BI. Ad hoc analysis. Exploratory data analysis. This is the holy grail that companies are looking for, right? At least that’s what the BI dashboard vendors are telling us.
But do these tools really help organizations? In the short-term, yes. But the long-term effects of repeated Ad hoc analysis might do more harm than good.
In many situations, Ad hoc analysis is conducted in response to some event. Perhaps your company needs to understand why a department performed below expectation for the quarter. Perhaps you need to see how sales results were affected by a competitor’s price change. Answering these types of questions is important, but they share one common trait that limits their usefulness – the fact that we’re looking for an explanation of past events. And while past performance can be indicative of future results, this isn’t always the case.
Other issues can plague an organization’s excessive dependence on Ad hoc analysis. For starters, this type of analysis is often performed with an anticipated cause in mind. Confirmation bias can often lead analysts to search for evidence to back up their claim. On the other extreme, an overdependence on Ad Hoc Analysis might encourage analysts to come up with any hypothesis, thus risking a series of blind guesses.
Despite these shortcomings, there are benefits to Ad hoc analysis and self-serve tools:
- Ad hoc analysis is agile, and can be performed at any time
- Reports tend to be highly accessible when developed using Cloud-based tools
- Users can interact with reports
Where Ad hoc analysis really causes issues is when these reports get shared beyond their intended audience. Often, the report that you prepare as part of exploratory data analysis gets widely distributed, especially when it’s beautifully formatted. Exploratory analysis is almost always inappropriate for broad distribution.
Exploratory data analysis and ad hoc analysis are incredibly powerful tools, particularly as the starting point for building a Business Intelligence Strategy that includes Data Governance, automated reporting and ultimately Predictive Analytics. The journey towards becoming data-led implies that a company needs to be forward-looking, not mired in explaining past events.