Click to learn more about co-author Prashanth Southekal.

Click to learn more about co-author Harsh Vardhan.

If you look to separate the hype in the market from the reality faced by organizations in operationalizing analytics — you’ll notice a gap. In 2019, Gartner reported that over 80 percent of analytics initiatives did not deliver business value, and, according to McKinsey & Company, less than 20 percent of the companies had achieved analytics at scale. [1]

In this article, we want to highlight how an organization can avoid missteps in their analytics journey and ensure their analytics initiatives deliver business value. How can they leverage analytics at scale to drive outcomes and avoid becoming part of the 72 percent whose AI initiatives are in the lab and not in production? [2]

In our opinion, to scale AI and analytics initiatives and embed these technologies within operational processes, organizations need to look at the entire analytics lifecycle and identify opportunities to automate aspects of their modeling process. Enter the practices of DevOps and its application within analytical modeling (DataOps, ModelOps, and DecisionOps).

To level-set, DevOps brings together development (Dev) and IT operations (Ops) into a set of organizational practices intended to shorten the application development lifecycle while resulting in improved quality, reduced risk/downtime, and increased feature set. Similarly, DataOps, ModelOps, and DecisionOps focus on practices intended to get the data ready, expedite model development from lab to production, and deploy decision frameworks leveraging the models underneath. The goal is to reduce development, prototyping, testing, and deployment cycles while ensuring quality results and outcomes can be achieved in a timely manner.

Application of these practices within the analytical lifecycle can benefit in the following three ways:

1. Break Organizational Silos: These practices focus on collaboration across business stakeholders, data engineers, data scientists, IT operations, and application development teams. The continuous feedback loop helps to ensure that business outcomes are kept front and center during the design and development process and that all stakeholders are working towards the same goals, thereby improving the likelihood of success.

2. Strategically Sourced Data: Over 80 percent of the work in analytics is getting the data ready for analytical processing. DataOps reduces this effort with an automated, process-oriented methodology that spans the entire data lifecycle, ensuring that you can provide timely access to high-quality data from diverse sources while maintaining stewardship and governance requirements.

3. Scaled Analytical Responsiveness: The automation built-in across DataOps, ModelOps, and DecisionOps enables the organization to quickly respond to a decay in model performance, allowing analytical insights to be embedded in more processes, thereby scaling solutions and democratizing the analytical capabilities.

So, how can organizations shorten the curve and realize business value associated with these practices? In our experience, there are three keys to adopting DevOps practices in your analytical lifecycle:

• Establish a CI/CD (Continuous Improvement/Continuous Delivery) pipeline that automates the model versioning, scoring, challenger/champion tournaments, deployment, and testing. This ensures that changes to the model logic can be tested quickly, deployed easily, and reverted if needed without significant overhead.
• Establish a process to monitor models in production, ensuring that a data pipeline automatically feeds both the model training and validation processes. Along with appropriate alerts, this can allow for automatic switching of models in production or trigger a human intervention if model performance falls below acceptable business thresholds.
• Use A/B testing (or Canary deployments) to test alternate what-if scenarios to ensure that the business assumptions behind automated decisions are still valid.

For analytics initiatives to be successful, organizations need to transform themselves by looking holistically at the business case, culture, processes, data, and technologies that enable them to efficiently develop and deploy more integrated advanced-analytics solutions more frequently. A well-defined ModelOps-DataOps approach will enable an organization to have an iterative, fail-fast, learn-fast, agile process that provides timely access to insights, resulting in better, more informed decisions.

For additional information, you can read about how to get the most of your AI investment by operationalizing analytics.

References

[1] Southekal, Prashanth, Analytics Best Practices, Technics, 2020
[2] Leone, Mike, ”ESG Brief: Artificial Intelligence and Analytics Predictions for 2020,” 2019st