Generative AI (GenAI) has undoubtedly taken the spotlight as this year’s defining innovation. Models like ChatGPT and LLama can generate text and code, perform exploratory data analysis, and automate documentation, which introduces countless opportunities for data science efficiencies. 

However, tech work is often a compromise between implementation efficiency and performance. In some cases, GenAI can unnecessarily complicate workflows – like using a cannon to kill a fly. 

While GenAI solutions are effective for tasks involving unstructured data, many projects are better served by simpler, off-the-shelf data techniques. A growing number of IT leaders echo this sentiment, and many are reassessing their use of GenAI, narrowing it to areas where it delivers the most value. 

So instead of using GenAI for the sake of it, take a step back and evaluate where it truly adds value and whether a simpler approach would be more effective. 

Four Considerations to Guide Smarter GenAI Implementations

GenAI is capable of handling a wide range of tasks, including many traditional data science applications. But while it excels at working with text and images, GenAI can require significant time and resource investments to achieve results comparable to conventional data science techniques on structured data. 

With that in mind, consider these four factors when determining whether GenAI will truly benefit a project:

1. Correctness: GenAI can deliver impressive outcomes for certain tasks, but the reliability of large language models (LLMs) varies. If a project requires the processing of structured data and consistent, precise outputs – such as predicting equipment failure in manufacturing – traditional data science techniques like regression modeling may offer more accurate results. When accuracy is critical, it’s important to evaluate GenAI’s performance against long-established methods before making a decision.

2. Transparency: The lack of transparency in GenAI models makes it difficult to understand the reasoning behind outputs – a major drawback for tasks that demand accountability and auditability. For example, building a credit scoring model requires visibility into decision-making to ensure fairness and regulatory compliance. The need for transparency becomes even greater in applications like medical diagnostics. Traditional data techniques like random forest, support vector machines (SVMs), decision trees, or linear regression are better suited for these endeavors. 

3. Control: Conventional data science techniques offer clear parameters to control algorithmic behavior. For example, increasing the threshold reduces the number of outliers, while adding more trees usually improves the reliability of the response. In contrast, GenAI tools offer limited parameters to control outputs, like temperature or prompt adjustments. Prompt engineering lacks transparency because even a small change to a prompt can lead to unpredictable and unexplainable shifts in output. With control closely tied to transparency, projects that require an explanation of algorithm implementation are often better served by traditional methods that provide more direct control. 

4. Cost: Implementing applications that rely on GenAI can be costly, whether it’s due to licensing costs for advanced models or developing custom solutions in-house. It also takes time and resources to prepare data, craft prompts, and post-process results. Additionally, Gartner predicts that organizations will abandon 30% of GenAI projects after proof of concept by the end of 2025 due to factors like high costs and unclear business value. Therefore, if a simpler, less resource-intensive approach can achieve the same outcomes, it’s probably the smarter choice.

How do these considerations play out in real-world scenarios? Take, for example, the task of categorizing customer feedback for a retailer. 

Thanks to GenAI’s ability to understand and process natural language, it excels at categorizing content into topics or sentiments. In this use case, transparency isn’t a major concern, slight variations in correctness are acceptable, and one can manage outputs through relatively simple prompt adjustments. The time and resources saved by automating categorization ultimately outweigh any risk.

On the other hand, consider tasks like retail customer segmentation or fraud detection. GenAI struggles in these use cases, and slight prompt adjustments can lead to unexplainable changes in results. While it’s technically possible to refine GenAI to match the accuracy and reliability of traditional techniques, the time and resource investment can quickly outweigh the simplicity and efficiency of using off-the-shelf methods. 

In this scenario, the retailer is better off leaning on standard, reliable, and transparent classification and outlier detection techniques.

GenAI Is a Strategic Tool, Not a Default Solution

GenAI offers a valid approach for many data science use cases due to its ability to generate text, images, code, and more. But it’s far from the universal answer to every task. In many cases, traditional data analysis techniques are more efficient and just as effective as AI.

While GenAI is undoubtedly an exciting tool that will continue to shape the future of data science, it’s crucial to understand its limitations – and know when to lean on simpler and more transparent solutions.