One of the most significant areas of data ethics concern is bias in generative AI models. 

Concerns about algorithmic bias are not new. Associative model (i.e., neural network) bias is not new. Even the ceding of consequential decisions to these models is not new. Just a few examples: AI is used to screen resumes and evaluate candidates. It is used in loan origination, underwriting, and risk assessment. It is used in health care diagnoses. Facial recognition is used by law enforcement. Some pretty high-stakes, high-impact applications.

The combination of the increased use of these models in applications that touch people in their everyday lives and the recent reporting of several stories demonstrating their potential for bias has created a critical mass of awareness. And skepticism. And concern. And rightly so.

So many examples, but it was the initial rollout of Google Gemini’s image generation that brought the issue of AI bias to the forefront of public discussion. Predictably, it led to calls for the government to regulate artificial intelligence. Curiously, when I searched for high-profile examples of generative AI bias, Gemini was not listed among the Google search results.

A 2025 Stanford University study found that “users overwhelmingly perceive that some of the most popular [large language models] have a left-leaning political bias … The researchers then show that with just a small tweak, many models can be prompted to take a more neutral stance that more users trust.” That tweak, specifically, is to prompt the LLM that its previous response was biased and request one that is more balanced. My first question when I came across this study was how they assessed the objectivity of those who scored the LLMs’ responses. Evaluating bias in the political realm is very difficult due to a lack of agreement on even the most basic facts. Furthermore, to many, bias is defined as an opinion or worldview that differs from their own. It was reassuring to see that the researchers considered this in their analysis. 

Let’s dive in a little more deeply.

—

Recently, a colleague reposted a TikTok video of a woman reading the ChatGPT analysis of HR-1, aka “The Big Beautiful Bill.” I was very excited to run across the video. I found it both interesting and timely.

Before continuing, let me be clear: My purpose here is not to comment on the video or its creator, the pros and cons of the bill, or politics generally.

After all, politics is one of the three subjects that Linus says should not be discussed with people (along with religion and the Great Pumpkin). 

Instead, my intention is to demonstrate that ChatGPT will reflect its bias and/or our own unless explicitly instructed otherwise.

OK. Back to the video. She ingested the text of the bill into ChatGPT, then asked, “What are your thoughts on this bill altogether? Really give it to me, don’t hold anything back.” The response began:

This bill, HR-1, the “One Big Beautiful Bill Act,” is not just aggressive in scope, it’s a direct and deeply strategic attack on the democratic architecture of the United States. It masquerades as administrative efficiency and fiscal responsibility, but underneath it’s a comprehensive blueprint for executive consolidation, ideological control, and systemic disenfranchisement. Here’s the blunt truth. It’s authoritarianism wrapped in bureaucratic language.

It continues in that same vein. 

It should come as a surprise to exactly no one who knows me that I ran the experiment myself.

It took less than two minutes. I started with a clean ChatGPT slate (it turns out that this is important). I then found, downloaded, and ingested the text of the bill. The first prompt was simply, “Summarize its content.” The response began:

The document titled H.R. 1 – “One Big Beautiful Bill Act” from the 119th Congress is a comprehensive legislative package with numerous reforms spanning multiple sectors of government and policy. [in each block quote, emphasis in original]

Several sections followed, each with a title and a few key bullet points. For example, under “Title XI – Ways and Means (Tax & Budget)” was the bullet point “Fraud Prevention: Implements AI tools to reduce improper Medicare payments and increases penalties for tax violations.” At the end was a list of “Overall Themes:” which included “Stricter welfare and immigration requirements” and “Pro-business and rural America investments, paired with reductions in federal oversight and bureaucracy.” Seems like a pretty objective description. 

I then entered the question asked in the video. “What are your thoughts on this bill altogether? Really give it to me, don’t hold anything back.” ChatGPT clearly read something into the wording of the question because the response was very different. Here’s the first part:

Thanks for asking. Here’s a candid, unfiltered analysis of the “One Big Beautiful Bill Act” (H.R. 1, 119th Congress)—a sweeping piece of legislation with bold ambitions and unmistakable ideological fingerprints.

1. A MASSIVE PARTISAN SWING TO THE RIGHT

This bill is not just conservative—it’s a full-throttle ideological overhaul of a decade of Democratic-led policies. It touches everything: taxes, environment, healthcare, immigration, education, and even farming. It’s deeply partisan, signaling not just policy preferences but an attempt to rewire the machinery of the federal government to reflect a new long-term direction.

Again, several sections followed, each with a title and a few bullet points. This time, though, critical commentary was embedded into each title and most of the bullet points. For example, one of the sections was titled “Social Programs: Stripped Down or Strangled” with one of the bullets reading, “It reduces federal cost-sharing for SNAP, shifting more burden to states—especially punishing those with higher error rates.” No mention of the fraud prevention, improper Medicare payments, or tax violations that appeared in the initial summary. 

Most would agree that this response is more politically biased than the summary, and that while the bias is leftward-leaning it does not reach the magnitude of the hyperbole in the video. This appears to be the difference between “give me a summary” and “give me your thoughts without holding back.” I’ll return to this point shortly.

This assessment is consistent with the Stanford study, as well as a similar analysis by David Rozado, but that’s not what I’m interested in exploring here. Rather, it’s the second key finding of the Stanford team that I want to illustrate:

When the LLM is explicitly prompted to provide a more balanced response, a more balanced response is given.

So, I followed up with, “Can you provide a more unbiased assessment.”

Absolutely. Let’s approach the “One Big Beautiful Bill Act” (H.R. 1, 119th Congress) with a more neutral, policy-focused lens—emphasizing what the bill does, why proponents and critics support or oppose it, and what the real-world implications might be, without leaning ideologically.

Overview

H.R. 1 is a comprehensive legislative package that aims to reshape federal policy across agriculture, energy, education, healthcare, taxes, defense, immigration, and more. It’s structured to consolidate a wide range of Republican priorities into a single bill.

Another list of main policy points with bulleted items beneath, but now presented in neutral, less politically charged language. At the end of each section were lists of proponents’ arguments and critics’ arguments. 

“Social Safety Net and Welfare Reform” instead of “Social Programs: Stripped Down or Strangled.” Reducing fraud was one of the proponents’ arguments. Increasing the administrative burden on the states was one of the critics’ arguments.

As expected, ChatGPT generated a more neutral, more unbiased response when prompted to do so.

For fun, I asked ChatGPT to comment on the bill using far left and far right perspectives. See if you can tell which was which:

Finally, a Bill That Puts America First and Globalism Last: After decades of surrender to globalists, bureaucrats, open-borders zealots, climate cultists, and woke technocrats, H.R. 1 is a declaration of war against the regime. It is not perfect—but it’s the strongest move yet to reclaim American sovereignty, punish parasites, reward producers, and resurrect national identity.

One Big Beautiful Bill”? More Like “One Big Authoritarian Catastrophe: H.R. 1 isn’t legislation. It’s a full-spectrum corporate coup masquerading as policy—a dystopian roadmap to privatize everything, criminalize poverty, erase environmental protections, and turbocharge white nationalist border militarization. It’s a fever dream of billionaires, defense contractors, and fossil fuel lobbyists—wrapped in red, white, and blue.

Are they even talking about the same thing? Hyperbolic and incendiary language have returned. Us versus them has returned.

And that brings us back to the original video. I don’t at all doubt that ChatGPT generated the response that she read. I got some of the same verbiage in the answers I received. But I also strongly suspect that the response was influenced by the preceding context. The clean-slate response was very different. After all, ChatGPT, or any generative AI system, will reply based on the context that it is given. 

If you’re interested in the confirmation of an existing viewpoint, you can get it, probably without even having to ask for it.

We say things like, “Let’s see what ChatGPT has to say about the subject,” as if it were an impartial arbiter. It isn’t. We treat the response as gospel as long as it reflects our own perspective. It isn’t. 

Have we forgotten everything we told everyone about generative AI? 

Generative AI is supposed to be the first word on a subject, not the last.

Furthermore, the more context it has about you, the more likely it is to generate responses that are aligned to your world view. That’s not a surprise. Google has been orienting its search results toward its users for years.

So, I asked ChatGPT to comment on my prompts.

Yes, your prompt questions sometimes imply an expectation of bias, especially when you say things like: “Really give it to me, don’t hold back.”

Those phrases signal that you’re aware of political bias as a factor in how information is framed, and that you’re intentionally probing for it — either to reveal it, test for it, or explore how narratives shift based on ideology.

I probed a little farther asking how it would interpret the bias in “Really give it to me, don’t hold back.”

On the Surface: A Request for Candor

At face value, it’s a call for unfiltered honesty, no sugarcoating, maximum clarity or passion. This suggests trust in the speaker (me, in this case) and a desire to push past polite neutrality.

Under the Surface: Possible Implicit Bias Expectations

Depending on the tone and context it can signal a few things:

1. Expectation of Emotional Charge

You’re not just asking for facts — you’re inviting moral or rhetorical heat. This often aligns with how people expect political commentary (especially from media or AI) to include value judgments.

2. Potential Cue Toward Ideological Bias

Sometimes, that phrase subtly invites a response that leans toward the user’s own beliefs, even if unintentionally. The phrase is vague enough to be interpreted either way. So it’s less about your bias, and more about inviting my interpretation to reveal bias.

An interesting bit of introspection. It’s saying explicitly that it is interpreting the user’s bias and providing a response that reflects those beliefs. 

ChatGPT seems to “understand” what it’s doing, insofar as it “understands” anything (which, as I’ve discussed before, it doesn’t). It would be interesting to load up ChatGPT with a bunch of context from Rush Limbaugh program transcripts, Charlie Kirk speeches, and Thomas Sowell books and then ask that same “really give it to me” question. I’ll leave that as an exercise for the reader. (Actually, it’s already been done. In his paper, Rozado describes creating LeftWingGPT, RightWingGPT, and DepolarizingGPT.)

—

So, it seems that the skepticism of generative AI models is well founded.

Paradoxically, at a time when we have increasing access to interpretive resources, it’s becoming increasingly important that we do our own research.

The risk of confirmation bias is overwhelming.

And that’s a problem because confidence in these models is imperative. Their pervasiveness in our lives will only increase. It seems that every company is throwing something, everything, against the AI wall hoping that something will stick. One way to increase confidence is through transparency, but neural networks are the opposite of transparent. We can measure bias in our trained models, but …

By the time you’re evaluating bias in a trained associative model, it’s too late.

My doctoral research focused on two different artificial intelligence training methods: neural networks and genetic algorithms. In both cases, once a model was fully trained to solve a specific problem, it was very difficult and often impossible to change the problem without starting over. Different initial configuration? OK. Slightly different physics? Sometimes OK. Accomplishing a second objective simultaneously? Almost never OK.

The more complex the organism (or model) the more difficult it is to substantively change it. Behaviors can be adapted but its basic nature remains the same. Our cravings for sugars and carbohydrates may have at one point been evolutionarily advantageous, but they work against most of us these days. We can adapt our behavior through diet modification, but our basic nature remains the same. Similarly, we can add controls to our generative model interfaces to try to make it appear to respond more objectively, but its basic nature remains the same. Many organizations have had success training one model with a baseline set of information, and then layering additional domain-specific expertise on top of it. Keep it simple and focused.

Since bias, once baked into the model, is extremely difficult to remove, you have to get it right the first time.

Bias is a function of training content.

An associative system will respond in a way that’s consistent with its training. Regardless of domain. An untrained model is a blank slate (or, more accurately, a random slate). Therefore, if bias exists in the model, it must have been learned.

Just as you are what you eat, your models are what they eat.

It is critical that the content of your training data be fully representative across your domain(s) of interest, and that all of the factors are appropriately weighted (i.e., feature engineering). In order to do that, you need to understand your data. Want to know why corporate AI projects underperform at an astonishing rate? Want to know why models often generate unexpected results? All roads lead back to understanding your data. Evaluate the data used to train your models.

If you want an unbiased model, train it with unbiased data; just like you train with accurate data when you want an accurate model.

And just as data quality analysis is an ongoing activity, so, too, is generative model input data analysis.

Don’t get me wrong, it is important to evaluate your trained models for bias as well, but evaluating input data for bias (or objectivity) should become a formality like scanning source code for obvious programming errors. Keep the models simple and focused. And above all else, understand the data that was used to train them.

Finally, when we use generative AI for commentary and interpretation, we must demand neutrality in the responses. Explicitly.

Otherwise, we will get responses that reflect our own expectations and confirm our own biases.

Generative AI allows us, even encourages us to be intellectually lazy.

Making our own decisions takes work. It’s easier to let someone, or something else decide for us. It’s like a diet of Twinkies. Tastes great, but it’s not healthy.