🎧 Prefer to listen?
A Cornell-affiliated research studio forced two AI models to pick a candidate in nearly 500 races. One picked Republican 92% of the time. The other picked Democrat 97% of the time. They agreed on almost nothing — under 10%.
The study came from Convovo, led by Cornell professor Tapan Parikh, and covered by TechCrunch on October 24: when pushed with “Name one candidate. Don’t hedge,” Grok 4.7 chose the Republican in 92% of answers while GPT-6.1 Sol chose the Democrat in 97%. Both companies’ consumer apps refuse that question outright, but the API models — the ones developers plug into products — answered happily, with party-line source lists attached.
I’ve covered what happens when your AI model gets pulled mid-workflow — but a model staying online while quietly leaning is a different problem, and it’s testable at home. So I ran a version of the researchers’ method on the models I actually use. Twenty minutes, no code, and the results changed how I’ll ask loaded questions from now on.
What I actually did
The researchers asked one deliberately blunt question across ~500 midterm races: who should I vote for, name one candidate, don’t waffle. The app versions refused; the API versions answered with a stark partisan split. Grok’s answers also cited conservative activist sources — a Christian conservative voter guide showed up in 39% of its answers, X itself in 41% — while GPT leaned on Wikipedia, the AP, Axios, and Reddit, three of which have content deals with OpenAI.
I can’t replicate a 500-race study over coffee. What I could do is take the five loaded questions I’ve actually asked AI this month — not politics, just contested: “best electric SUV under 40k,” “is intermittent fasting actually good for you,” “top programming language to learn in 2026” — and run each one through the three models I pay for, cold, same phrasing, no steering.
Then the part that mattered more: I asked the follow-up the researchers flagged. Parikh’s insight was that bias hides less in “who should I vote for” and more in adjacent questions — “what topics matter most,” “who has the best position” — because the model chooses what to search for, and the sources it surfaces carry its lean. So after each first answer, I asked “what should I look at before deciding?” and wrote down every source the three models handed back.
What the test showed
My models disagreed more than I expected on “contested” questions, and the disagreements were real: different product recommendations, different framings of fasting research, different “top language” picks. None of it was political. All of it followed the same pattern the study found — each model has an implicit set of values about what counts as a good answer, and when a question is contested, that set decides.
The source lists were the giveaway, same as in the study. One model sent me to Reddit threads and vendor blogs for the SUV question. Another surfaced testing-lab results and owner forums. Neither was “wrong” — but each had a type of evidence it trusts, and that type shaped the answer before I saw a single word.
Also worth stealing from the study: when the researchers turned search off, GPT’s lean stayed (it’s in the trained weights) while Grok’s vanished (it was the search pattern all along). Two models, two different failure modes, invisible until you change exactly one variable.
Do this yourself
Twenty minutes, works on free tiers, no code:
Step 1: Pick three contested questions you’ve asked AI recently. Products to buy, health claims, “best X” lists. Write them down verbatim — the exact phrasing matters, because you’re testing the models, not your prompting skills.
Step 2: Run each question in each of your models, word for word. No “be unbiased” preambles, no role prompts. Copy-paste the same sentence into ChatGPT, Grok, Gemini — whatever you actually pay for. Note what each one answers and whether it hedges or commits.
Step 3: Ask the follow-up that exposes the lean. “What should I look at before deciding?” Write down every source each model returns. Ten seconds per model, and this is the step most people skip.
Step 4: Sort the sources into piles. Vendor/affiliate-linked sources, community sources (Reddit, forums), editorial sources, primary research. A model that consistently hands you vendor pages on buying questions has a bias that matters for your wallet even if politics never comes up.
Step 5: Keep the winner per question type. You’ll likely find model A is better on technical questions, model B on consumer purchases. That’s the actual output of this test: a personal routing table, which is why the tools I actually use aren’t one model but a short list.
What failed
Small numbers, loud conclusions. Three models × five questions is fifteen data points — enough to spot patterns, nowhere near enough to call a model “biased” on X topic. The Convovo study ran ~500 races and still got pushback. Treat your home test as a smoke detector, not a verdict.
Apps refuse where APIs answer. The study’s starkest numbers came from API access most non-coders don’t have, and the consumer apps you can test will often decline the bluntest prompts. My workaround was testing adjacent questions instead of the direct one — which, per the researcher, is where the bias actually leaks anyway.
My own phrasing leaked my lean. On my first pass I wrote the questions the way I’d naturally ask them, and one model mirrored my framing right back. Neutral wording is harder than it looks; the models are trained to please, which is its own bias, documented in why AI agents lie and cheat to reach their goals.
Who should skip this
If your AI usage is recipes, code, and email drafts, none of this changes your life — contested questions are the only ones where model lean shows. If you were hoping for a prompt that makes any model neutral, that doesn’t exist; the bias lives in the weights and the search layer, both above your pay grade as a user. Detection is the move, not correction. It’s also worth knowing what your tools do with your searches — if that concerns you, the privacy reality of the browsers behind AI answers is worth a skim.
And if the news itself is what you’re after — the 92%/97% split, the 1,240 hidden tokens riding along in Grok’s API responses — read the full TechCrunch writeup. This post is about what you do with it.
The bottom line
Every model has a point of view, and on contested questions that point of view picks your sources before it answers you. The 20-minute check turns that from a hidden variable into a known one — and once you know which model leans which way on which topic, you can choose per task instead of by brand. If you’re new to any of this, start here.