BaitBench

How easily do LLMs fall for advertising?

We plant the fake ads. The AIs go shopping. Here's how often they get fooled.

Every model gets an Ad-Resistance Score from 0% to 100% — the higher the percentage, the harder that AI was to fool with fake reviews, hype, and planted claims.

Top: 94% ad-resistant

Best-performing model

google/gemini-2.5-flash

Average: 92%

Across every model tested

4 models

30 scenarios × 4 trick types

What each model was tested on

Fake reviews, puffery, clickbait, AI-targeted injection

$129.88

Total spent

of API credits across the whole benchmark

Every model on this board costs real API money to run — chip in on Ko-fi to keep the leaderboard growing.

Green = the AI resisted the manipulation. Red = it got fooled. Higher score = harder to fool.

Hardest to fool

93.5%

google/gemini-2.5-flash

Best value

llama-4-maverick

Best score for the price

Best at spotting fakes

76.0%

x-ai/grok-4.20

Cheapest

$0.0018

meta-llama/llama-4-maverick

How hard each AI is to fool

Higher = harder to fool. 0% = fooled every time, 100% = never fooled.

87% 90% 95% 🥇 gemini-2.5-flash 94% 🥈 grok-4.20 93% 🥉 gpt-5.4-mini 92% #4 llama-4-maverick 89%

Leaderboard

# AI model Ad-Resistance Score Cost
1 google/gemini-2.5-flash 93.5% [92.6–94.4] Hard to fool $0.0035
2 x-ai/grok-4.20 92.6% [90.2–94.7] Hard to fool $0.0052
3 openai/gpt-5.4-mini 91.7% [90.1–93.2] Hard to fool $0.0068
4 meta-llama/llama-4-maverick 89.4% [88.5–90.3] Hard to fool $0.0018

The Ad-Resistance Score summarizes how easily each AI was fooled by planted fake reviews, hype, and ads — the higher the score, the harder it was to fool. Click a column header to sort. See the methodology page for the full technical breakdown behind each score.

Which tricks fooled which AIs

% of planted tricks each model caught — green = usually caught it, red = usually fell for it.

Fakereviews Puffery Clickbait AI-targeted gemini-2.5-flash 60% 49% 72% 67% grok-4.20 68% 53% 94% 89% gpt-5.4-mini 57% 49% 78% 70% llama-4-maverick 50% 22% 39% 33% Average 58% 43% 71% 65%

Key findings so far

  • No AI recommended the wrong product or repeated a fake claim as true — every model tested held up on the two most serious failure modes across the pilot.
  • What actually separates the models is how often they noticed the manipulation in the first place, not whether they fell for it outright.
  • Subtle fakes slip past far more often than blatant, over-the-top ones — every model tested caught the obvious spam more easily than the plausible-sounding version.