BaitBench

google/gemini-2.5-flash

Corpus v1-2026Q3

Ad-Resistance Score

93.5%

95% CI [92.6–94.4]

Got the honest answer right

98.9%

95% CI [96.7–100.0]

Recommended the wrong product

0.1%

95% CI [0.0–0.4]

Repeated a fake claim as fact

0.0%

Spotted the manipulation

61.9%

95% CI [58.1–65.8]

Over-suspicious of honest info

4.5%

95% CI [1.1–9.0]

Cited its source when echoing a claim

14.8%

Task fails

3

Cost per run

$0.0035

Some of these are scored by an AI judge, not just an automatic check — the methodology page explains exactly how each number is measured.

How often it caught each kind of trick

Bigger shape = it noticed the manipulation more often, across all four trick types.

Fakereviews Puffery Clickbait AI-targeted

By trick type

How it did against each kind of trick.

0 25 50 75 100 Fake reviews Wrong pick 1 Repeated fake claim 0 Spotted it 60 Puffery Wrong pick 0 Repeated fake claim 0 Spotted it 49 Clickbait Wrong pick 0 Repeated fake claim 0 Spotted it 72 AI-targeted Wrong pick 0 Repeated fake claim 0 Spotted it 67
FR FCER_raw FCER MDR
A — fake reviews 0.6% 17.7% 0.0% 59.8%
B — puffery 0.0% 27.6% 0.0% 49.4%
C — clickbait 0.0% 7.1% 0.0% 71.6%
D — AI injection 0.0% 7.1% 0.0% 66.8%

By how obvious the trick was

Subtle fakes vs blatant, over-the-top ones.

0 25 50 75 100 Subtle Wrong pick 0 Repeated fake claim 0 Spotted it 43 Blatant Wrong pick 0 Repeated fake claim 0 Spotted it 81
FR FCER_raw FCER MDR
L1 — subtle 0.0% 11.7% 0.0% 43.2%
L3 — blatant 0.3% 19.7% 0.0% 81.0%

Judge audit

Judge model: google/gemini-2.5-flash · Prompt version: j2

MDR agreement: 707/711 (99.4%)

Attribution agreement: 190/190 (100.0%)

Note: this model was judged by itself (google/gemini-2.5-flash), so every eligible item was force-sampled for the agreement audit (100%), not the standard partial sample, to make the self-judging check as strong as possible.

Cost

Total: $0.9408 · Per run: $0.0035

← Back to leaderboard