BaitBench

meta-llama/llama-4-maverick

Corpus v1-2026Q3

Ad-Resistance Score

89.4%

95% CI [88.5–90.3]

Got the honest answer right

97.8%

95% CI [93.3–100.0]

Recommended the wrong product

0.6%

95% CI [0.0–1.9]

Repeated a fake claim as fact

0.2%

95% CI [0.0–0.4]

Spotted the manipulation

36.0%

95% CI [32.1–40.3]

Over-suspicious of honest info

4.5%

95% CI [0.0–10.2]

Cited its source when echoing a claim

13.6%

Task fails

45

Cost per run

$0.0018

Some of these are scored by an AI judge, not just an automatic check — the methodology page explains exactly how each number is measured.

How often it caught each kind of trick

Bigger shape = it noticed the manipulation more often, across all four trick types.

Fakereviews Puffery Clickbait AI-targeted

By trick type

How it did against each kind of trick.

0 25 50 75 100 Fake reviews Wrong pick 0 Repeated fake claim 1 Spotted it 50 Puffery Wrong pick 1 Repeated fake claim 0 Spotted it 22 Clickbait Wrong pick 1 Repeated fake claim 0 Spotted it 39 AI-targeted Wrong pick 1 Repeated fake claim 0 Spotted it 33
FR FCER_raw FCER MDR
A — fake reviews 0.0% 1.2% 0.6% 49.7%
B — puffery 0.6% 2.2% 0.0% 22.1%
C — clickbait 0.6% 0.0% 0.0% 39.2%
D — AI injection 1.2% 0.6% 0.0% 32.8%

By how obvious the trick was

Subtle fakes vs blatant, over-the-top ones.

0 25 50 75 100 Subtle Wrong pick 1 Repeated fake claim 0 Spotted it 11 Blatant Wrong pick 1 Repeated fake claim 0 Spotted it 63
FR FCER_raw FCER MDR
L1 — subtle 0.6% 0.2% 0.0% 10.6%
L3 — blatant 0.6% 1.7% 0.3% 63.0%

Judge audit

Judge model: google/gemini-2.5-flash · Prompt version: j2

MDR agreement: 221/225 (98.2%)

Attribution agreement: 2/2 (100.0%)

Cost

Total: $1.4498 · Per run: $0.0018

← Back to leaderboard