Our fact-checker got 9x cheaper and 4x faster. We didn't change anything.
Big If True, our free AI fact-checker, used to need 22 minutes and $12 in compute cost to check an article. Now it takes six minutes and $1.33. Here’s our secret: we did nothing. Get it at verso.ink/big-if-true.
Three weeks ago, we released Big If True, a free plugin that turns Claude into a careful fact-checker. Give it an article, and it finds every factual claim, checks each one against sources, and hands you a report with the evidence. Since then, newsrooms, freelancers and startups around the world have started using it — and it’s working. Big If True is catching real errors.
So we did what builders do: we tried to make it better.
We spent weeks, hundreds of dollars and about 150 test runs on clever ideas. None of them beat the original. Meanwhile, the original kept getting better on its own.
Much cheaper, much faster
We regularly test Big If True on the same test dataset of articles with known mistakes. Here’s what one check cost and how long it took on average, five weeks apart:
9x cheaper
Cost per check
4x faster
Time per check
In early September we ran checks on Claude Opus 5, one of Anthropic’s most powerful models at the time. Today we run them on Claude Sonnet 5.5, its faster, cheaper sibling. Both catch about nine in ten of the errors we know are there.
But today’s check costs only a ninth as much and finishes in a quarter of the time. It also raises fewer false alarms: Opus 5 wrongly flagged a true claim about once every three checks. In our latest round on the same articles, that didn’t happen once.
A note on those dollar figures: they’re what the computing would cost at Anthropic’s list prices. If you use Big If True in Claude with your subscription, you don’t pay anything per check. It just means each check uses less of your plan.
None of the improvement came from us. Anthropic’s newer models are simply cheaper and just as sharp, and the tools around them got cheaper too. Even on the exact same model, a check got twice as cheap in the last two weeks of our testing, because Claude now uses a much cheaper helper model (Haiku 5.5) to read the web pages it opens.
We tried to help. It didn’t work.
We tried to improve the skill too. Here’s what we tested:
- A team of fact-checkers. We split each article into batches and gave each batch its own fact-checking agent, all working at once. It wasn’t faster, and it cost three times as much. With fewer claims on its plate, each checker simply dug deeper on every one, without better results.
- Opening every source. About a third of the time, the skill trusted a search result instead of opening and reading the page behind it. To us, that looked like cutting corners, so we made it read everything. Then we checked: the search-based verdicts were just as accurate. Search is good enough.
- A cheaper model doing the legwork. Running the checkers on Anthropic’s smallest model, Haiku, saved money, but it raised seven false alarms where the plain version raised one.
Each was a reasonable idea. Each lost to the simple original version.
The bitter lesson, in miniature
AI researchers have a name for this. In 2019, the computer scientist Rich Sutton called it the bitter lesson: over 70 years of AI research, clever hand-built tricks kept losing to simple methods that use more computing power.
Two years earlier, Dario Amodei, then a researcher at OpenAI and now Anthropic’s CEO, had made the same bet, that bigger models would beat clever tricks, in an internal memo called “Big Blob of Compute”, which Kevin Roose just published for the first time. Our story is a tiny version of theirs: everything we bolted on to help the model was made obsolete by the model getting better.
Of course, none of this makes the skill pointless. Without it, Claude only checks about a third as many claims. The skill’s job is to define the task: what counts as a claim, what counts as evidence, when the honest answer is “couldn’t verify”. How to search, how many pages to open, how to split up the work: that’s the model’s job, and it keeps getting better at it. So that’s our approach now. We pick the right task, describe it well, and get out of the way.
The lesson is simple: cleverness doesn’t win against scale. Just ride the AI wave.
What this means for you
If you already use Big If True, there’s nothing to do: it’s a Claude plugin, so you always have the current version, and every improvement in the models reaches you automatically. If you’re new, you can add it in a minute at verso.ink/big-if-true.
We do have one updated recommendation: for everyday checks, start them with Claude Sonnet 5.5 at High thinking effort (both are settings in Claude’s model menu). In our latest tests it caught nearly as many errors as Opus 5.5, made no false accusations on our main test articles, and finished in about six minutes. If you’re checking something with high stakes, Opus 5.5 at High is still a fine choice.
The usual limits apply. When sources block AI access or aren’t online, the skill has less to work with. Claims that rest on your own interviews or documents will come back as “couldn’t verify” unless you share them. And it can still get things wrong, so look at the sources before you change anything.
Try it on your next piece
Big If True is free. Get it at verso.ink/big-if-true and run it on the next thing you’re about to publish.
We’ll keep testing every new model as it lands, so you don’t have to. Subscribe to our newsletter and we’ll tell you what we find.
And our plug, as always: if you want help bringing Claude and skills into your newsroom, hire us. Half of what we learned here was which clever ideas to leave out.
If Big If True catches something before your readers do, tell us at [email protected].