Why we get things wrong
Seven things that make checking a claim hard.
Software decides every Polygraph verdict, and some are wrong. These are the seven reasons, and what we do about each.
01The post leaves out the context.
Who is speaking, which city, since when: a reader knows, and the words alone carry almost none of it.
@forthecity· Tuesday
Crime is up 40%. They did nothing.
The claim, as the words give it
“Crime is up 40%.”
- Who is speakingA candidate for mayor, a week before the vote
- Which placeOne city, never named
- Which periodSince the mayor took office
- Which countReported violent crime only
- Against which baseThe lowest year in a decade
The claim, with its context
“Reported violent crime in the city is up about 40% from its lowest year in a decade.”
02Many records sit where we have no reader yet.
Dockets, state PDFs and county databases each need their own reader, and we build them one at a time.
“Trump won Georgia by about five points in 2016.”
- News and wire servicesThe result, rarely the marginThe result, rarely the marginRead
- Fact-checker archivesEarlier checksEarlier checksRead
- EncyclopediasThe winner, the electoral votesThe winner, the electoral votesRead
- Government statistics pagesA page per tableA page per tableRead
- PDF reports and filingsPage 40 of a 200-page reportPage 40 of a 200-page reportBuilding
- Court docketsA filing, as filedA filing, as filedBuilding
- State and county databasesThe county tally that settles this claimThe county tally that settles this claimNo reader yetwhere this claim’s record lives
- Video archivesWhat was said on airWhat was said on airNo reader yet
- Paywalled archivesReporting from before the webReporting from before the webNo reader yet
03The same words mean different things in different mouths.
A claim is checked on its own, so the speaker and the place have to travel with it.
“Crime is up.”
A mayor
Of the city, where violent crime rose
True
The local record supports it.
A senator
Of the country, where it fell
False
The national record contradicts it.
A post that quotes another carries two claims: that the words were said, and what the words say.
@sen_press· quoting the agency
They finally admit it: “the program has failed.”
The post’s claim
Did the agency say that?
The claim it carries
Did the program fail?
The same post from a satire account:
read straightFalseas meantTrue
04Readers forgive a word. Software must be told where.
To a reader these two posts make one claim, but our rules saw two and gave two verdicts.
“The United States is currently engaged in a war with Iran.”
Trueconfidence 1.0
“The United States government is currently engaged in a war with Iran.”
Falseconfidence .9
A readerOne claim, said two ways.
Our rulesTwo claims, two searches, two verdicts.
With numbers the line is exact: inside the estimates is true, a little high is mixed, double is false.
Estimates of the cost
True
“Up 40%”, from three bases
05Video and images say things the transcript misses.
A chart’s numbers can be only pixels, and tying words to picture is what we do least well.
The transcript
“…and as you can see right here, it has doubled since they took office.”
The transcript keeps “right here” and loses what it points at.
06Reliable sources disagree, and the verdict must say so.
Every source is rated, and a verdict weighs them by rating, never by how many pages agree.
- Agency statisticsPrimary12,400
- Wire serviceTier 1about 12,000
- Advocacy groupTier 340,000
- Anonymous blogUnrated100,000+
Truefor a post that said 12,000, disagreement noted
07Check early, and the record may not exist yet.
A check takes minutes, but the reporting that settles a claim takes hours or days.
“The plant shut down because of the new rule.”
Our ongoing mission.
These six are the standing commitments the verifier is held to, measured against claims that people settled first.
- 01Context that travelsWe check a claim with the account, the place and the period behind it.
- 02Readers for every recordWe build readers for dockets, PDFs, state databases and video.
- 03One claim, one verdictNear-identical claims get the same answer. Contested words come back mixed.
- 04Checked again laterWe re-check a first-hour verdict once the reporting exists, both dated.
- 05Never wrong while sureA confident verdict is one you can trust outright.
- 06Reading the whole postWe read the chart, the clip and the caption with the words.
Getting this right takes all of us.
Your fact-checks, feedback and support make it possible.
Just $5 to start
Last revised October 10, 2026.