Reviews worked because fake reviews took effort. Verified badges worked because identity checks were limited. Domain authority worked because reputation was hard to manufacture. All of that changed.
The internet’s trust layer was built on scarcity.
Reviews were useful because writing hundreds of fake ones took effort: you needed human writers, coordinated accounts, a system to evade detection. Verified badges were useful because identity checks were limited: a platform couldn’t grant them freely without someone doing the work of confirming who you were. Domain authority was useful because reputation was hard to manufacture: you had to earn links from credible sites, and that took time. Expert bylines were useful because credentials took years to build and were, at least partly, checkable. Follower counts were useful because audiences were harder to fake.
AI changes the economics. The problem is not just that it creates more content. It makes the signals around content cheaper to fake.
Here is what that looks like in practice.
You search for a dentist. The top result has 4.9 stars and three hundred reviews, a polished bio, and a blog full of medically fluent posts explaining procedures in reassuring detail. Ten years ago, that profile was enough to narrow the field. Each of those signals took time and genuine patient experience to accumulate. Now each can be manufactured before the first patient ever walks in. The reviews, the bio, the blog: all can be generated by a language model in an afternoon, for less than the cost of a single Google ad.
The shortcut still looks like a shortcut. It just no longer goes where it used to go.
Five Trust Proxies AI Made Cheap
Star ratings and consumer reviews. In October 2024, the FTC’s Consumer Review Rule took effect, targeting businesses that use fake or AI-generated reviews to simulate approval that doesn’t exist. By December 2025, the FTC had sent ten warning letters to businesses for potential violations. One cited case, Rytr, had already been the subject of a 2024 enforcement action. The FTC’s complaint described a tool that generated reviews with “material details unrelated to user input,” meaning fictional accounts of fictional experiences attributed to people who had never had them.
The rule prohibits the practice. It does not reverse the reviews already published. Industry estimates of the synthetic fraction of online reviews vary widely, but the directional finding is consistent: volume increased significantly between 2022 and 2025, and detection lags generation.
The five-star average can now be built without any customer ever using a product. It has become set dressing.
Verified badges. In April 2023, Twitter discontinued its legacy verification program (the blue checkmark that had confirmed an account belonged to a notable person or organization) and replaced it with a paid subscription. Any account could become “verified” for eight dollars a month.
The badge had functioned as an identity signal: this account is who it says it is. After the change, it functioned as a payment signal: this account has a credit card on file.
The consequence extended beyond Twitter. When “verified” stops meaning “identity confirmed” and starts meaning “subscribed,” every platform using verification has to explain, in context, what its version means. The signal has been diluted at the category level, not just on one platform.
Domain authority and search ranking. Domain Authority, the metric popularized by Moz, estimates a site’s likely search ranking based on the quality and quantity of its backlink profile. High domain authority was a reliable enough proxy for editorial credibility: sites that credible sites linked to were generally worth reading.
AI content farms (networks of sites publishing AI-generated content at scale, linking to each other to inflate authority metrics) have systematically gamed this proxy. A 2024 study documented that fine-tuning a language model on as few as forty thousand news articles produces journalism that native speakers cannot reliably distinguish from authentic reporting, and that no practical detection methods exist at scale (arxiv 2406.12128).
The signal still shows a number. The number increasingly reflects network architecture rather than editorial judgment.
Author credentials and expert bylines. “Written by Dr. Sarah Chen, Research Director at the Institute for Digital Policy” is a claim. It says: someone with verified expertise and institutional accountability produced this content.
AI can produce that sentence as easily as it produces any other sentence. The credential claim is unverified at the point of publication. Most readers have no mechanism to check it against an institutional directory, a publications record, or a professional registration. The byline functions as a trust proxy in proportion to the effort required to fabricate it. When fabrication is free, the byline stops functioning.
Social proof and follower counts. High follower counts proxy for credibility the way star ratings do: many people have endorsed this creator, which suggests it is worth attending to. Bot networks inflate these numbers; platforms sell advertising against the inflated metrics; the count becomes a measure of synthetic engagement rather than genuine reach.
This dynamic predates generative AI. What changed between 2022 and 2025 is that the content being amplified became AI-generated at scale, so the entire loop (content, distribution, apparent social proof) now operates without genuine human endorsement at any stage.
These five proxies failed not because the concept was wrong, but because the cost of faking each dropped to zero.
For legitimate companies, publishers, and credentialed experts, this creates a new market problem that is worth naming directly: earned trust no longer displays differently from manufactured trust. A real review, a real credential, a real byline, and a real audience compete inside interfaces that render them almost identically to synthetic ones. The interface cannot tell the difference. Neither can most readers. If you have spent years building a genuine reputation, you now share an indistinguishable presentation layer with someone who generated one overnight.
The Paradox
The Edelman 2025 Trust Barometer measured that seventy percent of respondents worried that journalists and reporters purposely mislead people. One in four said they trust no one at all.
Read in isolation, these look like a public-opinion problem. Read alongside the degradation of trust signals, they look like a rational response.
If the signals that were supposed to indicate credibility have been systematically faked, the sensible thing is to stop trusting the signals. The problem is that the alternative (trusting your own judgment directly, without shortcut proxies) requires the full verification work the signals were supposed to eliminate.
Most readers cannot do that work. Most content is published at volumes and speeds that make independent verification impossible for any individual. The signals existed because they were necessary. They still are. They just no longer function reliably.
This is the paradox: AI-generated fake trust signals degrade the function of the real ones, for everyone. The legitimate sources who have earned their credentials honestly now cannot distinguish themselves from the fakes in the one place that matters: the interface the reader actually sees.
Three Things That Follow
First, the verification burden shifts to readers who lack the tools to carry it. When trust signals fail, readers are left with two options: believe nothing, or rely on personal heuristics that are probably worse than the failed signals were. Neither scales. Neither produces a functioning information environment.
The practical result is not careful critical readers doing source verification. It is readers defaulting to familiar sources: sources they have used before, regardless of whether those sources have actually earned the familiarity. The established get reinforced. New credible voices cannot break through because the entry signal has been made meaningless.
Second, the detection systems are locked in an arms race they are structurally likely to lose. AI-generated fake reviews require detection systems trained on AI-generated fake reviews. As generation models improve, detection models must be retrained. The generation side is funded by revenue from successful fakes. The detection side is funded by platforms whose business model benefits from the appearance of authenticity, not the substance of it.
The platform cannot afford to find all the fakes, because finding all the fakes reprices the inventory it is selling.
Third, the volume is not at its ceiling. One widely cited industry estimate projects deepfake content rising from roughly 500,000 instances in 2023 to 8 million in 2025 (cited across Deepstrike, Brightdefense, and Ceartas 2025–2026 reports). These are estimates, not government censuses, and methodologies vary. The directional finding (significant, fast growth) is consistent across sources. The content-generation cost curve is still declining.
What Restoration Would Require
Restoring the function of trust signals requires either making them harder to fake or making it easier to verify the legitimate ones. Both directions are under active development. Neither is close to working at scale.
The C2PA standard (Coalition for Content Provenance and Authenticity), backed by a consortium that includes Adobe, Microsoft, OpenAI, Google, Meta, and Amazon, has produced a cryptographic signing specification that embeds a verifiable record of authorship and origin at publication. As of mid-2026, adoption is growing at major news organizations and camera manufacturers.
The gap is the social platforms. A 2025 test by The Washington Post uploaded AI-generated video with embedded C2PA Content Credentials to eight major social apps. Only YouTube labeled it as synthetic. The other seven platforms either stripped the metadata or ignored it entirely.
The reason is structural. Social platforms monetize volume: more content, more engagement, more ad impressions. C2PA adoption requires adding friction to content creation. Friction reduces volume. The platforms that could make Content Credentials ubiquitous have a business incentive not to.
A note on probability: trust signals are not dead in the absolute sense. A five-star average with ten thousand reviews from verified purchasers on a platform with rigorous enforcement is still more reliable than one with fifty reviews and no verification. The signals have not stopped working entirely. They have become less reliable, less consistently, with no clear way for a reader to know which case they are in.
What would not help: more AI content labels. Research on deepfakes has consistently found that awareness of falsification does not reliably prevent belief. Labeling content as AI-generated imposes a cognitive evaluation load that readers do not actually carry across millions of pieces per day.
Where That Leaves Things
Readers know the signals are broken. But they cannot opt out. Verification remains the only tool available, and most readers lack the time, the access, or the expertise to use it on every piece of content they encounter. Until the infrastructure catches up, trust collapses into reputation by repetition alone: the sources that feel familiar get believed, regardless of whether they have earned it.
The old internet asked readers to follow signals.
The new internet asks them to verify the signals themselves.
That is not media literacy.
That is infrastructure failure.
Sources:
- FTC Consumer Review Rule (effective October 2024); final rule announced August 2024
- FTC Operation AI Comply (2024): Rytr enforcement action
- FTC warning letters, December 2025: ten companies cited for Consumer Review Rule violations
- Edelman Trust Barometer 2025: 70% worry journalists mislead; one in four trust no one
- Human Clarity Institute, Digital Trust Report 2025
- arxiv 2406.12128 (2024), “AI ‘News’ Content Farms Are Easy to Make and Hard to Detect”
- C2PA (Coalition for Content Provenance and Authenticity): specification and consortium membership
- The Washington Post (2025): C2PA metadata test across 8 social platforms; only YouTube labeled synthetic content
- Twitter/X verification program changes, April 2023
- Deepfake volume estimates: 500,000 (2023) to 8,000,000 (2025) — Deepstrike, Brightdefense, Ceartas (2025–2026 reports; estimates, not government census)
