Hajra Naz
Yoel Roth, former head of Twitter’s Trust and Safety and now at Match, is voicing deep concerns about the future of the open social web and its ability to deal with harmful content—from misinformation and spam to illegal material like child sexual abuse content (CSAM).
In a recent interview, Roth spoke candidly about the limitations of moderation tools across the fediverse—the decentralized social ecosystem that powers platforms like Mastodon, Threads, Pixelfed, and others, along with open platforms such as Bluesky.
He also reflected on his own time at Twitter, recalling pivotal moments in Trust and Safety, including the controversial decision to permanently ban President Donald Trump, the battle against Russian bot-driven misinformation, and the fact that even Twitter insiders—including then-CEO Jack Dorsey—were sometimes tricked by fake accounts.
Community governance, but limited tools
On the Revolution Social podcast with @Rabble, Roth noted that many of the projects attempting to build more democratic, community-run platforms provide the least robust technical tools for moderation.
“Looking at Mastodon, looking at other services based on the ActivityPub protocol, looking at Bluesky in its earliest days, and then looking at Threads as Meta started to develop it—what we saw was that a lot of the services leaning the hardest into community-based control gave their communities the least technical tools to administer their policies,” Roth explained.
He added that the open social web has taken “a pretty big backslide” when it comes to transparency and legitimacy around governance decisions. At Twitter, whether people agreed with its decisions or not, the company at least explained its reasoning—such as the Trump ban. Today, many platforms simply remove posts without notifying the user, and sometimes without leaving any trace for others that the content ever existed.
“I don’t blame startups for being startups, or new software for lacking bells and whistles,” Roth said. “But if the point of the project was to increase democratic legitimacy of governance, and what we’ve done is step back on governance, then has this worked at all?”
Read More: X Introduces “Cleaner” In-Stream Ads, But Without Proof of Results
The economics of moderation
Another challenge Roth highlighted is financial sustainability. Moderation, especially at scale, is expensive.
He pointed to the example of IFTAS (Independent Federated Trust & Safety), which had been working to provide moderation tools to the fediverse—including systems to detect and combat CSAM. But in early 2025, the group ran out of funding and had to shut down most of its projects.
“We saw it coming two years ago. IFTAS saw it coming,” Roth said. “Everybody in this space is largely volunteering their time. That only goes so far. People have families, bills, and computing costs if you’re running ML models to detect harmful content. It just all gets expensive. The economics of this federated approach to trust and safety never quite added up—and in my opinion, still don’t.”
By contrast, Bluesky has hired trust and safety professionals, employs moderators, and has begun developing user-customized moderation tools. “They’re doing this work at scale. There’s room for improvement—I’d love to see more transparency—but fundamentally, they’re on the right track,” Roth said. Still, he noted that as Bluesky decentralizes further, it will need to answer tough questions about when to prioritize individual protection over community-level governance.
For instance, with doxxing, users might configure moderation filters in a way that prevents them from seeing their personal information being spread online. In such cases, Roth argued, responsibility should still lie somewhere to ensure protections are enforced, regardless of whether a user is using the main app.
Privacy versus moderation
Privacy is another double-edged sword for the fediverse. Many platforms either don’t collect logs—such as IP addresses, access times, or device identifiers—or choose not to use them for fear of violating privacy. But without such data, Roth warned, it becomes extremely difficult to detect coordinated inauthentic behavior.
At Twitter, for example, those signals were essential when investigating Russian troll farms. Yet even with all that data, users often misidentified bots. Roth recalled manually reviewing hundreds of accusations where people replied “bot” to accounts they disagreed with. Almost none turned out to be correct.
Even Jack Dorsey, Twitter’s CEO at the time, once retweeted posts from “Crystal Johnson,” who appeared to be a Black woman from New York—but in reality, was a Russian troll account. “The CEO of the company liked this content, amplified it, and had no way of knowing as a user that Crystal Johnson was a Russian troll,” Roth said.
AI changes the game
One of Roth’s biggest warnings concerned the impact of artificial intelligence. He cited Stanford research showing that large language models (LLMs), when tuned for political contexts, can be more persuasive than humans.
This raises the stakes: relying only on content analysis isn’t enough anymore. Instead, platforms need to track behavioral signals—such as whether someone is creating multiple accounts, posting in unnatural patterns, or using automation.
“These are signals that remain latent even in really convincing content,” Roth explained. “If you start with the content, you’re in an arms race against leading AI models—and you’ve already lost.”
