Anthropic's Misuse Report: AI Hacking, Dating Scams, Rival Lab Claims
Anthropic's misuse report details AI-run cyberattacks, a 5,000-persona dating scam, and claims that DeepSeek and Moonshot routed traffic to Claude.
What did Anthropic’s misuse report actually find?
Anthropic published a report on September 10th documenting how its Claude models have been abused in the wild, organized around seven categories of harm: cyber operations, influence operations, surveillance, scams, fraud, and related misuse. The headline shift is that attackers are no longer just using AI to write phishing emails or debug malware. They’re handing AI agents the keys and letting them execute parts of the attack directly, from reconnaissance to exploitation.
The report also documents a large-scale romance scam operation that used roughly 5,000 AI-generated personas to run conversations with tens of thousands of people across a dating app, with victims believing they were chatting with real humans. Beyond scams, Anthropic made pointed allegations against other AI labs, including claims tied to Alibaba and to Moonshot AI and DeepSeek, two of China’s highest-profile model makers.
Why does AI-driven hacking matter more than AI-assisted hacking?
There’s a real difference between an attacker using a chatbot to help draft code and an attacker deploying an agent that independently probes a network, identifies a vulnerability, and acts on it. Anthropic’s report describes the latter: cyber operations where agentic AI systems handled meaningful chunks of the attack chain rather than just assisting a human operator at each step.
One coffee. One working app.
You bring the idea. Remy manages the project.
This matters for a few reasons. Agents can work continuously without fatigue, they can be run in parallel across many targets at once, and they lower the skill floor required to execute a technically sophisticated attack. A single operator with access to a capable agent can potentially do what used to require a small team. That’s the core worry security researchers have raised about frontier models for the past two years, and this report is Anthropic’s own documentation of that pattern showing up in real incidents rather than in a lab test.
How did the dating scam operation actually work?
According to the report, the scam network built around 5,000 distinct AI personas and used them to run conversations with tens of thousands of targets on a dating platform. The personas were designed to seem like real people forming genuine romantic connections, a classic “pig butchering” style setup where trust is built over time before a financial ask arrives.
What makes this notable isn’t the scam format itself, which predates AI by years. It’s the scale. Running 5,000 believable, individually consistent personas across tens of thousands of simultaneous conversations is not something a human crew of scammers could do without a large staff. AI personas remove that staffing bottleneck, turning what used to be a labor-intensive operation into something closer to a software product.
What are the allegations against DeepSeek and Moonshot?
This is the part of the report that reads most like a competitive dispute rather than a straightforward safety disclosure. Anthropic alleges that Moonshot AI (maker of the Kimi models) and DeepSeek secretly routed some customer traffic to Claude on the back end, even while marketing their own models to users. In other words, a user might believe they’re talking to Kimi or a DeepSeek model, while some of those requests are actually being processed by Claude.
Separately, Anthropic attributes over 151 million exchanges to what it describes as an unauthorized model training campaign connected to Alibaba, the implication being that Claude outputs were harvested at scale to train a competing model. Neither DeepSeek nor Moonshot has publicly confirmed these claims as of this report’s release, and it’s worth treating them as allegations from an interested party rather than settled fact. Anthropic has a commercial incentive to highlight misuse of its own models by rivals, and independent verification of the routing claims hasn’t surfaced yet.
Is the timing of this report a coincidence?
Probably not entirely. The report landed on September 10th, two days after a viral resignation post from a former Anthropic and OpenAI pretraining researcher, Jacob Coxon, who said both companies were racing toward self-improving superintelligence “gambling with our lives.” That post picked up outsized engagement almost immediately, drawing scrutiny over whether it was organically amplified given the account’s minimal prior activity. Anthropic’s alignment science lead, Evan Hubinger, then quote-tweeted his agreement, stating he personally believes there’s more than a 10% chance AI could kill all humans within the next decade, and that Anthropic doesn’t yet have a working plan to solve alignment for superintelligence.
The next day, Anthropic CEO Dario Amodei published a lengthy essay on AI risk. The misuse report followed shortly after. None of this proves coordination, but the sequence, viral resignation post, alignment lead’s public admission, CEO essay, and a misuse report all within roughly a week, reads like a company trying to shape the safety conversation on its own terms while attention on the topic was already high.
What is Dario Amodei’s “race to the top” argument?
Remy is new. The platform isn't.
Remy is the latest expression of years of platform work. Not a hastily wrapped LLM.
Amodei’s essay frames the core problem as commercial incentives creating a race to the bottom: every lab wants to ship the best model with the best benchmarks first, and slowing down to test more thoroughly means risking being overshadowed by a competitor who didn’t wait. He argues the goal should be to flip that dynamic into a race to the top, where safety itself becomes the thing labs compete on, not just capability.
His proposed three-part plan centers on:
- Embedded evaluators, third-party teams given ongoing, employee-like access inside frontier labs to check adherence to safety commitments in real time rather than through periodic audits.
- Democratic coordination, meaning governments and institutions inside democracies aligning on shared rules rather than each lab or country setting its own standards.
- Global coordination, explicitly including China, which Amodei describes as the country with by far the most advanced AI capabilities outside the US. He acknowledges the tension directly: if democracies restrain their own AI development hoping China does the same, and China doesn’t, the result could be a geopolitical imbalance rather than a safer world.
President Trump’s public response to the wider concerns was blunt: the US is leading in AI, intends to keep leading, and “whoever wins AI wins.” That stance captures exactly the dynamic Amodei’s essay warns about, national and commercial incentives pushing toward speed over caution, even from people who acknowledge the risk.
Is this report credible, or is it self-serving?
Both things can be true at once. Anthropic has a real track record of publishing safety and misuse research, and documenting agentic cyberattacks and large-scale scam networks is useful information for the security community regardless of who publishes it. At the same time, the allegations against DeepSeek, Moonshot, and Alibaba serve Anthropic’s competitive interests by casting rival labs, particularly Chinese ones, as bad actors who free-ride on Claude’s outputs or misrepresent their own products to users.
Readers building with AI should take the cyber operations and scam findings seriously as a preview of where agentic misuse is headed. The claims against specific rival labs deserve more scrutiny before being treated as fact, since they currently rest on Anthropic’s own attribution with no independent confirmation.
Frequently Asked Questions
What is Anthropic’s AI misuse report?
It’s a report published by Anthropic on September 10th documenting real-world misuse of its Claude models across seven harm categories, including AI-driven cyberattacks, large-scale romance scams, and allegations that rival labs misused or misrepresented Claude’s involvement in their own products.
Did DeepSeek and Moonshot confirm the routing allegations?
No independent confirmation from DeepSeek or Moonshot has surfaced. The claim that they secretly routed some user traffic to Claude while marketing their own models comes from Anthropic’s report and hasn’t been verified by a third party.
How many AI personas were used in the dating scam network?
The report describes an operation using roughly 5,000 AI-generated personas that conversed with tens of thousands of people on a dating platform, with victims believing they were talking to real humans.
What is Dario Amodei’s “race to the top” proposal?
It’s a plan to shift AI development away from a speed-driven “race to the bottom” toward competition based on safety. It includes embedding third-party evaluators inside frontier labs, coordinating among democratic governments on shared rules, and pursuing global coordination that includes China.
Why did Jacob Coxon’s resignation post go viral?
Other agents ship a demo. Remy ships an app.
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
Coxon, a former Anthropic and OpenAI pretraining researcher, posted about resigning over concerns that both companies were racing toward superintelligence irresponsibly. The post spread unusually fast for an account with no prior posting history, prompting questions about coordinated amplification, though Coxon has said he simply asked friends to help share it.
