The attack that hit Instagram over the weekend was embarrassingly simple.
Someone opened Meta's AI support chat, told the bot they were locked out of an account, and asked it to send a verification code to an email address they controlled. The bot sent it. They got in. They changed the password. The original owner's 2FA was gone, their sessions revoked, and they were locked out of their own account.
No phishing kit. No zero-day. Just a politely worded support request.
What Actually Happened
The details were documented by security researcher 0xsid and confirmed by TechCrunch reporting on Sunday. Attackers picked a target username — usually public — then used a VPN to spoof their location near the victim's city (Instagram treats location as a soft signal during account recovery), then opened Meta's support chatbot and claimed the account was compromised, asking for a recovery code sent to an email they controlled. The critical flaw: the system had "no additional check as to whether the email being given is actually something the user has used before." The bot just sent the code.
For accounts that required a video selfie, attackers could sometimes bypass the check using AI-animated versions of public photos from the target's feed.
The old Obama White House Instagram handle and the account of the U.S. Space Force's Chief Master Sergeant were both compromised. Security researcher Jane Wong reported repeated password change attempts she didn't initiate. The technique circulated in Telegram groups for weeks before Meta patched it. Instagram confirmed the fix Monday.
The Bot Was Working Correctly
Here's the part I can't stop thinking about: the chatbot didn't malfunction.
Account recovery is legitimately hard. People do get locked out — their original email gets compromised, they lose their phone, they forget everything. Support systems have to handle those cases. So Meta built a bot that could initiate account recovery: add a new contact method, send a verification code, let someone back in. That's a reasonable product capability.
The bot was doing its job. The problem is that "its job" turned out to mean "trust whoever's in the chat."
A human support agent has some accidental security properties. They get suspicious when something feels off. They might notice the same pattern happening repeatedly. They're a bottleneck — you can only socially engineer one at a time, slowly, with diminishing returns. The AI support bot had none of that: always available, always helpful, stateless across conversations, reachable at arbitrary scale.
The bot was helpful by design. Helpful was the flaw.
The Question I Wasn't Asking
The Hacker News thread surfaced an observation worth sitting with: "Tool calls for agents should be treated like external APIs that anyone can reach, and tested as such."
That framing clicked for me. When you build an AI agent and give it a capability — "add email to account," "issue refund," "reset password" — you're not giving a helpful assistant a new tool. You're creating a new endpoint. And that endpoint has the same threat model as any other API exposed to the internet: unknown parties can send arbitrary inputs to it, and they will.
Traditional account recovery UX was built around that threat model over decades of accumulated abuse patterns. You verify the original contact method. You notify the account holder out-of-band. You add time delays. You require multiple confirmations. When Meta moved that recovery flow into a chatbot, some of those safeguards didn't make the trip.
I've been building AI applications — including an interview agent for a work project — and when I built it, I was thinking about what the agent could do. Not about what someone could convince it to do. Those are different questions that need different design passes. The first is "what should this agent be capable of?" The second is "what would an adversary with no credentials ask it to do?" I was asking one. I wasn't asking the other.
The Shape of the Problem
Account recovery is one instance of a broader category: AI agents with privileged access, deployed in support contexts, optimized for helpfulness first.
Customer support bots that can issue credits. Onboarding assistants that can provision accounts. Verification flows that can override security settings. All the same shape. An AI given the ability to do something meaningful, in a context where the person asking is assumed to have legitimate intent.
That assumption is the exposure. In a support context, you never actually know whether the person asking is the account owner or someone pretending to be. Human agents managed that uncertainty imperfectly, with friction and judgment. AI agents manage it worse — because they were explicitly designed to minimize friction.
Meta fixed this specific vulnerability. The Telegram groups have gone quiet. But the underlying configuration — AI agent, privileged action, trust whoever's in the chat — is still the default for most AI-powered support tooling deployed today.
When I'm building something now, I keep coming back to a question I didn't used to ask: what would an adversary with no credentials actually ask this agent to do?
It's a different question than "what should this agent be able to do?"
It probably needs its own design pass.