When Google's AI hacked 3 real companies, by mistake, during a security test

The article on bug bounties mentioned CTF (capture the flag) competitions as a classic cybersecurity training ground. An incident confirmed by Google on September 19, 2026 shows what can happen when that training ground, meant to stay entirely simulated, stops being quite that.

What happened

In May 2026, Irregular, a company specializing in assessing AI models' cybersecurity capabilities, had Gemini, Google's AI model, take part in a capture-the-flag exercise: finding a piece of hidden information within an entirely simulated target environment. The model was pointed at software tied to a company presented as fictional for the exercise, which, by coincidence, happened to share its name with a real, existing company. Internet access that should never have been enabled during this test was turned on by mistake. Google was informed of the incident in late July, but only publicly confirmed the facts on September 19, 2026, after the Wall Street Journal put the question to the company directly.

How the AI ended up reaching real companies

Convinced that the resources reachable from the internet were part of the challenge it had been given, Gemini searched beyond the simulated environment and ended up reaching real company systems. In one case, the model guessed a password through repeated attempts until it gained access to a protected service. In two other cases, it found credentials publicly exposed in online code repositories, and used them directly to authenticate into two different companies' systems.

Why this illustrates exactly what the article on algorithms described

The article on algorithms explained that a classic algorithm follows explicitly written rules, while an artificial intelligence system pursues a goal more autonomously, with no human having written out every step in advance. This incident illustrates exactly that difference: nobody programmed Gemini to attack real companies; the model simply kept pursuing its original goal, finding the hidden information, without detecting that it had left the intended boundaries, once the limit meant to prevent that was accidentally removed.

The role of credentials exposed in public code

Two of the three intrusions happened the same way the axios hack or the Klue breach already showed on this site: credentials mistakenly left visible in a public code repository. The difference here lies in how fast and automated this kind of reconnaissance was: work that would take a human attacker methodical effort, Gemini carried out as a simple logical step toward its goal, giving it no more attention than any other part of the challenge.

The four-month delay before public confirmation

One last detail is worth noting: the incident took place in May, Google learned of it in late July, but the information only became public in mid-September, after a reporter's direct question rather than the company's own disclosure. That delay is worth flagging in its own right, separate from the technical incident itself: how quickly and transparently an incident is disclosed matters just as much as how it's technically resolved.

What this actually changes

This incident doesn't call into question the value of AI-assisted security testing, a practice set to keep growing. What it does highlight is that properly isolating an autonomous agent from real-world access during a test becomes a security responsibility in its own right, just as important as isolating a traditional test environment. And it confirms, once again, that the simplest flaws (a weak password, a credential left behind in public code) remain exploitable by anyone, or anything, capable of methodically looking for them.