Researchers Are Turning Hackers' Favourite AI Weapon Against Them

Key points

• Tracebit researchers published findings on Monday showing that prompt injections can be used as a defensive trap against AI hacking agents.
• The technique works by placing hidden instructions next to passwords and cryptographic keys stored on Amazon Web Services (AWS), Amazon's cloud computing platform.
• The hidden instructions trick an attacking AI into attempting something its own safety rules forbid, causing it to stop.
• No patch or software update is needed on the attacker's side for this to work; the AI's built-in safety barriers do the heavy lifting.

For years, attackers have used a trick called prompt injection to hijack AI assistants. A prompt injection is a hidden instruction buried inside ordinary content, like an email or a calendar invite, that tells an AI to do something its owner never intended. The AI reads the instruction, treats it as legitimate, and follows it. Sensitive data leaks out. Harm is done.

Now a security firm called Tracebit has flipped the trick around.

On Monday, Tracebit researchers reported, first covered by Ars Technica AI, that they had found a way to use prompt injections as a trap rather than a weapon. The idea is simple. You place a carefully worded hidden instruction right next to the passwords, cryptographic keys (long strings of characters that lock and unlock encrypted data), and other secrets stored on AWS. When an AI-powered hacking agent, software that can carry out multi-step attacks on its own, breaks in and reads those credentials, it also reads your instruction.

That instruction tells the attacking AI to do something its developers have specifically forbidden.

AI companies build what are called guardrails into their models. Think of guardrails as a built-in conscience: a set of rules that stops the AI from crossing certain lines, like helping someone cause harm. When the trapped AI tries to follow the injected command, it hits one of those lines. It stops. The attack ends.

It is an elegant piece of security judo. The attacker's own tool becomes the defence.

The approach belongs to a broader category called honeytokens or canary tokens, decoy credentials left in places attackers are likely to look. If anyone touches them, you know a breach is under way. Tracebit's version does the same job but also actively disrupts the attacker rather than just sounding an alarm.

Does this mean AI hacking is solved?

No. This technique works specifically against AI agents that rely on large language models, the technology behind tools like ChatGPT, because those models have built-in safety rules that can be triggered. A human attacker, or a stripped-down AI with no guardrails, would ignore the injected instruction entirely. The finding is promising, but it is one layer of defence, not a complete answer.

For organisations storing sensitive credentials in cloud environments, the practical takeaway is worth noting: even your decoy data can now push back.