Generative artificial intelligences are useful, but they have a serious problem: they hallucinate. When a model invents data or code, it not only causes errors but can also open doors to real attacks. An assistant that generates a fake URL or a non-existent API key can deceive security systems and expose critical information.
How Technical Hallucinations Compromise Systems 🛡️
In development environments, an AI might suggest libraries that do not exist or invented security patches. A trusting programmer could implement that code, unknowingly installing a malicious package created by an attacker who registered that fictitious name. This attack vector, known as hallucination poisoning, exploits the user's trust in the model's responses. The authenticity of the information becomes a moving target.
The Assistant That Recommends You Buy a Bridge 🌉
You asked your AI for a safe recipe, and it suggested cloud ceviche. Now, security systems receive instructions from an assistant that firmly believes the Brooklyn Bridge is on sale. If the AI hallucinates that an employee is an administrator, it might grant them full access. Next time, before running its code, make sure it's not a fairy tale disguised as a script.