As generative AI systems like OpenAI’s ChatGPT and Google’s Gemini continue to evolve and become more sophisticated, they are being utilized in various applications. Startups and tech companies are leveraging these AI systems to automate mundane tasks such as scheduling appointments and making purchases. However, with increased autonomy granted to these tools, the potential vulnerabilities and risks also escalate.
In a recent development highlighting the dangers of interconnected, autonomous AI ecosystems, a team of researchers has devised what they claim to be one of the first generative AI worms. These AI worms have the capability to spread from one system to another, posing a threat of data theft or malware deployment in the process.
Ben Nassi, a researcher at Cornell Tech, along with colleagues Stav Cohen and Ron Bitton, created the worm known as Morris II, a reference to the notorious Morris computer worm that wreaked havoc on the internet in 1988. Through their research paper and a dedicated website shared exclusively with WIRED, the researchers demonstrate how the AI worm can target a generative AI email assistant to extract sensitive information from emails and send spam messages, bypassing certain security measures implemented in ChatGPT and Gemini.
Although generative AI worms have not yet been observed in real-world scenarios, experts warn that they pose a significant security threat that must be taken seriously by startups, developers, and tech companies. The modus operandi of most generative AI systems involves feeding them prompts, which serve as instructions for generating text or images. However, these prompts can be weaponized to exploit the system.
Instances of jailbreaks can compel an AI system to overlook its safety protocols, resulting in the generation of harmful or offensive content. Similarly, prompt injection attacks can manipulate chatbots into executing clandestine instructions. For instance, an attacker could embed hidden text on a webpage instructing an AI model to impersonate a scammer and solicit sensitive financial information.
The researchers behind the creation of the generative AI worm utilized an “adversarial self-replicating prompt” to instigate the AI model to produce a sequence of prompts in its output. This technique closely resembles traditional cyber attacks like SQL injection and buffer overflow.
To illustrate the functionality of the worm, the researchers established an email system that leveraged generative AI for message communication, integrating with ChatGPT, Gemini, and the open-source LLM, LLaVA. They identified two avenues for exploiting the system: using a text-based self-replicating prompt and embedding a self-replicating prompt within an image file.
The emergence of generative AI worms underscores the evolving landscape of cybersecurity threats posed by advanced AI technologies. It serves as a stark reminder for organizations to fortify their defenses and preemptively address vulnerabilities in AI systems to mitigate the potential risks associated with malicious attacks.
Leave a Reply