Autonomous cyber-attacks: AI as a game-change

Cybercriminals are putting ideas for using AI into practice at an impressive pace. They do this by smuggling seemingly harmless software onto their victims’ computers. But this is merely a cover; these programmes contain their own AI models designed to generate and execute malicious code directly.

Written by Uwe Sievers

Abstract representation of AI-malware on a laptop infecting connected devices

AI is now ubiquitous, for example in sport and agriculture. The soccer ball used in the World Cup, for instance, uses sensors to continuously provide data that AI utilises for match analysis. In agriculture, AI recognises plants and enables targeted, laser-based weed control, which is considered more environmentally friendly. But these are just the legitimate examples – cybercriminals are also capitalising on the potential of artificial intelligence. This includes novel types of attacks that are putting defenders under pressure.

Artificial Intelligence & IT Security

Cybercriminals are turning to new forms of AI-based attacks

Yet advanced use of AI is also found in the criminal sphere. The methods are becoming increasingly sophisticated, as cybercriminals implement creative ideas at an impressive pace. For example, they smuggle seemingly harmless software onto their victims’ computers. This does not normally appear suspicious to any endpoint protection system (EDR). However, the software contains algorithms designed to generate malicious code using AI and deploy it directly. This malware is then adapted to the system, taking into account installed applications, and is also tailored to the endpoint security solution in use.

Endpoint Security in the Age of RaaS

One such piece of malware is PromptFlux. This malware was identified by Google’s Threat Intelligence Group (GTIG). Their analysis revealed that PromptFlux contacts Google’s AI by using the API of the Gemini AI system to rewrite and obfuscate its own VBScript code at regular intervals. This provides cybercriminals with tools that can dynamically alter their own behaviour at runtime. PromptFlux thus also attempts to remain undetected over a prolonged period. Promptsteal operates on a similar principle. However, this malware is used more to spy on systems and exfiltrate information. The malicious code has already been deployed against Ukraine. Promptsteal disguises itself as an image generator and embeds itself, amongst other places, in the Windows autostart folder.

PromptLock takes an even more sophisticated approach. This malware is considered the first ransomware with genuine AI integration and was discovered by researchers at the security specialist ESET. “PromptLock comes with its own language model, installs it on the local system and uses it to automatically generate scripts during the ongoing attack,” explains Juraj Janosik, Director of Artificial Intelligence at ESET, during his presentation at a security conference. The malware uses a freely available language model. It operates without a connection to the cloud and can act independently of command-and-control servers. The AI therefore creates the attack scripts directly on the infected computer and decides independently which local files to search and encrypt. It runs cross-platform on Windows, Linux and macOS and generates scripts in the Lua programming language, which include the interpreter required for execution.

AI in the workplace is becoming a risk – awareness training is essential

Smartphones are also affected

Malware utilising AI does not stop at mobile devices either. ESET has also discovered PromptSpy, the first known Android malware to integrate generative AI such as Google Gemini directly into its execution process on smartphones. The malware disguises itself as a harmless application, but uses the AI to analyse and manipulate the smartphone’s user interface. This enables sophisticated and automated attacks, such as the theft of banking details.

Security experts point out that the code of some malware suggests it is still under development. Final versions could be significantly more sophisticated and radical, and are likely to start causing trouble very soon.

AI attacks on social media

Cybercriminals recently used AI in a very different way on Instagram. They exploited a vulnerability in Meta’s AI-powered account recovery system for Instagram. This involves an automated recovery process using an AI chatbot. The attackers targeted third-party user accounts, primarily those of celebrities.

However, this attack did not require a great deal of AI expertise. “The attackers simply had to ask Meta’s AI support bot to link the relevant accounts to a new email address in order to take over the account,” writes the German news magazine “Der Spiegel”. According to the report, they targeted, amongst others, the accounts of prominent US politicians, which they used to spread Islamist propaganda. According to Meta, more than 20,000 Instagram accounts were compromised.

Experts warn that the automation of such highly sensitive processes poses particular risks, especially when an AI system can make security-related changes without supervision. If an AI system actively takes over security functions, the support workflow itself becomes a point of vulnerability, they say. This is precisely where the security significance of this case lies.

For IT security researchers, AI malware such as that mentioned above serves as a clear warning sign. Whilst major LLM operators such as Google or OpenAI adapt their AI models in response to such discoveries to prevent misuse, cybercriminals usually adapt quickly to these changes. The never-ending race between attackers and defenders thus continues on a new level.

In addition, Janosik highlights another aspect: “Interestingly, however, we are also observing that, thanks to these AI tools and agent-based workflows, many of the tools that were previously only available to typical APT groups and the ‘Big Four’ – Russia, China, Iran and North Korea – are now also falling into the hands of ordinary cybercriminals. In a sense, AI is therefore lowering the barrier to entry for attackers.”

The AI Orchestra: Why Multi-Agent Systems Need a Conductor