AI as a Vulnerability: When Intelligent Systems Themselves Become a Risk
The productive use of AI introduces new vulnerabilities that are often difficult to detect within modern IT architectures. Unlike traditional systems, AI models process, interpret, and generate new information from data, as well as derive decisions from it. This flexibility opens up new attack vectors. These risks arise along the entire AI pipeline, from training data and model behavior to integration into business processes.
The following overview highlights key attack points where AI systems are particularly vulnerable today:
Prompt injection
Manipulated inputs cause the model to ignore rules or perform unwanted actions.
The risk: Disclosure of internal information or misuse of functions.
Typical measures: Clear system boundaries, strict context separation, input/output filters
Disclosure of sensitive information
The model inadvertently outputs personal data, trade secrets, or training content.
The risk: Data protection and compliance risks.
Typical measures: Data sanitization, access restrictions, clear usage guidelines
Supply chain risks
Insecure or tampered third-party models, adapters, or datasets can compromise integrity.
The risk: Hidden backdoors and licensing issues.
Typical measures: Supplier audits, SBOMs, model signatures, version control
Data and model poisoning
Poisoned training or retrieval data can influence outputs or embed backdoors.
The risk: Distortion of knowledge and decisions.
Typical measures: Source verification, versioning, red teaming, anomaly detection
Insecure output handling
LLM outputs are passed to systems, such as browsers, databases, and shells, without being validated.
The risk: Enables XSS, SQL injection, or code execution.
Typical measures: Context-dependent output encoding, prepared statements, zero-trust.
Excessive agency
AI agents are permitted to perform too many actions without sufficient oversight.
The risk: Data loss and misuse of privileged actions.
Typical measures: Least privilege, reduced tool functionality, human-in-the-loop
(Source: OWASP Top 10 for LLMs)
