AI Penetration Testing

AI penetration testing is a manual, adversarial security assessment of an application built on a large language model or other machine-learning model, covering the model endpoint, prompts and guardrails, retrieval pipelines, agents and tools, and the surrounding application. It tests for prompt injection, data exfiltration, excessive agency, improper output handling, supply chain weaknesses and abuse, and reports exploitable findings with evidence. It differs from a model evaluation or benchmark because it tests what an attacker can do to your deployment, not how the model scores in isolation.

It is the current OWASP list of the most critical risks in LLM applications: prompt injection, sensitive information disclosure, supply chain, data and model poisoning, improper output handling, excessive agency, system prompt leakage, vector and embedding weaknesses, misinformation and unbounded consumption. We use it as the coverage baseline for every AI penetration test, together with MITRE ATLAS, which catalogues real adversary techniques against machine-learning systems, so you can show precisely which risks were tested and with what result.

Yes, and this is where the highest-impact findings now sit. We enumerate every tool and Model Context Protocol server the model can call, test what an injected instruction can make it do, with which permissions and against which systems, and whether human-in-the-loop approvals can be skipped or exhausted. A chatbot that leaks its system prompt is a finding; an agent that can be steered into sending email or changing a record is an incident.

Prompt injection is the manipulation of a model through untrusted text, either directly by the user or indirectly through content the model reads, such as documents, web pages or tool results. It cannot be fully prevented at the model layer with current technology, so a secure design limits what a compromised model can reach: least-privilege tools, scoped retrieval, output validation and approval steps for consequential actions. Our test measures how well those boundaries hold, not whether a clever prompt exists.

The EU AI Act (Regulation (EU) 2024/1689) requires high-risk AI systems to achieve an appropriate level of accuracy, resilience and cybersecurity under Article 15, with the main high-risk obligations phasing in during 2026 and 2027, and governance frameworks such as NIST AI RMF and ISO/IEC 42001 call for testing and risk treatment of AI systems. DORA and NIS2 treat AI systems as ICT systems and require their security to be tested like any other. An AI penetration test produces the evidence these frameworks ask for, mapped per finding.

No. Automated red-teaming tools are useful for coverage and we use them for that, but they generate refusals and jailbreaks in isolation. They do not chain an injection into a retrieval scope to reach another tenant's documents, do not reason about your business logic and do not know which tool call would matter. Every finding we report has been confirmed manually, reproduced and tied to a concrete impact in your system.

Every test is quoted on scope; there is no fixed price list. The drivers are the number of models and applications, the retrieval sources and tenants, the number and power of the tools and agents, whether the surrounding application and APIs are in scope, the cost ceiling for inference during testing and whether a retest is included. After a scoping call you receive a written proposal within a few working days with the day count and the named lead tester.