What is AI-led Penetration Testing?

A couple of years ago, “automated penetration testing” mostly meant a vulnerability scanner with better marketing, but that has changed. AI agents built on large language models can now plan an attack, run the tools a human tester would use, read the results, decide what to try next and chain several small weaknesses into a working exploit, all with very little human input. 

Vendors have rushed to sell this as AI-led penetration testing, and buyers are understandably trying to work out what it is, how much of it is real, and whether it can replace the consultant-led test they already pay for.

This guide covers what AI-led penetration testing is, how it works in practice, what it is genuinely good at, and where it still needs an experienced human tester alongside it.

A quick note on terminology. “AI penetration testing” is used to mean two quite different things. One is testing your AI systems, such as chatbots and LLM-powered features, for weaknesses like prompt injection. The other, which is the subject of this article, is using AI to carry out the penetration test itself.

What is AI-led penetration testing?

AI-led penetration testing is a security assessment in which an AI system, usually one or more autonomous agents built on a large language model, performs much of the work that a human penetration tester would traditionally do. That includes mapping the target, identifying likely weaknesses, attempting to exploit them, and writing up what it found.

The important word here is “led”. Automated security testing has existed for decades, but older tools followed fixed scripts, meaning that they checked for known issues in a set order and stopped there. 

An AI agent behaves differently because it reasons about what it is seeing. If it finds an exposed login page, it can decide to look for default credentials, notice an unusual error message, adjust its approach and try again, which is much closer to how a human attacker thinks than anything a traditional scanner can do.

How does AI-led penetration testing work?

Most AI pentesting platforms follow a similar loop, even if the branding differs.

The agent starts with reconnaissance, gathering information about the target’s domains, open ports, exposed services and web applications. It then forms hypotheses about where weaknesses might be, based on what it has found and what it has learned from vast amounts of security material during training. Next, it acts on those hypotheses by running real tools, sending crafted requests and attempting exploits. Crucially, it then reads the response and adapts, which is the step that separates an agent from a script. If an attempt fails, it can reason about why and try a different route, and if it succeeds, it can use that foothold to look for the next step.

Better platforms also try to validate each finding before reporting it, for example by actually demonstrating that a vulnerability can be exploited rather than simply flagging that it might exist. That validation step matters a great deal, because it is the main defence against the flood of false positives that has always plagued automated testing.

How good is AI-led pentesting, really?

The headline moment came on the 24th of June, 2025, when XBOW, an autonomous AI pentester, reached the top of HackerOne’s US bug bounty leaderboard, which was the first time an autonomous system had done so. 

According to XBOW’s own write-up, it had submitted roughly 1,060 vulnerabilities at that point, including findings later rated critical and high severity by the programme owners.

Interestingly, the detail behind that headline is just as instructive as the headline itself. XBOW’s security team reviewed the findings before they were submitted, partly to comply with HackerOne’s rules on automated tools. 

A sizeable share of submissions were also marked as duplicates, informative or not applicable, which is a useful reminder that volume and value are not the same thing.

The UK’s National Cyber Security Centre has also taken notice, although mostly from the attacker’s side. On the 4th of May, 2026, the NCSC warned that AI is speeding up the discovery of software vulnerabilities and that organisations should expect a “patch wave” of urgent updates as a result. 

Then, on the 21st of May, 2026, it published ten considerations for organisations using AI models to find vulnerabilities themselves. Perhaps most importantly, its central point was that simply identifying vulnerabilities does not inherently improve security, meaning that finding issues is only worthwhile if you have a process to prioritise and fix them without overwhelming your team.

Where AI-led penetration testing performs well

AI agents are strongest where the work is broad, repetitive and well documented.

They are fast and tireless, which makes them well suited to covering a large external attack surface, re-checking the same estate frequently, and catching common web application flaws such as injection issues and misconfigurations. 

They also make continuous penetration testing far more practical, because an agent can re-run its checks every time something changes rather than once a year. For organisations that currently test annually and leave long gaps in between, that more regular coverage is a genuine improvement.

There is also a cost argument because an agent can do a lot of the groundwork without a consultant’s day rate attached; AI-led and automated penetration testing can bring a reasonable level of assurance within reach of smaller organisations that might otherwise only ever run a vulnerability scan.

Where AI-led penetration testing falls short

The limitations are real, and they tend to sit in exactly the places where a breach does the most damage.

Business logic flaws are the clearest example because an AI agent can spot a textbook SQL injection, but it often struggles to recognise that a discount code can be applied twice, or that a junior user can approve their own expense claim by changing one value in a request. Spotting that kind of issue requires understanding what the application is supposed to do, which usually comes from talking to the people who built it.

Context and judgement are another gap because a good consultant knows that a low-severity finding on a payroll server matters more than a medium-severity finding on a test box, and will rate and explain risk in the terms your board and auditors actually care about. Agents are improving here, but they do not know your business.

Safety and scope control deserve careful thought too. An agent that is allowed to exploit vulnerabilities on its own is, by definition, taking actions against live systems. On a fragile internal network, or in an operational technology environment where an unexpected request can disrupt physical processes, that autonomy needs firm boundaries and a human who can step in.

Finally, there are data and assurance questions. The NCSC’s guidance specifically raises where AI models are hosted, which legal jurisdiction they sit in, and what the provider’s terms say about retaining the data they process. If an AI platform is reading your network traffic and application responses, you need to know where that information goes. Notably, many compliance frameworks and client contracts still expect testing to be carried out by qualified, accredited people, so an AI-only report may not satisfy an auditor, an insurer or a customer’s security questionnaire.

AI-led vs consultant-led penetration testing

The most useful way to think about it is as a division of labour rather than a contest.

In practice, the strongest approach for most organisations combines the two. Automation and AI agents handle the wide, repetitive coverage between engagements, while experienced consultants focus their time on the high-value targets, the logic flaws and the judgement calls that machines still get wrong. That is also why consultant-led penetration testing remains the benchmark that AI pentesting tools are measured against.

Should you use AI-led penetration testing?

If you have a large or fast-changing external estate and currently rely on an annual test, AI-led testing can be a sensible way to close the gaps between engagements. It is less suited to being your only form of assurance, particularly if you handle sensitive data, operate in a regulated sector, or need a report that will stand up to scrutiny from auditors and clients.

Before buying any AI pentesting service, it is worth asking a few direct questions. Does it validate findings by exploiting them, or just flag possibilities? Who reviews the output before it reaches you? What can the agent do on its own, and how is it stopped from causing disruption? Where is your data processed and stored? Clear answers to those questions will tell you far more than any marketing claim about autonomy. Our guide to [what you are actually paying for when you buy a penetration test] covers the wider set of questions worth asking any provider.

Finally, always remember that every LLM comes with a warning that states that AI can get things wrong. You should always think long and hard about using AI penetration testing when you know that such disclaimers exist. 

Want testing that combines speed with real expertise?

Fortifi’s CREST-accredited consultants deliver penetration testing that goes beyond what automated tools can find on their own, from business logic flaws to complex attack chains, with every finding explained in terms your team and your auditors can act on. If you are weighing up AI pentesting tools against consultant-led testing, or want to know how the two could work together for your organisation, we will give you a straight answer. Book a call today.

Frequently asked questions

What is AI-led penetration testing in simple terms?

It is a penetration test in which an AI agent does much of the hands-on work, which means discovering targets, probing for weaknesses, attempting exploits and reporting what it finds. Unlike a traditional scanner, the agent reasons about results and changes its approach as it goes, in a way that resembles how a human tester works.

Is AI pentesting the same as automated penetration testing?

They overlap, but they are not identical. Automated penetration testing traditionally meant tools following fixed scripts and checks. AI pentesting uses language models that can plan, adapt and chain steps together, which makes it considerably more flexible than older automation, although the two terms are often used interchangeably in marketing.

Can AI replace human penetration testers?

Not yet, and probably not entirely. AI agents are very effective at broad, repetitive testing and at finding well-understood vulnerability types quickly. They remain weaker at business logic flaws, contextual risk rating and the judgement needed on sensitive systems, which is why most credible providers still keep experienced humans reviewing and directing the work.

Is AI-led penetration testing safe to run on live systems?

It can be, provided the scope is tightly defined and the platform has strong controls over what the agent is permitted to do. Because an AI agent can take real actions against your systems, you should understand its limits before it starts, and be particularly cautious with fragile internal networks and operational technology.

Will an AI pentest satisfy compliance requirements?

Often not on its own. Many frameworks, contracts and cyber insurance policies expect testing by qualified, accredited professionals, and an AI-only report may not meet that bar. AI-led testing works best as a supplement that improves coverage between accredited, consultant-led engagements.