LLM Pentesting: The 2026 Checklist, Methodology, and Best Tools

TL;DR: LLM pentesting is the systematic security testing of a large language model application, and it differs structurally from traditional application pentesting because the attack surface is natural language rather than structured input. It covers five surfaces: input/output, retrieval, agentic/tool-call, model layer, and runtime. A complete pentest needs a 5-phase methodology, a 30-item surface-specific checklist, and tools matched to your architecture — and it cannot stop at the API layer, because the application layer introduces attacks the endpoint never exposes. This guide covers all of it.

What LLM Pentesting Actually Tests#

Standard application pentesting targets a well-defined codebase. LLM pentesting is different: the attack surface shifts with every model update, every new tool integration, and every change to the system prompt. You are testing a probabilistic system, not a deterministic one.

The five surfaces every LLM pentest must cover:

1. Input/output layer: prompt injection, jailbreaking, encoding-based bypasses (Unicode variation selectors, Base64, token smuggling), output manipulation.
2. Retrieval layer: RAG poisoning, knowledge base manipulation, embedding space attacks, indirect injection through retrieved documents.
3. Tool-call and agentic layer: tool call hijacking, indirect prompt injection through external data sources, MCP server poisoning, cross-agent injection in multi-agent pipelines, privilege escalation through chained tool calls.
4. Model layer: fine-tuning backdoors, training data extraction, membership inference, adversarial examples against the base model.
5. Runtime layer: denial-of-wallet attacks through token exhaustion, context window manipulation, guardrail evasion, sycophancy exploitation, system prompt extraction.

Most 2024-era pentesting frameworks focused almost entirely on surface 1. Production attacks in 2025–2026 predominantly target surfaces 3 and 5.

How LLM Pentesting Differs From Traditional Application Pentesting#

This post covers the LLM layer specifically. For the broader engagement structure around it — reconnaissance, threat modelling, severity classification and reporting across any AI application — see our AI penetration testing methodology.

The core principles of application security still apply to LLM systems. Input validation, authentication boundary testing, API security assessment, and output handling verification all matter. But several things are structurally different.

The attack surface includes natural language. In a traditional web application, inputs are structured: form fields, API parameters, JSON payloads. In an LLM application, the primary input is free text, and that text can carry instructions as well as data. Prompt injection exploits exactly this conflation. The model cannot reliably distinguish between an instruction from the system operator and an instruction embedded in user-supplied content or a retrieved document. OWASP LLM01:2025 covers both direct and indirect variants. Indirect prompt injection through RAG pipelines is considered the more dangerous of the two because the attack surface is anything the model reads, not just what the user types. Repello's catalog of real-world prompt injection attack examples illustrates how diverse these attack paths are in production deployments.

Multi-turn conversation is a distinct test track. A single-request test misses a category of attacks that only manifest across conversation history. Many jailbreaking techniques work by gradually shifting the model's framing across multiple turns, sometimes called "many-shot" manipulation. Repello's deep-dive on AI jailbreaking techniques covers the full range of these approaches, including how attackers layer role-play scenarios and hypothetical framings across conversation turns to erode safety controls incrementally. Test suites that fire single requests and inspect responses will not catch these.

Tool integrations extend the blast radius. LLM applications with the ability to call external APIs, execute code, browse the web, or interact with file systems have a much larger attack surface than model-only deployments. OWASP LLM06:2025 (Excessive Agency) classifies this as a top-ten risk because a successful prompt injection in an agentic pipeline can trigger unauthorized transactions, exfiltrate data to external endpoints, or escalate permissions through a chain of individually plausible tool calls.

Output handling is part of the attack surface. If LLM outputs are rendered downstream, passed to other systems, or used to trigger actions, the content of those outputs becomes a security concern in its own right. OWASP LLM05:2025 (Improper Output Handling) addresses scenarios where unsanitized model outputs reach SQL interpreters, HTML renderers, or command execution environments, creating secondary injection paths.

MITRE ATLAS, which catalogs adversarial tactics and techniques against AI systems, currently documents 15 tactics and 66 techniques specific to AI attacks, the majority of which have no direct equivalent in the ATT&CK framework for traditional IT systems. A complete LLM pentest draws on both frameworks.

LLM Pentesting Methodology#

Phase 1: Threat Modeling#

Before touching the application, map what you're testing.

Identify trust boundaries: What data can the LLM access? What actions can it take? What happens downstream from its outputs?

Identify integration points: Is this a standalone chatbot, a RAG-backed assistant, an agentic system with tool access, or a multi-agent pipeline?

Identify the blast radius: If the model is fully compromised, what can an attacker read, write, or execute?

Agentic systems require separate threat modeling. A model with access to email, calendar, and file system tools has a fundamentally different attack surface than a Q&A chatbot. Research published on arXiv in 2025 found that multi-agent pipelines with tool access are 3.4x more susceptible to successful exploitation than single-model deployments.

Phase 2: Reconnaissance#

• Extract the system prompt (or infer its contents) through direct and indirect probing
• Map available tool calls and their permissions
• Identify the underlying model and version (relevant for known model-specific bypasses)
• Identify any pre/post processing filters; test with benign payloads to establish baseline behavior
• Check output format: streaming vs. batched affects certain injection timing attacks

Phase 3: Manual Probing#

This is where most of the real findings come from. Automated scanners miss context-specific vulnerabilities. A human tester understands the application's intent and can craft payloads that exploit the specific system prompt logic.

Prompt injection and jailbreaking

Test direct injection (malicious user input), indirect injection (malicious content in retrieved documents, tool outputs, or external data), and nested injection (instructions inside instructions). The OWASP LLM Top 10 classifies prompt injection as LLM01: the highest-priority risk for deployed language model applications.

Encoding bypasses

Test: Base64, ROT13, Unicode variation selectors (VS1–VS16), zero-width characters, homoglyphs, leetspeak, and BPE tokenization splits. The Repello AI Research Team demonstrated in original research that Unicode variation selectors can encode full attack payloads inside a single emoji character, bypassing commercial guardrail products including Azure Prompt Shield.

RAG and retrieval attacks

If the system uses retrieval-augmented generation, inject adversarial content into the knowledge base or test how the model handles retrieved documents containing conflicting or malicious instructions. RAG poisoning can cause aligned models to produce harmful or biased outputs at scale without any change to the model itself.

Agentic and tool-call attacks

For systems with tool access, test whether malicious instructions embedded in external content (emails, web pages, documents) can hijack tool calls. Research demonstrated zero-click data exfiltration from Google Drive through a single malicious email processed by an AI agent with no user interaction required. The MCP protocol introduces additional attack surface: poisoned tool descriptions, cross-server privilege escalation, and remote code execution through malicious schema definitions.

System prompt extraction

Attempt to extract the full system prompt through direct instruction, roleplay scenarios, and token prediction attacks. Leaked system prompts expose business logic, safety rule bypasses, and architectural details useful for further attacks.

Guardrail evasion

Breaking Meta's Prompt Guard documented a structural failure mode: guardrail classifiers run a different tokenizer than the underlying model, meaning payloads encoded in variation selectors are stripped before classification but processed by the model. Test whether the guardrail and the model see the same input.

Phase 4: Automated Scanning#

Manual testing finds the high-value logic-layer vulnerabilities. Automated scanning covers breadth: systematically checking hundreds of known injection patterns, jailbreak variants, and encoding bypasses that no human tester would enumerate manually.

Run automated scanning in parallel with manual work, not as a replacement for it. For teams that want to understand how automated and manual approaches compare structurally, breach and attack simulation for AI systems covers the tradeoffs in depth.

Phase 5: Reporting and Remediation#

For each finding, document:

• Attack vector and payload used
• Pre-conditions (e.g., specific RAG configuration, tool access required)
• Impact: data exfiltration, unauthorized action execution, guardrail bypass, etc.
• Reproduction steps
• Remediation recommendation

Prioritize by blast radius, not just exploitability. A prompt injection that leaks the system prompt is lower severity than one that hijacks a tool call with file system write access.

2026 LLM Pentesting Checklist#

Input / Output Layer#

• Direct prompt injection (user turn)
• Indirect prompt injection (retrieved documents, tool outputs)
• System prompt extraction
• Jailbreak via roleplay, persona assignment, and hypothetical framing
• Encoding bypass: Base64, ROT13, Unicode variation selectors, zero-width characters
• BPE tokenization split attacks
• Output manipulation (format override, data exfiltration via structured output)
• Multilingual bypass: test with non-Latin scripts if the system prompt is English-only

RAG / Retrieval Layer#

• Adversarial document injection into knowledge base
• Conflicting instruction injection via retrieved content
• Embedding space manipulation (if access to indexing pipeline)
• Cross-document injection chaining
• PII leakage through retrieval

Agentic / Tool-Call Layer#

• Tool call hijacking via indirect injection
• Privilege escalation through chained tool calls
• MCP server poisoning (malicious tool descriptions)
• Cross-agent injection in multi-agent pipelines
• Unauthorized action execution (file write, email send, API call)
• Zero-click exfiltration via agent-processed external content

Model Layer#

• Training data extraction (membership inference)
• Fine-tuning backdoor testing (if custom fine-tuned model)
• Adversarial example generation
• Sycophancy exploitation (overriding model outputs through social pressure)

Runtime / Guardrail Layer#

• Denial-of-wallet via token exhaustion
• Guardrail evasion (verify guardrail and model see the same tokenized input)
• Context window overflow / poisoning
• System prompt confidentiality (extraction and partial extraction)
• Rate limit and abuse control testing

Best Tools for LLM Pentesting in 2026#

Tools Comparison#

1. Garak#

Garak is NVIDIA's open-source LLM vulnerability scanner. It runs automated probes across 100+ attack categories including prompt injection, hallucination, data leakage, and jailbreaking, generating structured security reports. Best for broad coverage scans at the input/output layer.

GitHub: 4,200+ stars | Use case: Automated baseline scanning | Limitation: Coverage is shallow on agentic and tool-call surfaces; does not test RAG pipelines or production system context.

2. PyRIT (Python Risk Identification Toolkit)#

PyRIT is Microsoft's red teaming framework for generative AI. It supports multi-turn attack orchestration, allows custom attack strategies, and integrates with Azure OpenAI deployments. Well-suited for red teamers who want to script complex attack sequences.

GitHub: 2,100+ stars | Use case: Scripted multi-turn attack automation | Limitation: Requires significant engineering investment to configure for a specific application; coverage is only as good as the attack scripts you write.

3. PromptFuzz#

PromptFuzz applies fuzzing methodology to LLM inputs: systematically mutating prompts to discover bypasses that rule-based injection lists would miss. Particularly effective for finding encoding-based bypasses and model-specific edge cases.

GitHub: 800+ stars | Use case: Fuzzing and edge case discovery | Limitation: Output interpretation requires manual triage; does not produce actionable remediation guidance.

4. PurpleLlama / LlamaGuard#

Meta's PurpleLlama suite includes LlamaGuard (input/output classifier), CyberSecEval (security benchmark), and PromptGuard. Useful for evaluating model robustness against known attack categories and as a baseline comparator for custom guardrail configurations.

GitHub: 3,100+ stars | Use case: Benchmark evaluation, guardrail testing | Limitation: Benchmarks reflect known attack categories; novel or application-specific attack chains require manual testing on top.

5. PromptBench#

PromptBench is Microsoft Research's unified evaluation library for assessing LLM adversarial robustness. It provides standardized attack prompts across multiple adversarial categories (typos, character manipulation, word-level and sentence-level attacks) and outputs robustness scores for direct model comparison.

GitHub: 2,400+ stars | Use case: Adversarial robustness benchmarking across models | Limitation: Tests the model in isolation; does not account for system prompt logic, retrieval augmentation, or tool integrations that modify model behavior in production.

6. HarmBench#

HarmBench is a standardized safety evaluation benchmark from the Center for AI Safety, covering 400+ harmful behaviors across 7 functional categories. It provides a consistent attack methodology for comparing safety across models and fine-tunes, making it particularly useful for evaluating model regression after updates.

GitHub: 900+ stars | Use case: Cross-model safety comparison, fine-tune regression testing | Limitation: Covers safety-relevant harmful behaviors, not security vulnerabilities (prompt injection, tool hijacking, data exfiltration). Complements security testing; does not replace it.

7. Rebuff#

Rebuff is an open-source prompt injection detection framework from Protect AI. It combines an LLM-based classifier, a vector database of known injection patterns, and a canary token system that detects when injected instructions cause the model to leak internal context. Designed to integrate directly into application pipelines.

GitHub: 2,000+ stars | Use case: Prompt injection detection and pipeline-level filtering | Limitation: Detection-focused rather than attack-generation-focused; useful for building defenses but not for comprehensive pentest coverage.

8. Inspect AI#

Inspect AI is the evaluation framework developed by the UK AI Safety Institute (AISI). It provides a structured task and scoring architecture for evaluating LLM capabilities and safety properties across custom evaluation sets. Used internally by AISI for frontier model evaluations.

GitHub: 1,800+ stars | Use case: Structured safety evaluations, AISI-aligned testing methodology | Limitation: Framework-oriented; requires writing evaluation tasks rather than running out-of-box attack batteries.

9. LangChain / LangSmith#

LangChain is the most widely deployed open-source framework for building agentic LLM applications (100,000+ GitHub stars). LangSmith is the companion evaluation and observability platform: it traces every step in an agentic pipeline, logs tool calls, and supports structured evaluation runs. For teams already building with LangChain, LangSmith provides the closest approximation to continuous agentic security testing available in a commercial platform.

GitHub (LangChain): 100,000+ stars | Use case: Agentic pipeline evaluation and trace-level debugging | Limitation: LangSmith is a commercial SaaS product; evaluation quality depends on the test cases you configure. It covers observability and functional testing, not adversarial attack simulation.

10. ARTEMIS (Repello AI)#

ARTEMIS is Repello's automated red teaming engine built specifically for production LLM deployments. Unlike open-source scanners that test models in isolation, ARTEMIS tests the full application stack: RAG pipelines, tool integrations, and agentic workflows against the live system.

ARTEMIS covers all five attack surfaces in the methodology above, generates attacker-perspective reports with exploitation evidence, and includes ARTEMIS Browser Mode for red teaming AI agents that interact with web interfaces. For teams that need to pentest agentic systems and MCP integrations at scale and produce findings their engineering and security teams can act on, ARTEMIS replaces the manual coordination overhead of running Garak, PyRIT, and PromptFuzz separately.

Request a demo

How to Choose#

What attack surfaces does your deployment expose?

If your deployment is a standalone chatbot, Garak or PromptBench provide adequate automated coverage for input/output layer testing. If you are running an agentic system with RAG, tool integrations, or MCP connections, you need a framework that tests the full application stack against live production behavior. ARTEMIS is the only tool in this list that covers all five surfaces against production deployments; for bespoke attack scripting, PyRIT or Inspect AI can be layered on top.

Do you need continuous testing or a point-in-time assessment?

Open-source tools run as point-in-time scripts: you schedule them, interpret the output, and act on findings manually. ARTEMIS and LangSmith support continuous testing workflows where each model or system update triggers an automated re-assessment. For teams with frequent deployment cadences, continuous coverage is significantly more effective than scheduled point-in-time scans. The case for continuous AI red teaming covers the operational argument in detail.

Do you need scripting flexibility or out-of-box coverage?

PyRIT and Inspect AI are frameworks: powerful when customized, but they require engineering investment to configure for a specific application. Garak and ARTEMIS provide out-of-box coverage against a comprehensive attack library without custom scripting. Teams with dedicated AI security engineers often combine both approaches: Garak or ARTEMIS for breadth, PyRIT or Inspect AI for bespoke attack chains targeting application-specific logic.

Why API-Level Testing Is Not a Complete LLM Pentest#

Most LLM security testing, whether manual or automated, operates at the API layer. The test fires a prompt at an endpoint and inspects the response. This is the right starting point, but it is not a complete test.

The application layer introduces security behaviors the API layer does not expose. The way a web application renders LLM outputs, handles file uploads, manages session state, or enforces user role restrictions can materially change what attacks are possible. A prompt that fails against the raw API may succeed when it arrives through the application UI with an active user session attached. A file upload that appears harmless at the API level may trigger a prompt injection when the model processes the file's contents.

Agentic pipelines compound this further. An agent that can browse the web, execute code, or submit forms presents attack paths that require simulating the full execution environment, not just the model endpoint. Repello's research into zero-click exfiltration via MCP tool access in 11.ai shows exactly how this plays out in practice: a single malicious calendar invite can trigger unauthorized data exfiltration across connected tools without any direct user interaction. Testing an agentic system at the API level only is like running a web application pentest by querying the database directly: you are testing one component, not the system.

This is the gap that platforms like Repello address through their ARTEMIS testing framework. ARTEMIS browser mode navigates LLM applications exactly as a human attacker would: through the UI, across multi-turn sessions, through file upload paths, and through agentic tool chains that are structurally invisible to endpoint-only testing. Findings from those tests feed directly into ARGUS, Repello's runtime protection layer, which enforces adaptive guardrails in under 100 milliseconds without noticeable latency. The loop between testing and production protection closes: the pentest identifies what to guard against, and the guardrails enforce it at runtime.

Integrating ARTEMIS Into Your Pentest Workflow#

A standard LLM pentest workflow using ARTEMIS:

1. Scope definition: configure ARTEMIS with your application's endpoint, authentication, system prompt (if accessible), and tool manifest
2. Automated reconnaissance: ARTEMIS maps the attack surface, identifies the model, detects pre/post-processing filters, and enumerates tool capabilities
3. Automated attack execution: runs the full attack battery across all five surfaces, including agentic and MCP-specific attack vectors not covered by open-source tools
4. Manual tester augmentation: ARTEMIS surfaces the highest-priority findings; human testers dive deeper on logic-layer vulnerabilities specific to the application
5. Runtime hardening with ARGUS: findings from the pentest feed directly into ARGUS runtime policies, blocking the confirmed attack vectors in production

This workflow shifts LLM pentesting from a point-in-time engagement to a continuous security posture. Every model update or new tool integration can be re-tested against the same attack battery automatically.

Frequently Asked Questions#

What is LLM pentesting and how is it different from traditional pentesting?

LLM pentesting is the process of systematically testing a large language model application for security vulnerabilities. Unlike traditional application pentesting, which targets deterministic code paths, LLM pentesting must account for probabilistic model behavior, emergent capabilities, and attack surfaces that do not exist in conventional software: prompt injection, RAG poisoning, jailbreaking, and agentic tool-call hijacking. The OWASP LLM Top 10 provides the most widely used framework for classifying LLM vulnerabilities.

What are the most critical LLM vulnerabilities to test for in 2026?

Based on production incidents in 2025–2026, the highest-priority vectors are: indirect prompt injection through external content in agentic systems, MCP protocol attacks (tool poisoning and cross-server privilege escalation), guardrail evasion via tokenizer misalignment, and RAG knowledge base poisoning. Direct prompt injection at the chat interface, while still important, is now the most well-defended surface.

How long does an LLM pentest take?

A focused LLM pentest covering a single application typically takes 3–5 days for a skilled team. Agentic systems with multiple tool integrations take longer (5–10 days) because each tool integration introduces new attack paths that must be tested independently. Automated scanning with tools like ARTEMIS can compress the coverage phase from days to hours, freeing the team for manual logic-layer testing.

Can LLM pentesting be automated?

Partially. Automated scanners excel at breadth: covering known attack patterns systematically. Manual testers are necessary for context-specific attacks that exploit application logic, system prompt nuances, and novel attack chains. The best results come from using automation to establish baseline coverage and escalating to human testers for findings that require deeper exploitation.

How often should you pentest an LLM application?

At minimum, after every significant model update, new tool integration, or system prompt change. For production applications, continuous automated testing (as provided by ARTEMIS) is preferable to periodic point-in-time assessments. A model that passed a pentest in October may be exploitable by December if a new jailbreak technique targets its specific fine-tune.

What is the difference between LLM red teaming and LLM pentesting?

The terms are often used interchangeably but have a distinction in practice. LLM red teaming is adversarial simulation: a structured exercise to find the highest-impact vulnerabilities under realistic attack conditions. LLM pentesting is more systematic: working through a defined checklist of known vulnerability classes to verify coverage. Most enterprise security programs need both.

Run Your First LLM Pentest With ARTEMIS#

The checklist above covers what to test. ARTEMIS automates the execution: scanning all five attack surfaces against your live application, including RAG pipelines, tool integrations, and MCP-connected agents, and producing attacker-perspective reports your team can act on.

If you're running your first LLM pentest or scaling an existing red team program, book a demo with Repello to see ARTEMIS against your stack.