TL;DR

• A shadow AI audit is a structured, time-bounded assessment of every AI model, agent, integration, and workflow operating in your environment, documented or not.

• The five steps are: map AI asset classes, identify data flows per asset, assess blast radius, run adversarial testing on the highest-risk deployments, and establish a living inventory to sustain the findings.

• The audit is a starting point, not a recurring control. Ongoing detection, covered in the shadow AI detection guide, is what happens after the audit establishes a baseline.

• Most organizations find three to ten times more AI integrations during a structured audit than their self-reported inventory contains. Budget for surprises.

A shadow AI audit is not the same as ongoing shadow AI monitoring. Monitoring is continuous and automated; it catches new integrations as they appear. An audit is a structured, time-bounded exercise with a specific output: a complete picture of your current AI footprint, risk-classified and prioritized for remediation.

The audit answers three questions that monitoring alone cannot: What AI is already running that we do not know about? What data does each integration touch, and what can it do with that data? Which integrations need immediate attention, and in what order?

Most security teams that run a first audit are surprised by the results. The gap between the declared AI inventory and the actual AI footprint is consistently large, and the highest-risk integrations are usually not the ones that were on anyone's radar. The five-step process below is designed to close that gap systematically rather than leaving it to chance.

Step 1: Map your AI asset classes

The first step is enumeration: find every AI integration in the environment across all five asset classes. This is the most labor-intensive step because no single discovery method covers all of them. The shadow AI detection guide covers each method in detail; the audit uses all five in sequence.

Models and fine-tunes. This includes base models accessed via API, fine-tuned models derived from approved base models, locally running models (via Ollama, LM Studio, or similar), and models embedded in internal applications. Fine-tuned models are a consistent blind spot: organizations often treat the original model approval as covering derived models, but a fine-tuned model is a distinct artifact with potentially different capability boundaries and safety properties.

Agents and agentic workflows. Any AI integration that takes actions rather than just generating outputs: LangGraph, CrewAI, or AutoGen workflows, AI agents with tool access, scheduled pipelines that use LLMs for automated decision-making, and browser-use or computer-use agents that operate on behalf of users. Agentic integrations require closer scrutiny than generative-only integrations because their blast radius extends beyond the data they can read to the actions they can take.

Integration layer. API calls to external model providers embedded in internal tools, scripts, and applications. These are typically developer-built and live in code repositories. Code scanning with tools like Trufflehog or GitLeaks identifies them; the audit should cover all internal repositories, including personal developer repos if your policy permits.

MCP server connections. Model Context Protocol servers extend agents with tool access to file systems, databases, APIs, and external services. MCP connections are frequently set up without formal approval and are not visible through standard monitoring. MCP security is an active research area; MCP servers with access to production systems or sensitive data should be treated as high-priority audit items regardless of how they were originally deployed.

SaaS-embedded AI. AI features within approved SaaS products: Copilot in Microsoft 365, Einstein AI in Salesforce, Notion AI, Zendesk AI, Linear AI-assisted features, and equivalents. These are frequently overlooked because the underlying SaaS product was approved and employees treat AI features as part of the product. For audit purposes, each SaaS AI layer is a distinct integration that processes internal data using model infrastructure the organization has not separately evaluated.

The output of Step 1 is a raw inventory: every AI integration found, grouped by asset class, with owner, deployment date (if known), and data access scope. Do not attempt to risk-classify during enumeration. Classify in Step 3.

Step 2: Identify data flows per asset

For each integration in the raw inventory, document what data it can access, what data it receives as input, and where its outputs go. This step converts the inventory from a list of integrations into a map of data movement.

The three questions to answer for each integration:

What data can this integration access? An AI coding assistant has access to whatever files are open in the developer's IDE and whatever context is in the local codebase. An MCP server connected to a database has access to every table in that database. A Slack bot with message read permissions has access to every message in every channel it has joined. The access scope is the ceiling on blast radius; you need to know it before you can assess risk.

What data does it actually receive as input? Access scope and actual input are often different. A model with access to an entire database may in practice only receive queries about a specific table. Actual input patterns matter for assessing data exfiltration risk, because the exfiltration risk corresponds to what gets submitted to the model, not what the model theoretically has access to.

Where do outputs go? Generated outputs that are displayed to a single user carry different risk than outputs that are automatically written to a shared document, sent as an email, or written back to a database. Agentic outputs that trigger downstream actions carry the highest risk because the AI's output directly controls system behavior.

Document the data classification level of each input source using your existing data classification schema. An integration that processes only publicly available information is categorically different from one that processes customer PII, source code, or financial records. The data classification drives the risk tier in Step 3.

Step 3: Assess blast radius

Blast radius is the scope of potential damage if an integration is compromised, misconfigured, or manipulated through adversarial inputs. It is a function of three variables: data access (what data can be exfiltrated), action scope (what actions can be taken), and propagation (whether the integration's outputs influence other systems).

A generative AI integration that produces text displayed to a single internal user has a limited blast radius. A misconfigured agentic workflow with access to production APIs, a write-capable database connection, and automated output delivery has a blast radius that could affect live systems at scale.

Assign each integration a risk tier using the following framework:

Tier 1 (Critical): Agentic integrations with access to production systems, customer data, financial records, or authentication infrastructure. MCP servers with write access to sensitive data stores. AI integrations with external communication capabilities (email sending, API calls to external services). These require immediate security review regardless of whether they are authorized.

Tier 2 (High): Integrations that process sensitive internal data (source code, internal strategy documents, employee data) and transmit it to external model providers without a reviewed data processing agreement. Fine-tuned models trained on proprietary data where the training process has not been audited. AI integrations in customer-facing products that have not been adversarially tested.

Tier 3 (Medium): Authorized integrations that have not been reviewed under current policy, or authorized integrations where the approval pre-dates current security standards. SaaS-embedded AI features where the vendor's AI data handling has not been verified. Developer AI tooling with access to internal codebases.

Tier 4 (Low): AI integrations with access only to non-sensitive, non-proprietary data. Tools operating in sandboxed environments with no access to production systems or external services.

The blast radius assessment is the output Tier 1 and Tier 2 integrations need immediate action; Tier 3 integrations need a scheduled review; Tier 4 integrations need documentation in the inventory.

Step 4: Run adversarial testing on Tier 1 and Tier 2 integrations

Tier 1 and Tier 2 integrations need adversarial testing before they can be cleared for continued operation. An integration may be authorized, configured correctly, and processing the right data, and still be vulnerable to prompt injection, jailbreaking, or data exfiltration via adversarial inputs.

The test scope for a shadow AI audit differs from a full AI red teaming engagement. The goal is not comprehensive coverage across all OWASP LLM Top 10 categories; it is rapid identification of critical vulnerabilities that present immediate risk given what the integration can access and do.

For each Tier 1 and Tier 2 integration, test at minimum:

Prompt injection resistance. Can an attacker who controls content that the integration processes (document content, database records, external web content retrieved by the agent) inject instructions that override the system prompt and redirect the integration's behavior? Indirect prompt injection is the highest-risk variant for integrations that retrieve and process external content.

Data exfiltration paths. Can adversarial inputs cause the integration to include sensitive data in its outputs in ways that would not be caught by output filtering? This includes encoding attacks (Unicode smuggling, Base64 encoding of exfiltrated content) and indirect exfiltration through side channels.

Action boundary enforcement. For agentic integrations, test whether the agent can be manipulated into taking actions outside its intended scope. An agent designed to summarize documents should not be manipulable into sending emails or modifying database records, even if it technically has access to do so.

Repello's ARTEMIS automates this adversarial testing across 15M+ attack patterns, covering OWASP LLM Top 10, MITRE ATLAS, and NIST AI RMF categories. For a shadow AI audit context, ARTEMIS can be run against each Tier 1 and Tier 2 integration to produce a prioritized remediation report without requiring a full red team engagement timeline.

Document test results per integration: vulnerabilities found, severity, and remediation steps. Integrations with critical vulnerabilities should be suspended or restricted to non-sensitive data scopes pending remediation.

Step 5: Establish a living inventory

The audit produces a point-in-time snapshot. Step 5 converts that snapshot into a living inventory that stays current as the AI footprint evolves.

A living inventory requires three components: a data structure to hold it, a process to update it, and tooling to automate the updates.

The data structure. At minimum, each inventory record should capture: integration name and description, asset class, owner and team, deployment date, data access scope, data classification level, risk tier from Step 3, adversarial test results and date, current status (active, suspended, approved, under review), and last-verified date. An AI Bill of Materials format, aligned with NIST AI RMF and EU AI Act documentation requirements, is the recommended structure because it supports both internal governance and external compliance reporting.

The update process. New AI integrations should require a registration step before deployment, with risk tier assignment based on data access scope. Existing integrations should be re-reviewed when their access scope changes, when the underlying model changes, or when a new vulnerability class is discovered that affects their architecture. Quarterly manual reviews should validate the automated inventory against self-disclosure from teams.

The tooling. Manual inventory maintenance does not scale. Repello's AI Asset Inventory provides continuous automated discovery that populates and updates the inventory as new integrations appear, new code is committed, and new OAuth connections are authorized. Continuous discovery is the only approach that keeps the inventory current between manual audit cycles, because AI sprawl grows faster than quarterly review cadences can track.

The living inventory feeds directly into AI security posture management: the complete AI footprint, risk-classified and continuously updated, is the input every downstream security control requires.

What to do with integrations that fail the audit

Integrations found during the audit fall into four categories based on their authorization status and risk tier:

Unauthorized and Tier 1 or 2: Suspend immediately. Notify the owner, initiate a security review, and require remediation before resuming operation. Do not allow continued use pending review when the integration has access to sensitive data or production systems.

Unauthorized and Tier 3 or 4: Do not suspend automatically, but require the owner to submit the integration for formal approval within a defined window (30 days is reasonable). Integrations not submitted within the window are suspended at the deadline.

Authorized and Tier 1 or 2 with critical vulnerabilities: Apply runtime controls (restrict data access scope, add output filtering, disable high-risk capabilities) while remediation is in progress. Repello's ARGUS provides runtime guardrails that can be applied to reduce the effective blast radius of a vulnerable integration while a more complete fix is developed.

Authorized and Tier 3 or 4: Add to the review queue. No immediate action required, but the integration should be reviewed against current security standards within the next quarterly cycle.

Frequently asked questions

What is a shadow AI audit?

A shadow AI audit is a structured, time-bounded assessment of every AI model, agent, integration, and workflow operating in an organization's environment, regardless of whether it was formally approved or is in any official inventory. It produces a complete current-state picture of the AI footprint, risk-classified by data access scope and blast radius, with prioritized remediation actions for the highest-risk integrations.

How long does a shadow AI audit take?

For a mid-size organization (500 to 2,000 employees), a well-run audit using the 5-step process takes two to four weeks: roughly one week for enumeration across all five asset classes, one week for data flow mapping and blast radius assessment, and one to two weeks for adversarial testing of Tier 1 and Tier 2 integrations. Larger organizations with complex internal AI development activity take longer, primarily at the enumeration and testing stages.

What is the difference between a shadow AI audit and ongoing shadow AI monitoring?

An audit is a structured, time-bounded exercise that produces a complete inventory baseline and risk classification. Monitoring is continuous and automated, designed to catch new integrations as they appear after the baseline is established. The audit is the starting point; monitoring is what sustains the findings. Running monitoring without a prior audit means you are tracking changes from an incomplete baseline.

Should unauthorized AI integrations be immediately shut down?

Not always. The response should be proportional to the risk tier. Unauthorized Tier 1 and Tier 2 integrations (those with access to sensitive data or production systems) should be suspended pending review. Unauthorized Tier 3 and Tier 4 integrations can be given a window to complete formal approval before any suspension. Immediate shutdown of all unauthorized integrations, regardless of risk, creates operational disruption without corresponding risk reduction and typically drives future AI usage underground rather than through proper channels.

How do I get developer teams to participate in the audit?

Frame the audit as a registration process rather than an enforcement action. Developers who self-disclose integrations during the audit window should receive expedited review rather than automatic suspension. The goal is to bring the actual AI footprint into visibility, not to punish the teams who built useful tools without going through a slow procurement process. Post-audit, a streamlined AI tool registration process reduces the friction that creates shadow AI in the first place.

What frameworks should the audit documentation align with?

NIST AI RMF (Govern, Map, Measure, Manage functions), EU AI Act Article 9 (risk management documentation for high-risk AI systems), and MITRE ATLAS (for adversarial testing coverage). Aligning documentation with these frameworks means audit outputs can be reused directly for regulatory compliance reporting rather than requiring a separate documentation effort.

The audit is a one-time exercise with lasting value only if Step 5 is executed: the living inventory has to be maintained, automated, and integrated into the security program. An audit that produces a spreadsheet that ages in a shared drive is not a shadow AI audit; it is a shadow AI census with a known expiry date.

To see how Repello's AI Asset Inventory automates the discovery and tracking that makes Step 5 sustainable, book a demo or visit repello.ai/inventory.