Cisco Skill Scanner: What It Does, What It Misses, and When to Use Something Else

TL;DR: Cisco's open-source skill scanner (cisco-ai-defense/skill-scanner) is the most technically comprehensive AI skill security tool available today. It is also a CLI-first, developer-only tool requiring Python 3.10+, three separate API keys, and meaningful setup time. This post gives an honest breakdown of what it covers, where it falls short for different use cases, and what alternatives exist — including Repello's SkillCheck, which requires nothing to install and works in your browser.

What Cisco's Skill Scanner Actually Does

Released by Cisco AI Defense in early 2026 in response to the OpenClaw skill security crisis, cisco-ai-defense/skill-scanner is a Python-based static analysis tool for AI agent skill files. At time of writing it has over 900 GitHub stars and is actively maintained.

The scanner combines four analysis methods:

Static analysis — pattern-based detection using YAML signature rules and YARA rules. This catches known malicious patterns: specific prompt injection templates, recognized exfiltration instruction patterns, and common payload delivery signatures. Fast, works offline, no API key required for this layer alone.

Bytecode analysis — inspects any compiled or interpreted code files included in the skill zip for suspicious operations: subprocess spawning, network connections to non-standard endpoints, filesystem access outside expected scope.

Behavioral dataflow analysis — traces how data moves through the skill's instruction set. Identifies paths where user inputs or environment variables could be routed to external endpoints.

LLM-as-judge — uses a configurable LLM (defaults to Claude 3.5 Sonnet) to semantically evaluate the SKILL.md instruction set for intent and policy alignment. This is the layer most likely to catch novel prompt injection patterns that signature rules miss.

The scanner also supports a REST API for programmatic integration, custom policy authoring, and scan result export. For a security engineer who wants deep, scriptable analysis of skills at scale, it is the right tool.

What It Requires to Run

This is where the tool loses most non-developer users:

# Step 1: Ensure Python 3.10+
python --version

# Step 2: Install (uv recommended, pip also works)
uv pip install cisco-ai-skill-scanner

# Step 3: Set three environment variables
export SKILL_SCANNER_LLM_API_KEY="your_api_key"      # Required for LLM analysis layer
export SKILL_SCANNER_LLM_MODEL="claude-3-5-sonnet-20241022"
export VIRUSTOTAL_API_KEY="your_virustotal_api_key"   # Required for binary scanning
export AI_DEFENSE_API_KEY="your_aidefense_api_key"    # Required for Cisco AI Defense integration

# Step 4: Run a scan

# Step 1: Ensure Python 3.10+
python --version

# Step 2: Install (uv recommended, pip also works)
uv pip install cisco-ai-skill-scanner

# Step 3: Set three environment variables
export SKILL_SCANNER_LLM_API_KEY="your_api_key"      # Required for LLM analysis layer
export SKILL_SCANNER_LLM_MODEL="claude-3-5-sonnet-20241022"
export VIRUSTOTAL_API_KEY="your_virustotal_api_key"   # Required for binary scanning
export AI_DEFENSE_API_KEY="your_aidefense_api_key"    # Required for Cisco AI Defense integration

# Step 4: Run a scan

Three separate API key registrations. A Python environment at the right version. A package manager preference (uv vs pip). This is a realistic developer workflow — it is not accessible to the security practitioner, the IT administrator, or the developer who just wants to know if a skill is safe before installing it.

Without the LLM API key, the scanner runs only static and bytecode analysis — missing the layer most likely to catch sophisticated prompt injection. Without the VirusTotal key, binary scanning is disabled. The tool's full capability requires all three.

What Cisco's Scanner Does Not Cover

No web interface. Every scan requires command-line access. There is no upload-and-scan workflow for non-technical users.

OpenClaw-primary focus. The scanner's documentation, examples, and rule sets are built around OpenClaw's skill format. Cursor and OpenAI Codex skill formats are listed as supported, but coverage depth is demonstrably thinner — the YARA rules and signature sets reference OpenClaw-specific patterns heavily.

No threat intelligence catalogue. Cisco's scanner analyzes skills you already have. It does not show you a catalogue of skills that have already been scanned and assessed. There is no "browse scanned community skills" workflow.

No continuous monitoring. The scanner is a point-in-time tool. Running it once tells you about the skill as it exists today. Skills can be updated post-installation to introduce malicious behavior — the scanner has no mechanism to monitor for changes.

Alternatives and When to Use Each

SkillCheck by Repello AI

Best for: Skill end-users who need a fast answer before installation, non-technical security reviewers, teams without a Python environment or API key budget.

SkillCheck is a browser-based skill security scanner. Upload a zip, get a score out of 100 and a verdict (Safe / High / Critical) with a breakdown of detected attack patterns. Detection covers prompt injection, policy violations, payload delivery patterns, and environment variable exfiltration signatures.

No installation. No API keys. No Python. Works in any browser, across Claude Code, OpenClaw, Cursor, Windsurf, and other agent platforms.

SkillCheck also maintains a catalogue of community-submitted skills with their scan results — a threat intelligence layer that Cisco's tool does not have.

What SkillCheck does not do: Custom rule authoring, dataflow analysis at the depth of Cisco's behavioral engine, or REST API integration for programmatic workflows. If you need those capabilities, Cisco's scanner is the right tool for that layer.

Manual SKILL.md Audit

Best for: Individual developers, quick initial triage before running an automated scanner.

Read the SKILL.md file before anything else. Flags to look for: environment variable references in instruction text, instructions that append parameters to external URLs, trigger conditions that activate under common circumstances with hidden secondary effects, references to files included in the zip that are not documentation. Five minutes of manual review catches the most obvious issues and takes no tooling at all.

VirusTotal

Best for: Detecting known malware binaries included in skill zips.

VirusTotal scans file hashes against databases of known malware signatures. It will catch a skill zip that includes a binary already identified as malicious by any of VirusTotal's engine partners. It will not catch prompt injection in a SKILL.md file — VirusTotal has no concept of AI agent instruction semantics. It is a useful complement to AI-aware scanning, not a replacement. Cisco's skill scanner actually integrates VirusTotal optionally for the binary scanning layer.

Choosing the Right Tool for Your Situation

The two tools are not competing for the same user. Cisco's scanner is for security engineers who need scriptable, deep analysis. SkillCheck is for everyone else — the developer who wants a 60-second answer before installing a skill, the security team member who does not have a Python environment set up, the IT administrator reviewing a shortlist of approved community skills.

Both tools cover substantially the same threat surface. The right choice depends on your workflow, not on which tool is technically superior.

The Underlying Problem Neither Tool Fully Solves

Per-skill scanning — whether with Cisco's tool or SkillCheck — is a necessary first step. It is not sufficient for organizations running AI agents at scale.

The threat model for enterprise AI agent deployments extends beyond individual skill vetting:

• Skills that are safe at installation can be updated post-deployment to introduce malicious behavior

• A compromised skill in one developer's environment can pivot to shared credentials and internal systems through the agent's network access

• The agent's behavior at runtime — what it actually does with the access a skill requests — is not visible to pre-installation scanners

This is the gap that ARTEMIS fills: automated red teaming of the full agentic stack, not individual skills in isolation. ARGUS adds runtime monitoring — detecting when an installed skill begins behaving anomalously after deployment. If you are evaluating skill security tooling for an organization rather than for personal use, that is the conversation to have.

FAQ

Is Cisco's skill scanner free? Yes. Cisco's skill-scanner is open-source under a permissive license (GitHub: cisco-ai-defense/skill-scanner). The tool itself is free. Running it at full capability requires API keys for the LLM analysis layer (any LLM provider works, including Anthropic or OpenAI), VirusTotal, and optionally Cisco AI Defense — each of which may have their own costs at scale.

Does Cisco's skill scanner work on Claude Code skills? Cisco's scanner is primarily built and documented for OpenClaw skills. It officially supports "formats used by OpenAI Codex and Cursor Agent Skills" in addition to OpenClaw, but the rule sets and documentation focus on OpenClaw patterns. Claude Code skills use a similar SKILL.md-based format and can be scanned, but coverage may be less complete than for OpenClaw.

Which scanner is better for detecting prompt injection? It depends on what you're optimizing for. Cisco's LLM-as-judge layer can catch novel, custom-crafted injection patterns that static rules miss — but only if you've configured an LLM API key. Without it, Cisco falls back to signature-only analysis. SkillCheck covers documented real-world attack patterns from the ClawHavoc campaign, known marketplace threats, and pre-scanned catalogue entries — without any setup. For the large majority of community marketplace skills, SkillCheck's coverage is comparable. Cisco's edge is in bespoke enterprise environments where you're building custom rules for novel internal threat patterns.

Can I use both tools together? Yes, though they serve different users more than different stages. SkillCheck covers the gap Cisco doesn't: non-engineers, multi-platform environments (Claude Code, Cursor, Windsurf), and instant scans without API keys or Python setup. Cisco covers what SkillCheck doesn't: automated CI/CD pipeline integration, custom YARA rules, and deep behavioral dataflow analysis for teams that have the engineering capacity to operate it. They're not redundant — they solve different access problems.

What if neither scanner catches something and I get compromised? Pre-installation scanning reduces risk but cannot guarantee safety. Skills can be updated post-installation, scanners have coverage gaps for novel techniques, and skilled attackers actively test their payloads against known scanners. Runtime monitoring is the detection layer that catches what pre-installation scanning misses. ARGUS monitors agent behavior at runtime and flags anomalous actions — including behavior introduced by skills that passed pre-installation review.