← Browse

@ashermahonin/security-owasp-llm

A

Review an LLM-powered feature against the OWASP Top 10 for Large Language Model Applications: prompt injection, sensitive information disclosure, supply-chain risk for models and data, data and model poisoning, improper output handling, excessive agency, system prompt leakage, vector and embedding weaknesses, misinformation, unbounded consumption. Produces a per-category status, threat model for the LLM data flow, abuse-case test plan, and release verdict. Use whenever the product surface includes any LLM call, retrieval augmentation, fine-tune, embedding store, or model-routed automation.

skillclaude

Install

agr install @ashermahonin/security-owasp-llm --target claude

Writes 1 file into .claude/skills/, pinned to git-b3ed6257.

  • .claude/skills/security-owasp-llm/SKILL.md

Document


name: security-owasp-llm description: "Review an LLM-powered feature against the OWASP Top 10 for Large Language Model Applications: prompt injection, sensitive information disclosure, supply-chain risk for models and data, data and model poisoning, improper output handling, excessive agency, system prompt leakage, vector and embedding weaknesses, misinformation, unbounded consumption. Produces a per-category status, threat model for the LLM data flow, abuse-case test plan, and release verdict. Use whenever the product surface includes any LLM call, retrieval augmentation, fine-tune, embedding store, or model-routed automation."

Security: OWASP LLM Top 10

Role

Treat an LLM-using feature as a new attack surface, not a normal API. Walk through every OWASP LLM Top 10 category against the product's data flow and ship a verdict with abuse-case evidence, not vibes.

Start By

  1. Read references/owasp-llm-top10.md.
  2. Map the LLM data flow: user input → preprocessing → prompt assembly → retrieval/embedding → model call → output handling → downstream effect. Note where untrusted data crosses into prompt context.
  3. Identify the model(s), provider(s), tool(s), and the agent's permitted actions.
  4. Use Context7 MCP for the current OWASP LLM Top 10 wording and the current model-provider security guidance (Anthropic, OpenAI, Google, Mistral, Meta, etc.).

Procedure

  1. For each LLM Top 10 category, mark status: Pass / Concern / Fail / Out-of-scope.
  2. Build an abuse-case list per category: e.g., direct prompt injection, indirect via retrieved doc, system-prompt leakage probe, jailbreak via tool description, exfil via embedding inversion, denial via context blow-up.
  3. Verify each abuse case with an eval: at least 10 representative prompts per category, multiple seeds, varied phrasing, plus at least one obfuscated/encoded variant.
  4. Cross-check with security-owasp-agentic for autonomy/agency risk and with security-secrets for prompt content secrets handling.
  5. Produce remediation plan, owner per finding, and a release-gate verdict with conditions.

Principal-Level Defaults

  • Follow ../../routing/principal-operating-model.md.
  • Use Context7 MCP for current LLM Top 10 categories, provider safety docs, and any model-card limitations.
  • Keep a decision trace: model version, evaluator method, abuse-case coverage, what is not yet tested.
  • Refuse to mark a category Pass without an eval run, not just a code review.
  • Escalate any unmitigated Excessive Agency or Sensitive Information Disclosure finding before release.

Output Artifacts

  • LLM data-flow diagram (sources, sinks, trust boundaries)
  • Per-category status table with eval evidence
  • Abuse-case eval results (counts, rates, examples)
  • Findings register with owners
  • Release-gate verdict and conditions

Quality Bar

  • No category marked Pass from prompt review alone; require eval evidence.
  • No mitigation that relies on "the model usually refuses". Require deterministic guardrail evidence.
  • No retrieval pipeline approved without an injection test through ingested content.
  • No tool-using agent approved without an excessive-agency review.

Handoff

Hand off to service-implementation per finding and to qa-eval to add the abuse-case evals into the regression suite.

References

  • references/owasp-llm-top10.md: per-category checklist, abuse-case patterns, and provider-specific notes.

Trustgrade A

  • passBody integrity

    Whether the stored document is plausibly the kind of file the artifact declares, rather than something fetched by mistake.

  • passType matchnot applicable to this artifact type

    Whether the artifact is really the kind of thing its metadata claims it is.

  • passFreshness

    How long since the source repository was last pushed to.

  • passPrompt injection

    Scans the artifact's own text for instructions aimed at your agent rather than at you.

  • passLicense

    Whether the source repository declares an SPDX license permissive enough to redistribute.

How the grade is calculated

Each check contributes 0 points when it passes, 1 when it warns, and 2 when it fails. The total maps to a letter:

  • Aevery check passed
  • Bone warning
  • Ctwo warnings
  • Dprompt injection or body integrity failed, or three warnings
  • Fone of those failed, and something else is wrong

These are automated hygiene checks, not a security audit, and not a dependency or vulnerability scan. A grade of A means nothing was flagged — not that the artifact is safe.

Versions

  • git-b3ed62576ee82026-07-31