Saucebox Registry
← Browse

@codagent-ai/capture-eval-issues

A

Capture noteworthy review violations for the eval framework. Use when validator-run finds review failures — judges violations and saves notable ones to evals/inventory.yml.

skillclaude

Install

agr install @codagent-ai/capture-eval-issues --target claude

Writes 2 files into .claude/skills/, pinned to git-29215f97.

  • .claude/skills/capture-eval-issues/SKILL.md
  • .claude/skills/capture-eval-issues/judge-prompt.md

Document

name: capture-eval-issues description: >- Capture noteworthy review violations for the eval framework. Use when validator-run finds review failures — judges violations and saves notable ones to evals/inventory.yml. disable-model-invocation: false allowed-tools: Bash, Task, Read

/capture-eval-issues

Capture noteworthy review violations into the eval inventory.

Input

This skill receives one or more review JSON file paths as arguments (space-separated). Example: /capture-eval-issues validator_logs/review_src_claude@1.0.json validator_logs/review_src_gemini@2.0.json

Procedure

  1. Read judge-prompt.md from this skill's directory
  2. Spawn a subagent to judge and capture violations:
    • Task tool: Task with subagent_type="general-purpose", model="sonnet", prompt= judge-prompt content + "\n\nReview JSON files: <file paths>\n\nSkill directory: <this skill's directory path>"
    • The subagent handles everything: reading files, judging, calling the append script
  3. Report the subagent's capture summary (the CAPTURED: line)

Trustgrade A

  • passBody integrity

    Whether the stored document is plausibly the kind of file the artifact declares, rather than something fetched by mistake.

  • passType matchnot applicable to this artifact type

    Whether the artifact is really the kind of thing its metadata claims it is.

  • passFreshness

    How long since the source repository was last pushed to.

  • passPrompt injection

    Scans the artifact's own text for instructions aimed at your agent rather than at you.

  • passLicense

    Whether the source repository declares an SPDX license permissive enough to redistribute.

How the grade is calculated

Each check contributes 0 points when it passes, 1 when it warns, and 2 when it fails. The total maps to a letter:

  • Aevery check passed
  • Bone warning
  • Ctwo warnings
  • Dprompt injection or body integrity failed, or three warnings
  • Fone of those failed, and something else is wrong

These are automated hygiene checks, not a security audit, and not a dependency or vulnerability scan. A grade of A means nothing was flagged — not that the artifact is safe.

Versions

  • git-29215f97cbbd2026-07-31