@benavlabs/vibe-check
BAI optimizes for making your code work, not for making it safe.
Install
agr install @benavlabs/vibe-check --target codexWrites 1 file into AGENTS.md, pinned to git-46d43213.
- AGENTS.md
Document
Security Rules
These rules apply to all code generated in this project. They are non-negotiable.
Secrets
- NEVER put API keys, database credentials, or tokens in frontend code (anything under src/, app/, pages/, components/, public/)
- NEVER put secret keys in environment variables prefixed with NEXT_PUBLIC_, VITE_, or REACT_APP_ (these are bundled into the client)
- NEVER hardcode credentials in source files. Use environment variables loaded server-side only
- The .env file MUST be in .gitignore before the first commit. Verify this before creating any .env file
- Use .env.example with placeholder values only, never real credentials
Database
- Enable Row Level Security on EVERY Supabase table before deployment. Default policy: deny all. Write explicit policies scoped to auth.uid()
- NEVER set a Supabase RLS policy to
USING (true)orFOR ALLwithout a WHERE condition - Firebase Security Rules MUST require
request.auth != nulland scope access torequest.auth.uid - NEVER use
pickle.loads,pickle.load, or any deserialization on user-supplied data. Use JSON for all network data exchange
Authentication and Authorization
- EVERY API route that returns or modifies user data MUST have authentication middleware that runs BEFORE the handler, not inside it
- Unauthenticated requests to protected endpoints MUST return 401
- EVERY route that takes a resource ID MUST verify the authenticated user owns that resource:
current_user.id == resource.owner_id. This is a SEPARATE check from authentication - Admin endpoints MUST verify admin role and return 403 for non-admin users
- Session cookies MUST set
httpOnly: true,secure: true, andsameSite: 'lax'
Input and Output
- NEVER concatenate user input into SQL queries. ALWAYS use parameterized queries or ORM methods
- NEVER use
dangerouslySetInnerHTML,v-html, orinnerHTMLwith user-supplied content unless it is first sanitized with DOMPurify - ALL user input MUST be validated server-side. Client-side validation is for UX only
- File uploads MUST validate file type by reading magic bytes, not by checking the filename extension. Rename all uploads to UUIDs server-side. Store on a separate domain (S3, R2, GCS), never on the app origin
URL Fetching (SSRF Prevention)
- If the application fetches URLs provided by users (link previews, image proxies, URL validators), it MUST:
- Block all private/internal IP ranges: 127.0.0.0/8, 10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16, 169.254.0.0/16, ::1
- Allow only http and https schemes
- Resolve the hostname and check the IP BEFORE making the request
Security Headers
- Set these headers on ALL responses via a single global middleware:
Content-Security-Policy: default-src 'self'(adjust as needed for your app)Strict-Transport-Security: max-age=31536000; includeSubDomainsX-Frame-Options: DENYX-Content-Type-Options: nosniffReferrer-Policy: strict-origin-when-cross-origin
- In Express, use the
helmetpackage. In Next.js, set headers in next.config.js
CORS
- NEVER set CORS origin to
*(wildcard). Use an explicit allowlist of your actual domains - NEVER combine
origin: '*'withcredentials: true
Rate Limiting
- Login, registration, and password reset endpoints MUST have rate limiting (block after N failed attempts per IP within a time window)
- Do NOT trust X-Forwarded-For for rate limiting unless behind a trusted reverse proxy
Payments
- Stripe webhook endpoints MUST verify the signature using
stripe.Webhook.construct_event(or equivalent) on every request. Reject any request with an invalid or missing signature - Webhook handlers MUST track processed event IDs and skip duplicates (idempotency)
- Handle the full event lifecycle: payment_intent.succeeded, invoice.payment_failed, customer.subscription.deleted, customer.subscription.past_due
Error Handling
- NEVER expose stack traces, SQL errors, file paths, or library names in API responses
- Production error responses MUST return only generic messages:
{"error": "Something went wrong"} - Full error details go to server-side logs only
- Debug mode / development error pages MUST be disabled in production
Password Hashing
- ALWAYS use bcrypt, Argon2, or scrypt for password hashing
- NEVER use MD5, SHA-1, or plain SHA-256 for passwords
Dependencies
- Before installing any package, verify it exists on the official registry with a reasonable download count and history
- Pin exact versions in package.json / requirements.txt (no ^ or ~ in production)
- Commit lock files (package-lock.json, poetry.lock, yarn.lock)
Repository README
Describes benavlabs/vibe-check as a whole, which may contain artifacts other than this one. Where this artifact had no useful description of its own, its summary was taken from here.
How it works
Three layers, no overlap:
AGENTS.md— Security rules your AI tool reads while it writes code. Copy into your project root. Prevents vulnerabilities from being created.AI-CHECKLIST.md— A prompt that tells your AI to audit your entire project. It investigates your codebase, writes reports, creates fix plans, implements them, and verifies.manual-checklist.md— Tests you run yourself for the things AI can't catch.
Setup
Step 1: Copy the rules file into your project
Cursor, Copilot, Codex, Windsurf, or Gemini CLI:
cp AGENTS.md /path/to/your/project/AGENTS.md
Claude Code:
cp AGENTS.md /path/to/your/project/CLAUDE.md
Not sure? Copy both:
cp AGENTS.md /path/to/your/project/AGENTS.md
cp AGENTS.md /path/to/your/project/CLAUDE.md
Commit it. Your AI tool reads it automatically from now on.
Step 2: Run the AI security audit
Give AI-CHECKLIST.md to your AI coding assistant:
Run the security audit defined in AI-CHECKLIST.md against this project.
Go through each vulnerability one at a time.
It will investigate your codebase for each of the 17 vulnerability categories, create reports, write fix plans, implement fixes, and verify. Results go in a security/ folder in your project.
Step 3: Run the manual checks
Open manual-checklist.md and go through each test. These verify things like: can you access another user's data, is your .env exposed, can login be brute-forced.
If you only do 5, do the first 5. They cover what took down every company on the list.
What this covers
17 most common vulnerabilities found in vibe coded apps, based on documented breaches and security research:
| # | Vulnerability | Severity |
|---|---|---|
| 1 | Misconfigured database (no Row Level Security) | Critical |
| 2 | Unprotected API routes (no auth middleware) | Critical |
| 3 | Committed secrets (.env on GitHub) | Critical |
| 4 | Broken access control (IDOR) | Critical |
| 5 | Secret API keys in frontend code | Critical |
| 6 | Server-Side Request Forgery (SSRF) | High |
| 7 | Missing CSRF protection | High |
| 8 | Missing security headers | Medium |
| 9 | Wildcard CORS | High |
| 10 | No rate limiting | Medium |
| 11 | SQL injection | High |
| 12 | Cross-site scripting (XSS) | High |
| 13 | Unverified Stripe webhooks | High |
| 14 | Insecure file uploads | Medium |
| 15 | Verbose error messages | Low |
| 16 | Weak password hashing | Medium |
| 17 | Hallucinated packages (slopsquatting) | High |
Items 1–5 are what took down every real company on this list. None required a sophisticated attack.
⚠️ Warning: This will not make your app bulletproof. It covers the basics that have actually taken down vibe coded apps in production. When you have real traction and real user data, hire a pentester. No checklist replaces someone actively trying to break your stuff.
Skip the checklist entirely
This repo helps you fix what you already built. If you're starting something new, consider starting from a foundation that already passes all 17 checks out of the box.
FastroAI is a production-ready full-stack template (FastAPI + Astro + Stripe + PydanticAI) built by the same team behind this checklist. Auth with CSRF and rate limiting, Stripe webhooks with signature verification and idempotency, security headers, parameterized queries, production validation that blocks deployment if your secrets are weak or debug mode is on. 90%+ test coverage. You vibe-code the product on top of it, not the foundation.
Sources
Based on documented incidents and security research:
- Escape.tech — 5,600 vibe coded apps scanned (2,000+ vulnerabilities, 400+ exposed secrets)
- Tenzai — 5 major AI coding tools compared (69 vulnerabilities across 15 apps)
- Carnegie Mellon SusVibes — 61% functional, 10.5% secure
- Georgia Tech Vibe Security Radar — 74+ CVEs from AI-generated code
- Veracode — GenAI Code Security Report 2025
Contributing
Found something that should be on this list? Open a PR. Include what the vulnerability is, how to test for it, and how to fix it.
License
Contact
Benav Labs – benav.io github.com/benavlabs
Trustgrade B
- passBody integrity
Whether the stored document is plausibly the kind of file the artifact declares, rather than something fetched by mistake.
- passType matchnot applicable to this artifact type
Whether the artifact is really the kind of thing its metadata claims it is.
- passFreshness
How long since the source repository was last pushed to.
- warnPrompt injection1 hit(s): credential_access
Scans the artifact's own text for instructions aimed at your agent rather than at you.
- line 10 — References credentials, tokens, or key material
- passLicense
Whether the source repository declares an SPDX license permissive enough to redistribute.
How the grade is calculated
Each check contributes 0 points when it passes, 1 when it warns, and 2 when it fails. The total maps to a letter:
- Aevery check passed
- Bone warning
- Ctwo warnings
- Dprompt injection or body integrity failed, or three warnings
- Fone of those failed, and something else is wrong
These are automated hygiene checks, not a security audit, and not a dependency or vulnerability scan. A grade of A means nothing was flagged — not that the artifact is safe.
Versions
git-46d4321353bc2026-08-04