AI-built apps do not fail randomly — they fail in patterns. I review AI-generated and vibe-coded codebases professionally, and the same ten defects account for nearly everything serious I find. Here is the checklist, with a fast way to verify each item yourself. It works in the demo and breaks in production is not a paradox; it is the signature of exactly these ten gaps.
1. Does the server trust prices from the browser?
Find where an order or checkout total is created. If any amount, price, or discount arrives from the client and is charged without being recomputed from the database, anyone can pay whatever they type. This is the single most common revenue-critical defect in AI-built commerce flows.
Check it: Grep the API layer for amount, total, or price fields read from request params and passed toward the payment call.
2. Are webhooks idempotent?
Payment providers retry webhooks by design. If your handler creates a record per delivery with no event-ID guard, retries create duplicate orders, double emails, and double fulfillment.
Check it: Search the codebase for the webhook handler and look for a stored event ID or idempotency key. Zero occurrences means the answer is no.
3. Is authorization enforced by the API, or just drawn by the UI?
Hidden buttons are not security. Every state-changing endpoint must check ownership and role server-side. AI assistants reliably generate the UI check and reliably skip the API one.
Check it: Log in as a regular user, open the network tab, and replay an admin request or another user's object ID directly. If it succeeds, so will an attacker.
4. Are secrets committed to the repository?
API keys, service-account files, and database URLs pasted into code during a vibe session tend to stay there — and survive in git history even after deletion.
Check it: Run a secret scanner (gitleaks or trufflehog) across the full history, not just the current tree. Rotate anything found; deleting the file does not un-leak the key.
5. Can the database be rebuilt from the repo?
If schema changes were applied live by the assistant rather than through migrations, your repository cannot reproduce your database — which means no reliable staging, no clean onboarding, and a fragile disaster-recovery story.
Check it: Run the migrations against an empty database and boot the app. If it cannot, your production data is the only copy of your schema.
6. Where do errors go?
AI-generated code loves the silent catch block: failures logged nowhere, users shown success, data quietly lost. Payment and signup paths deserve special suspicion.
Check it: Grep for empty or log-only catch/rescue blocks in the critical flows, and check whether any error-tracking service is wired at all.
7. Do tests exist — and do they assert anything?
Many AI codebases ship tests that run green while asserting nothing meaningful, which is worse than no tests: it manufactures false confidence in every future change.
Check it: Open the three newest test files and read the assertions. Then change a price calculation on a branch and see if any test fails.
8. How old and how abandoned are the dependencies?
Assistants pin whatever version their training favored — sometimes end-of-life frameworks with known CVEs, sometimes packages abandoned years ago in the critical path.
Check it: Run the ecosystem audit tool (npm audit, bundler-audit, pip-audit) and check the framework's own support status.
9. Is there any rate limiting or abuse control?
Signup, login, password reset, and anything that sends email or SMS will be found by bots within days of launch. AI scaffolds almost never include throttling unless asked.
Check it: Hit the login endpoint 50 times in a minute from one IP. If nothing pushes back, budget for the spam wave.
10. What leaks through logs and responses?
Check what the API returns and what gets logged: full user objects with password hashes, stack traces with connection strings, PII in analytics events. Exposure grows quietly with every AI-generated endpoint that serializes a whole record.
Check it: Inspect the JSON of your three main API endpoints for fields the client never needed, and read one day of logs for anything you would not want subpoenaed.
Scoring it honestly
Zero to two failures: unusual — harden what failed and ship. Three to five: normal for an AI-built app; fix items 1–4 before taking real users' money. Six or more: stop adding features — every new feature multiplies the surface built on these gaps, and the continue-or-restart question deserves a real answer before more money goes in.
You can hand this to me instead. The AI-Build Rescue Review runs this checklist and everything behind it against your actual codebase in one working day: $1,450 fixed, evidence-graded findings, a continue-or-restart verdict, and your first ten fixes in order. The sample report shows exactly what comes back.
Want your score first? The free AI app safety check asks these ten questions interactively and hands you a verdict in three minutes.
Frequently asked questions
Is AI-generated code safe to use in production?
It can be — after review. AI-built codebases fail in patterns: they trust the client, skip idempotency, enforce permissions in the UI instead of the API, and handle errors by swallowing them. None of these are exotic; all of them are invisible in normal use, which is why apps that work perfectly in demos still fail this checklist.
What is the most common security issue in vibe-coded apps?
UI-only authorization. The interface hides admin buttons from regular users, but the API endpoints behind them accept any logged-in session. Anyone who opens the browser's network tab can replay those calls directly. It ranks first because it combines high frequency with high impact: full data exposure, trivially exploitable.
Can I run this checklist myself without being an engineer?
Roughly half of it, yes — the checks include fast greps and browser-level tests a motivated founder can run. The other half needs someone who can read code. Running your half first is still worth it: if the self-serve checks already fail, you know the answer before paying anyone.
Written by Andrej Dragojevic, Stripe Certified Professional Billing Architect.