invocation.io

Free tool · 3 minutes

Is your AI-built app safe to launch?

AI-built apps fail in patterns, and these ten questions cover nearly everything serious I find when reviewing them. Answer honestly. "Not sure" is a valid answer, and it counts against you the way it would in production.

Question 1 of 10 · 0 answered

1. Does your server calculate prices itself, instead of trusting the amount sent from the browser?

How to check: ask whoever built it where the checkout total comes from. If any amount travels from the browser into the payment call, the answer is no.

2. If your payment provider sends the same notification twice, does your app handle it only once?

How to check: search the code for the webhook handler and look for a stored event ID check. Zero matches means no.

3. If a regular user sends an admin request directly to your API, does it get rejected?

How to check: log in as a normal user, open the browser's network tab, and replay an admin request or another user's record ID. Rejection is the right result.

4. Is your code repository free of API keys, passwords, and other secrets?

How to check: run a secret scanner (gitleaks or trufflehog) across the full git history, not just the current files.

5. Could you rebuild your database structure from the code alone?

How to check: run the migrations against an empty database and start the app. If it will not boot, the answer is no.

6. When something fails during a payment or signup, do you find out about it?

How to check: see whether any error-tracking service is wired up, and whether failures show the user an honest error instead of fake success.

7. Do you have automated tests that actually fail when something breaks?

How to check: change a price calculation on a branch and run the tests. Green across the board means they assert nothing.

8. Are the libraries your app is built on current and free of known vulnerabilities?

How to check: run the ecosystem audit tool (npm audit, pip-audit, bundler-audit) and note anything rated critical.

9. Is there anything stopping a bot from hammering your signup or login thousands of times?

How to check: try the login endpoint 50 times in a minute from one connection. If nothing pushes back, the answer is no.

10. Do your API responses and logs stay free of passwords, keys, and personal data?

How to check: open the JSON your three main endpoints return and read one day of logs. Look for fields the app never needed to send.

Answer all ten questions to see your verdict. 'Not sure' is a valid answer.

Scoring runs entirely in your browser. Nothing is stored or sent unless you email yourself the results. The questions mirror the 10-point AI-app safety checklist, which explains how to verify each one.