Your AI Writes Fast Code. But Is It Secure? A Hands-On Look at Scanning AI-Generated Output

I built a small Node.js API entirely with AI assistance. Then I scanned it for vulnerabilities.

The results were instructive — not because the AI wrote terrible code, but because the failure modes were so consistent and so predictable. Once you see the pattern, you can't unsee it.

This isn't a product review of any single scanner. I used several tools, including Vchk (a security scanner specifically built for AI-generated code), Snyk, and Semgrep. The goal was to understand what AI-generated code tends to get wrong and how to build scanning into a workflow that's actually useful rather than security theater.

---

The Project: A Simple REST API

I built a task management API in Node.js/Express. Features included:

• User registration and login with JWT authentication
• CRUD operations for tasks
• File attachment uploads
• Rate limiting on auth endpoints
• A simple admin endpoint for user management

I used an AI coding assistant throughout — it generated the route handlers, the middleware, the database queries, the file upload logic. I reviewed the code but deliberately did not fix anything before scanning. I wanted to see what a developer who trusted the output and moved on would end up with.

The assistant was fast. The code worked. It passed my manual tests. Then I ran the scanners.

---

What Vchk Found

Vchk is purpose-built for AI-generated code and focuses on patterns that AI models tend to produce due to how they're trained — generating plausible-looking code from patterns in their training data, which includes a lot of insecure legacy code.

Insecure JWT Implementation

The AI generated this validation pattern:

const decoded = jwt.verify(token, process.env.JWT_SECRET);
if (decoded.role === 'admin') {
  // allow admin access
}

The problem: jwt.verify() throws on invalid tokens, but the code had no try/catch around it. A malformed token would crash the middleware and expose a 500 error. More concerning, the role check was trusting the JWT payload without verifying that the JWT was actually signed with the expected algorithm.

If an attacker sent a JWT with { "alg": "none" }, some older JWT library versions would accept it. The fix is to explicitly specify the algorithm: jwt.verify(token, secret, { algorithms: ['HS256'] }).

Dependency Confusion Risk

The generated package.json included several dependencies with generic names — utilities the AI had referenced that sounded plausible but weren't from the official maintainers I expected. Two of the packages were real but had not been updated in four years and had known vulnerabilities. One was a package that didn't exist under that name in the npm registry at all — a dependency confusion risk if someone published a malicious package under that name.

Hardcoded Fallback Secrets

This one is subtle and I see it constantly in AI-generated code:

const secret = process.env.JWT_SECRET || 'default-secret-key';

The AI added this as a "helpful" fallback for development. In practice, if the environment variable is never set in production — a misconfiguration that absolutely happens — the app silently uses a predictable secret. Every JWT in the system is now forgeable by anyone who reads the source code on GitHub.

The correct pattern is to fail loudly:

const secret = process.env.JWT_SECRET;
if (!secret) {
  throw new Error('JWT_SECRET environment variable is required');
}

---

What Snyk Found

Snyk focused more on dependency vulnerabilities and infrastructure configuration.

Outdated Dependencies with Known CVEs

Three of my direct dependencies had known CVEs. The AI had generated a package.json with specific version pins — reasonable practice — but those pins pointed to versions that were current several months ago, not today. npm audit would have caught these too, but Snyk provided more context on the severity and fix guidance.

SQL Injection in One Route

One of the query handlers used string concatenation to build a query:

const query = `SELECT * FROM tasks WHERE title LIKE '%${searchTerm}%'`;

The AI was inconsistent. Most of its database queries used parameterized queries correctly. This one didn't. It was in a "search" feature that got added mid-session when I prompted for it, and the AI produced a faster but unsafe implementation.

Parameterized version:

const query = 'SELECT * FROM tasks WHERE title ILIKE $1';
const values = [`%${searchTerm}%`];

---

What Semgrep Found

Semgrep with the default Node.js/Express ruleset flagged several patterns:

Missing helmet Middleware

The AI-generated Express setup didn't include helmet. Helmet sets a collection of security-relevant HTTP headers — Content-Security-Policy, X-Frame-Options, X-XSS-Protection, and others. The app would work fine without it, but it's trivially easy to add and represents a whole class of browser-level protections:

const helmet = require('helmet');
app.use(helmet());

Unrestricted File Upload

The file attachment feature accepted any file type and stored files with their original client-provided filenames:

const upload = multer({ dest: 'uploads/' });

Two problems: the filename comes from the client (path traversal risk if you reconstruct file paths from it) and there's no MIME type validation. A user could upload a PHP file and if the uploads directory is web-accessible, potentially execute it.

The correct approach: validate MIME types server-side (not just extensions), generate your own filenames (UUID), and never serve uploaded files from a path that's handled as executable code.

---

The Patterns AI Gets Wrong, Consistently

After running these scans and reading the results, some failure patterns stand out:

Helpful defaults that are insecure defaults. AI assistants optimize for "the code works." || 'default-secret' makes development easier. { dest: 'uploads/' } requires fewer lines than a fully configured multer instance. These patterns are optimized for the happy path, not adversarial conditions.

Inconsistency under session drift. Early in a conversation, the AI used parameterized queries. By the fifteenth prompt, it didn't. The model doesn't maintain a consistent security posture across a long session — it just tries to satisfy the most recent prompt.

Training data bias toward legacy patterns. A lot of code on the internet uses JWT without algorithm pinning. A lot of tutorials hardcode development secrets. AI models reproduce these patterns because they're common, not because they're secure.

Missing the invisible requirements. Security headers, rate limiting on sensitive endpoints, input validation beyond the happy path — these weren't in my prompts, so they weren't in the output. The AI gives you what you asked for, and security is often what you forgot to ask for.

---

How to Build This Into Your Workflow

Running a scanner after the fact is useful for audits but misses the point. The goal is to not write the vulnerabilities in the first place.

Add scanning to your CI pipeline. Every PR should run npm audit, Snyk, and at least one SAST tool. Make failures block the build. This catches dependency vulnerabilities and common patterns before they reach production.

Write security requirements into your prompts. If you're generating an authentication flow, say: "Use parameterized queries, never hardcode secrets or use fallback defaults, include algorithm pinning on JWT verification, fail loudly if required environment variables are missing." The AI will follow explicit instructions.

Review AI output with a security lens, not just a functional lens. The question isn't only "does this work?" It's "what happens if the input is malicious? What happens if this environment variable isn't set? What happens if this file is a PHP script pretending to be an image?"

Use Vchk and similar tools as a second reader. They're not a substitute for understanding security, but they catch the patterns you're too close to the code to see.

---

The Honest Conclusion

AI-generated code is not uniquely insecure. Humans write SQL injection vulnerabilities and hardcoded secrets too. But the failure modes are systematic in ways that matter: the same patterns show up repeatedly because the models are producing code that looks like code they've seen, and a lot of the code they've seen is insecure.

The solution isn't to distrust AI-generated code. It's to verify it the same way you'd verify code from a junior developer who writes fast and tests manually: automated scanning, code review with a security mindset, and a clear set of requirements that include security properties, not just features.

Your AI assistant is a fast junior developer. You're still the senior engineer who ships it to production.

---

Shane is the founder of Grizzly Peak Software — a technical resource hub for software engineers, written from a cabin in Caswell Lakes, Alaska.