AI-Generated Code Security Risks: Secrets and Slopsquatting

AI-generated code can pass a happy-path demo while leaking credentials, trusting hostile input or importing a package that nobody on your team has approved. The useful question is not whether a model writes secure code in general. It is whether this particular change preserves your application's security boundaries.

Treat generated output as an untrusted contribution. Give it the same review, testing and dependency controls you would apply to unfamiliar code from outside the team, then add checks for the failure patterns assistants make especially easy to miss.

Map the boundaries before requesting code

Start with a threat-focused brief. Identify which inputs are controlled by users, which operations require authorisation, where secrets enter the process and which services can be called. For a document upload feature, that includes filenames, file contents, storage paths, download permissions and background processing. A function signature alone does not communicate those constraints.

Tell the assistant what it must preserve: server-side access checks, parameterised database access, output encoding and the existing dependency policy. Ask it to list assumptions before making edits. An assumption such as “all callers are administrators” deserves scrutiny even if the resulting code looks tidy.

Use OWASP's application security verification standard to choose requirements appropriate to the application. You do not need to paste an entire standard into a prompt. Select the controls relevant to the changed boundary and translate them into reviewable acceptance criteria.

Limit the initial change to one vertical slice. Smaller patches make security-sensitive data flows easier to trace from entry point to storage or response, and make it harder for unrelated refactors to conceal a missing check.

Keep secrets out of prompts and patches

Credentials can leak through context as well as source files. Terminal output, environment dumps, support tickets and copied request headers may contain tokens. Before sending material to an assistant, replace sensitive values with clearly fake placeholders and remove unnecessary customer records. Check the provider's retention and access settings rather than assuming a local editor means local processing.

Require environment-based configuration, but do not mistake it for complete secret protection. Environment variables can still appear in logs, browser bundles, crash reports or diagnostic endpoints. A generated frontend component must never receive a private service token merely because the value originated in an environment file.

Use the regex tester with synthetic examples to refine a narrow pattern for your organisation's token prefixes before adding it to a scanning rule. Do not paste real credentials into an online tester. Pattern matching supplements a dedicated secret scanner; it cannot recognise every credential or determine whether a key remains active.

If a real secret reaches a shared prompt, commit or log, follow your incident procedure: revoke or rotate it, investigate access and remove exposed copies where feasible. Deleting the line is not remediation. Record the exposure without repeating the secret in the ticket.

Review injection as a data-flow problem

Injection risk starts when untrusted data becomes executable meaning. Search generated code for SQL construction, shell execution, HTML insertion, template evaluation and dynamic file paths. Trace the values feeding those operations rather than searching only for suspicious function names.

For database access, prefer the project's established parameterised query interface. For command execution, avoid a shell when a direct library or argument-array API can perform the task. Escaping is context-dependent: a string safe for HTML text is not automatically safe for JavaScript, a URL or a database query.

A useful review prompt is deliberately narrow:

Review this patch without editing it.
Trace user-controlled input into database, shell, HTML and file operations.
For each concern, cite the changed location and an exploit precondition.
Separate confirmed defects from questions requiring application context.
Do not propose a new dependency.

The OWASP injection guidance provides background for those checks. Pair assistant suggestions with negative tests: another user's object identifier, an unexpected filename, malformed encoding or a value containing markup. Tests should assert the safe outcome, not simply that the application returns some error.

Remember authorisation separately. Parameterised SQL can safely execute a query that returns another customer's records. Verify the identity and resource relationship at the server boundary, including download and background-job paths.

Verify every proposed dependency

A plausible package name is not evidence that a dependency exists, is maintained or belongs to the expected author. Hallucinated packages are dangerous because an install command can turn a speculative suggestion into executable third-party code. Even an established package may be inappropriate for your trust or licensing requirements.

Before installation, inspect the official registry entry, linked repository, ownership, release history and installation instructions. Confirm the spelling against the project's own documentation. Review lifecycle scripts and transitive dependencies under your normal policy. Do not treat download counts or an assistant's explanation as a substitute for that inspection.

Prefer existing project dependencies or platform features when they meet the requirement. The AI coding assistants directory can help compare development tools, but choosing an assistant does not change your package approval process. Disable automatic installation where practical, or require approval before commands that modify dependencies run.

Commit the expected lockfile changes and inspect them. A one-line source edit accompanied by a large dependency graph deserves a clear explanation. Run the package manager's security checks, while recognising that advisory databases cannot identify every malicious or newly introduced package.

Make review and release evidence explicit

Ask for a security review record alongside the patch: changed trust boundaries, dependencies added, checks performed and unresolved assumptions. A statement that code is “secure” is not evidence. A named test with an observed result is much more useful, especially when another engineer can reproduce it.

Use the workflow in project instructions for AI agents to make safe commands and approval boundaries visible before work starts. Keep security requirements close to the relevant code, and ensure agents cannot silently weaken tests or configuration to make a change pass.

Use GitHub’s code security documentation to understand available repository checks. Run static analysis, secret scanning, dependency checks and targeted behavioural tests in your normal pipeline. Review the final diff after any automated repair. An assistant may remove a warning by deleting validation or broadening an exception handler, leaving the underlying risk worse than before.

Define release safeguards proportionate to impact: restricted rollout, monitoring for denied access or unexpected outbound requests, and a tested rollback path. Never log sensitive payloads merely to improve observability. For high-impact changes, require a reviewer who understands the application boundary rather than relying on another model's agreement.

Key takeaways

• Map trust boundaries before asking an assistant to implement a feature.
• Keep real credentials out of prompts, logs and browser-delivered code.
• Trace untrusted values into sensitive operations and test rejected inputs.
• Verify package identity and dependency changes before installation.
• Require reproducible security evidence and human approval for consequential releases.