Software teams now write code alongside an assistant that suggests entire functions, pulls in packages on request, and connects to models hosted by third parties. Productivity has risen, and so has the surface that attackers and accidents can reach. A developer who pastes a stack trace into a chat window, accepts a generated snippet without reading it, or downloads a pretrained model from a public hub has made three security decisions in under a minute, usually without recognizing them as such.
Engineering and application security teams are left to reconcile the speed of these workflows with the controls they were built to apply. The sections below examine the risks in AI-generated code, the supply chain around models and dependencies, the spread of unsanctioned tools, and the practices that bring all of it under reasonable control.
How AI Changed the Development Workflow
Before assistants, code came from a developer's head, a colleague's review, a search result, or a library. Each path had friction that gave security a chance to notice. Today a model can generate an authentication flow, a database query layer, or an infrastructure template in seconds, and the friction has dropped to near zero.
New workflows have appeared alongside the assistants. Developers build features that call hosted model APIs, ship retrieval pipelines that index internal documents, and connect agents to repositories, ticketing systems, and cloud accounts. Each of these creates assets that did not exist in the old inventory: prompts, embeddings, model files, API keys for AI providers, and tool permissions granted to agents.
Security programs designed around source code and containers do not automatically cover those assets. Review processes assume a human author who understands the change, and generated code can arrive in volumes that outpace review. The first step toward control is describing the new workflow honestly: which assistants are used, what data they see, what code they produce, and which models and packages end up in builds. Without that picture, policy becomes guesswork, and enforcement targets the wrong places.
Risks in AI-Generated Code
Generated code is not inherently worse than human code, but it has characteristic weaknesses. Models learn from large bodies of public code that include insecure patterns, so suggestions may contain injection-prone queries, weak cryptographic choices, missing input validation, or overly permissive configurations. The output often looks polished and confident, which encourages acceptance without scrutiny.
Context limits add risk. An assistant sees a slice of the codebase and may not know about the project's authorization model, its sanitization helpers, or its logging rules. It may produce code that works in isolation and quietly bypasses a control the rest of the system depends on. Secrets can slip in too, either as placeholder credentials that get committed or as real ones copied from the surrounding files.
Dependencies bring a distinct hazard. Models sometimes suggest packages that do not exist, and attackers have an incentive to register those names with malicious content, a tactic sometimes called slopsquatting. A developer who installs the suggested package without checking it may import malware. Defenses include requiring that all code, generated or not, pass the same review and automated checks. Running static analysis and secret scanning on every change, verifying package names and publishers before installation, and training reviewers to treat generated code as untrusted input all reduce exposure.
The Model and Dependency Supply Chain
Models are software artifacts with their own supply chain. Teams download pretrained weights from public hubs, fine-tune them, and package them into services. Some model file formats can execute code when loaded, which means a malicious model can compromise the machine that opens it. Safer serialization formats exist, and policies should prefer them.
Provenance questions apply as well. Who published the model, what data trained it, what license governs it, and has anyone modified it since release? Models can carry licensing restrictions that conflict with commercial use, and training data can introduce legal or bias concerns. Maintaining an inventory of the models in use, with source, version, hash, and license, gives teams the same traceability they expect for libraries.
The conventional dependency chain remains central to the picture. AI applications rely on large frameworks, inference servers, vector databases, and orchestration libraries that evolve quickly and carry vulnerabilities like any other software. Dependency tracking should cover them, and a bill of materials that includes models and datasets, not only packages, reflects reality. Many teams now use AI security tools that scan models for unsafe contents, flag risky packages, and track provenance alongside conventional dependency analysis. Pinning versions, mirroring approved artifacts in an internal registry, and verifying checksums limit the damage if an upstream source is compromised.
Shadow AI and the Tooling Sprawl
Shadow AI is the use of AI tools without the organization's knowledge or approval. A developer signs up for a free coding assistant with a personal account, a team adds a browser plugin that summarizes tickets, or someone pastes proprietary code into a public chatbot to debug it. Each case is usually well intentioned, since the tools help people work faster. Each also sends company data to a provider whose retention, training, and security practices nobody has reviewed.
Blocking everything rarely works. People route around bans, and the organization loses the visibility it needed in the first place. A more effective approach offers sanctioned alternatives that are good enough to use, with clear data-handling terms, and then monitors for the rest. Network and endpoint telemetry, browser extension inventories, and expense records often reveal which tools are in use.
When comparing ai security tools, teams can start by defining the problem. Discovery tools find where AI is used. Code and dependency scanners catch flawed output and risky packages. Gateways and policy engines control what data reaches external models. Runtime protections guard deployed AI features. No single product covers everything, so teams should map tools to specific gaps and avoid buying overlapping coverage. Integrations matter more than feature lists, since tools that fit into repositories, pipelines, and ticketing get used, and tools that live in a separate console are forgotten.
Building a Practical Program
A workable program starts small and grows with evidence. The following steps give engineering and AppSec teams a sequence:
- Inventory the assistants, models, datasets, and AI-related services in use, including unapproved ones.
- Publish a short policy covering approved tools, data that must never leave the company, and review expectations for generated code.
- Apply the same automated checks to all code, including static analysis, dependency scanning, and secret detection.
- Control the model and package supply chain through approved sources, pinned versions, and verified artifacts.
- Limit agent and assistant permissions to the minimum needed, with scoped credentials and logging.
- Train developers on the specific risks, using examples from their own stack.
- Review incidents and near misses regularly, and adjust the policy and tooling.
Measurement keeps the program honest. Useful indicators include the share of repositories with scanning enabled, time to remediate findings in generated code, the number of unapproved tools discovered, and the proportion of models with a recorded source and license. Culture matters as much as controls. Developers who understand why a rule exists follow it, and those who find it arbitrary invent workarounds. Involving engineers in choosing tools and writing policies tends to produce rules that survive contact with real work.
Final Analysis
AI changes how software gets built, and security has to follow the new workflow instead of the old one. Generated code needs the same scrutiny as any other contribution, models and their dependencies need inventory and provenance, and unsanctioned tools need visibility and sensible alternatives.
Teams that map their actual AI usage, apply consistent automated checks, and keep permissions narrow can keep the productivity gains without accepting hidden risk. Progress comes from incremental steps, steady measurement, and a willingness to adjust as the tools and threats keep changing.