Dependabot

Why Simple Dependency Bumpers Fail at Scale Automating dependency updates sounds straightforward. Tools like Dependabot and Renovate scan manifests, bump versions, and open pull requests. But in complex enterprise environments, this approach falls short. Modern security vulnerabilities rarely live in isolated manifests. They hide deep inside base OS images, transient binary downloads, or deeply coupled runtimes. At Form3, managing thousands of repositories created a endless backlog of vulnerabilities. Bumping a single runtime version often breaks linting rules or requires simultaneous updates to other local tooling. Traditional automation tools simply make the change and walk away, leaving developers to clean up a broken build. Solving this requires deep reasoning. An automation tool must understand the full surface of a security issue, change only what is necessary, and iterate until the continuous integration (CI) pipeline turns green. The Two-Tier Architecture of PatchPilot To bridge this gap, the engineering team built PatchPilot, an AI-assisted patching tool. The design divides operations into two distinct layers: a deterministic layer and an agentic layer. This architectural boundary serves as the primary security defense. The Deterministic Layer A boring, reliable Go application handles all orchestration. It discovers vulnerable artifacts, scans OCI images, and maps them to their respective source repositories. Crucially, this layer holds all sensitive credentials. It alone clones repositories, commits code, pushes to GitHub, and triggers CI workflows. The AI agent never touches these credentials. The Agentic Layer The LLM-powered agent acts as a file-system editor. It receives a localized directory containing the code and a highly structured prompt detailing the vulnerability. The agent modifies the files on disk to remediate the vulnerability, then returns control to the Go application. This strict separation limits the blast radius of potential prompt injections. Neutralizing the Lethal Docker Socket with Firecracker Security concerns multiply when an agent needs to verify its own work. To ensure a patched Dockerfile actually builds, the agent requires container tools. Developers typically resolve this by sharing the host's Docker socket with the agent's environment. This is highly dangerous. Access to the Docker socket allows a compromised agent to launch privileged containers, escape to the host system, plant SSH keys, and steal sensitive environment variables. To mitigate this risk, the team migrated the agent's environment inside Firecracker microVMs. This isolation strategy gives the agent its own dedicated kernel. If the agent attempts to escape the Docker daemon, it remains trapped inside the microVM. It cannot access host memory or neighbor processes. Restricting Network Access and Controlling Prompts Network connectivity presents another security challenge. The deterministic controller only needs access to GitHub and internal container registries. The agent's requirements depend entirely on the language ecosystem it patches. Python, Java, and Go require access to entirely different package repositories. The architecture solves this by applying isolated network policies. A DNS and TCP forwarder routes all microVM network traffic through a host process via a virtual socket (VSock). The host process applies strict policies based on target ports and hostnames. Additionally, the team uses prompt steering to warn the agent against trusting vendor directories or raw CI logs. They run automated evaluations against custom, vulnerable repositories to test if the agent will resist malicious instructions embedded in codebase files. The Future of Production Sandboxing Securing AI agents requires treating them as powerful, untrusted actors. Handing production credentials to a language model invites disaster. By isolating dangerous tasks within microVMs and executing API interactions through a deterministic manager, developers can deploy helpful agents without sacrificing system integrity.