Trivy

Scaling Beyond the Single Agent Sandbox Software development is undergoing a fundamental shift where the primary interface for code creation is moving from the IDE keyboard to the agent prompt. However, many developers remain trapped in the "first innings" of this evolution, using AI primarily for context-aware autocomplete or small, isolated snippets. While GitHub Copilot and similar tools have drastically improved individual productivity, they often fail when confronted with "toil" tasks—large-scale technical debt, dependency migrations, and codebase modernizations that span hundreds or thousands of files. The bottleneck isn't the AI's ability to write code; it is the limited context window and the compounding of errors over long trajectories. If you ask a single agent to refactor a monolith into microservices in one shot, it will likely suffer from the "laziness problem," completing a fraction of the work before suggesting you hire a human team for the rest. To overcome this, we move toward **agent orchestration**: the practice of managing multiple, parallel agents working in coordinated swarms to tackle massive refactors that are too large for any single execution. Prerequisites for Orchestration Before diving into the OpenHands SDK, you should be comfortable with the following: * **Python Proficiency**: The SDK and automation scripts are built using Python. * **Docker Fundamentals**: You will need to run a local or remote agent server (the workspace) to provide a secure, containerized environment for agents. * **LLM API Management**: Familiarity with obtaining and using API keys from providers like Anthropic or OpenAI. * **Git Workflows**: Understanding branching and pull request (PR) structures is essential, as orchestrated agents typically interact with code by opening and managing multiple PRs. Key Libraries and Tools * **OpenHands SDK**: An MIT-licensed framework for building autonomous coding agents that can run terminal commands, edit files, and use browsers. * **Trivy**: A comprehensive security scanner used within the agent's environment to detect vulnerabilities (CVEs) in software dependencies and Docker images. * **LiteLLM**: A library used under the hood to provide a unified interface for calling various LLM providers. * **UV**: An extremely fast Python package and project manager used for installing the OpenHands CLI and managing dependencies. Implementation: Automated CVE Remediation at Scale To see orchestration in action, we can build a script that identifies security vulnerabilities and assigns them to a fleet of parallel agents for resolution. This pattern avoids the risk of a single agent getting stuck on one complex bug and halting the entire process. Phase 1: The Scanner Agent The first step is to instantiate an agent whose sole job is to audit the codebase. This agent detects the project's language, identifies the appropriate scanner (like `npm audit` or `trivy`), and outputs a structured list of vulnerabilities. ```python from openhands.sdk import Agent, LLM, RemoteWorkspace 1. Initialize the LLM llm = LLM(model="anthropic/claude-3-5-sonnet", api_key="YOUR_KEY") 2. Set up the workspace (Docker container) workspace = RemoteWorkspace(url="http://localhost:8000") 3. Define the Scanner Agent scanner = Agent( llm=llm, tools=["terminal", "file_editor"], workspace=workspace ) 4. Execute the scan scan_task = "Scan this repo for CVEs using Trivy and save results to vulnerabilities.json" scanner.run(scan_task) ``` Phase 2: Parallel Resolution Swarm Once the `vulnerabilities.json` file is generated, the orchestrator script parses the JSON and spins up a dedicated agent for each unique CVE. This is where the massive productivity gains happen. While one agent might be struggling with a breaking API change in a legacy library, ten other agents are successfully merging PRs for simpler dependency bumps. ```python import json Load the vulnerabilities found by the first agent vulnerabilities = json.loads(workspace.execute_command("cat vulnerabilities.json")) for cve in vulnerabilities: # Spin up a sub-agent for each CVE solver = Agent(llm=llm, workspace=workspace) prompt = f"Solve {cve['id']} in {cve['package']}. Update the dependency and fix breaking changes." # Run in parallel (implementation would use asyncio or threading) solver.run(prompt) ``` Architecture for Massive Refactors When dealing with thousands of files, simply "going piece by piece" is often insufficient because files are interconnected. Robert Brennan and the OpenHands team suggest three specific architectural patterns for orchestration: The Dependency Tree Approach Instead of random batching, analyze the project's dependency graph. Start the agents at the **leaf nodes**—utility files and low-level components that have no internal dependencies. As these agents finish, they unblock agents assigned to the higher-level modules that import them. This "bottom-up" strategy ensures that by the time an agent reaches the application's entry point, all underlying dependencies have already been modernized. Scaffolding and Dual-Mode Execution For high-risk migrations, such as moving from Redux to Zustand, you can create "scaffolding" that allows the application to run both libraries simultaneously. Agents can then migrate individual components one by one. This allows for continuous human verification—you can click through the app and ensure the migrated component still works without waiting for the entire project to be finished. Agent-to-Agent Context Sharing A critical challenge in parallelization is that agents often hit the same wall. If ten agents are all trying to update the same outdated library and realize it requires a specific compiler flag, they shouldn't all have to "discover" that solution independently. Advanced orchestration involves a shared `agent.md` file or a broadcast tool where agents can post discovered facts. If Agent A finds a solution, it broadcasts it, and Agents B through Z instantly integrate that context into their next action. Syntax Notes and Best Practices * **Action/Observation Loop**: The SDK operates on a trajectory of "Actions" (what the agent does, like a tool call) and "Observations" (what the environment returns, like terminal output). Monitoring this stream is essential for debugging why an agent might be looping. * **Micro-Agents**: Use specialized `.md` files (like `repo.md` or `instructions.md`) to provide persistent context. This is often more efficient than long system prompts, as the agent can "read" these files only when needed. * **The 90% Rule**: Do not aim for 100% automation. Orchestration is about achieving 90% automation, which still provides an order-of-magnitude lift. The remaining 10%—the "truly hard" edge cases—should be flagged by the agent for human intervention. Practical Tips and Gotchas * **The Laziness Problem**: Large Language Models are famously "lazy" with repetitive tasks. If you give an agent 50 files to fix, it may fix three and then write a comment saying "repeat for the other 47." Break tasks down so each agent only sees 3–5 files at a time. * **Non-Determinism in Communication**: When agents talk to each other, the system becomes significantly more non-deterministic. Brennan warns that without strict constraints, agents can enter "politeness loops," where they spend all their tokens wishing each other "zen perfection" instead of writing code. * **Limit Concurrent Agents**: If you are just starting, limit yourself to 3–5 concurrent agents. Any more, and the volume of PRs and context switches will overwhelm the human reviewer's ability to maintain the "human-in-the-loop" safety check.