Graziano uses coding agents to push AI accuracy to 86 percent

The solution is not to write the perfect system prompt by hand. Instead, we must use a coding agent to build and optimize our target agent through automated, iterative feedback loops. This is the core practice of agents building agents.

Building the Guardrails With Golden Datasets

To program an agent with another agent, you need concrete metrics. You cannot optimize what you do not measure. This requires a golden dataset—a curated collection of inputs and expected outputs built alongside subject matter experts.

Unlike traditional test suites, an agentic golden dataset does not just measure static text matches. It tests complex, multi-step behaviors. Did the agent call the right database API? Did it pass the correct arguments? Did it fetch context in the right sequence?

Using Mastra to build a baseline agent, Graziano demonstrated how a naive system starts with a dismal 18 percent pass rate on complex datasets. The agent can answer basic questions embedded in its training weights, but fails on anything requiring external coordination. Fixing this manually becomes an endless game of prompt-engineering whack-a-mole.

The Self-Correction Loop in Action

Instead of manual edits, developers can use a dedicated coding assistant like Claude Code to run an automated optimization tool named AutoAgent. This tool operates on a simple principle first validated by AI researcher Andrej Karpathy in his auto-research experiments: coding models can successfully write and refine machine learning code to optimize a specific loss metric.

The loop runs systematically. First, the coding agent creates a new git branch for a specific hypothesis. It then modifies the target agent's system prompt, tool logic, or context retrieval mechanisms. Next, it runs the evaluation suite and generates a detailed performance report. If accuracy improves, the changes merge; if a regression occurs, the system rolls back to the last stable state.

By running this automated branch-and-evaluate cycle, AutoAgent pushed a real production agent's accuracy from a 67 percent baseline to a highly reliable 86 percent in just ten iterations. It achieved this by finding edge cases and fixing faulty tool logic that human developers missed.

Mining Live User Failures for Golden Data

Even a perfect golden dataset misses real-world user quirks. Live production data is messy. Users input unexpected queries, and edge cases inevitably slip through. To close this gap, Graziano structures a feedback system that directly ingests trace logs from live user sessions.

When a user flags a poor response or a subject matter expert annotates a failing run, the system saves the raw trace. Once the team collects a batch of traces, an offline agent runs a clustering analysis. It groups failures into distinct buckets—such as malformed formatting or missing API arguments.

An adversarial review step then traces these clusters back to their root causes and drafts a fix proposal. Crucially, these live failure modes are immediately converted into new test cases for the golden dataset, permanently preventing future regressions.

Why Harness Engineering Beats Prompt Tweaking

This entire approach hinges on what Graziano calls Harness Engineering. It moves the developer's role from writing manual code to building the environment in which code writes itself.

By establishing quality gates, automated evaluations, secure sandboxes, and deep observability, you build a structure where AI can iterate safely. Instead of guessing how to rewrite a prompt, you let the loop find the optimal configuration. This shift turns agent development from an unpredictable art into a repeatable engineering discipline.