Signal

Why dashboards are dead Traditional observability is failing because it was built for human eyes. We clicked through Grafana dashboards and stared at complex charts. When a system crashed at midnight, we had to guess which code path failed. Jason Lopatecki, founder of Arize, argues that telemetry must change. It is no longer just for human diagnostics. Instead, think of logs and traces as the smoke thrown off by your system. This smoke is raw fuel for autonomous agents. If you do not provide this fuel, your debugging agents are just guessing. Traces on a filesystem To make agents work, we have to feed them the right format. This is where Signal comes in. Coding tools like Claude Code struggle with abstract web dashboards but work wonders with raw files. Signal pulls production logs and traces down directly into your repository. Sometimes these files are ten megabytes. It puts this context right next to your code inside a secure sandbox. Because the agent has the exact historical trace on its filesystem, it knows the exact path the execution took. It no longer has to guess among millions of logical branches. You are effectively pairing the agent with the exact state of the system at the moment of failure. From responder to reviewer This shift fundamentally changes our daily software engineering workflow. We invert the traditional loop. Instead of waking up to a frantic pager alert and digging through logs for clues, you wake up to an already investigated issue. The agent has already pulled the relevant traces, found the bug, and generated a draft pull request. Your job shifts from active emergency responder to code reviewer. However, Lopatecki is honest about current limitations. One-line bug fixes are easy. Complex architectural issues still require a human to steer the investigation over the finish line. To make this autonomous engine run, you must change your telemetry habits. You need to log and trace ten times more than before. What feels like useless noise to a human is critical context to an AI agent. Why enterprises keep agents local Security remains a massive hurdle for autonomous debugging in the enterprise. Large firms like Uber or Booking.com cannot simply point their live databases or production logs at external model APIs. They will not allow sensitive data to leave their perimeter. To solve this, Arize runs Signal inside the customer's Virtual Private Cloud (VPC). Engineers run their choice of sandboxes, such as Daytona, and configure the precise skills and prompts. This open approach takes your local workspace debugging setups and scales them to run periodically in a secure cloud environment. Connecting traces to evaluation metrics We also must address where evaluation metrics fit. Traditional testing happens in staging, but online evaluations run directly on live production traces. When Arize AX runs these checks, it applies an LLM-as-a-judge layer to assess system health in real-time. If a specific failure occurs—like a prompt injection or a formatting error—the system flags the trace. The agent uses this evaluation data to pinpoint where the software drifted. By combining raw execution paths with continuous evaluation metrics, the agent obtains a complete picture of the failure before attempting a pull request.