Peter%20Wielander

Bridge the gap between local prototypes and production agents Most developers building with large language models hit a wall when transitioning from a local script to a production environment. In a local environment, an agent failing halfway through a task is a minor annoyance. In production, that failure means lost state, wasted compute tokens, and a broken user experience. Vercel has introduced the Workflow DevKit to solve exactly this: the "extra effort" required to make agents reliable, observable, and durable. The core problem with standard agent loops is that they are inherently volatile. If your AI SDK agent is running a long sequence of tool calls and the serverless function times out or the connection drops, you lose everything. Traditionally, fixing this required wiring up complex message queues, persistent databases, and custom retry logic. The Workflow DevKit replaces that infrastructure with a single TypeScript library that runs on any cloud, providing first-class observability and durability out of the box. Prerequisites and the essential toolkit To follow along with this implementation, you should be comfortable with TypeScript and the Next.js framework. Familiarity with the AI SDK (formerly Vercel AI SDK) is helpful, as we will be using its streaming and tool-calling patterns. Key Libraries & Tools - **Workflow DevKit**: The primary orchestration library used to define steps, manage persistence, and handle retries. - **AI SDK**: Used for managing the LLM interaction and streaming text/data to the client. - **Next.js**: The React framework providing the API routes and frontend structure. - **Vercel Sandbox**: A secure, isolated environment (micro-VM) used by the agent to execute code and manage files. Refactor the agent loop into an orchestration layer Converting a standard agent into a workflow-supported one involves moving the logic from a standard API handler into a deterministic orchestration function. This function acts as the "brain" that manages the sequence of operations. Step 1: Installation and Configuration First, install the necessary packages and update your compiler settings to handle the workflow logic. ```bash npm install workflow @workflow/next ``` In your `next.config.js`, you must wrap your configuration to allow the Workflow DevKit to bundle your workflow code separately. This separation ensures that the orchestration layer remains deterministic and free of side effects. ```javascript import { withWorkflow } from 'workflow/next'; const nextConfig = { // your existing config }; export default withWorkflow(nextConfig); ``` Step 2: Define the Workflow Function Create a new file (e.g., `code-workflow.ts`) and use the `use workflow` directive. This directive tells the compiler that this function is an orchestration layer. Under the hood, it compiles this into a separate bundle to ensure no state pollution occurs between runs. ```typescript "use workflow"; import { start } from "workflow/api"; import { DurableAgent } from "./agent-utils"; export async function codeWorkflow(messages: any[]) { const agent = new DurableAgent(); const writable = getWritable(); // Gets the stream for this workflow await agent.run({ messages, writable }); } ``` Isolate side effects using the Step Pattern In a workflow, you distinguish between the **orchestration layer** (which must be deterministic) and **steps** (where side effects like API calls and database writes happen). When a function is marked as a step, its input and output are cached. If the workflow needs to restart, it re-runs the orchestration code but skips the actual execution of the step, simply returning the cached result. Marking Tools as Steps For an AI agent, every tool call should be a step. This prevents the agent from, for example, charging a credit card twice if a network error occurs after the tool execution but before the next LLM call. ```typescript export const createSandbox = { execute: async (args: any) => { "use step"; // Marks this specific execution as a durable step const sandbox = await vercelSandbox.create(); return { id: sandbox.id }; } }; ``` By adding `"use step"` to your tool definitions, you gain automatic retries and durability. If your Vercel Sandbox creation fails due to a temporary API hiccup, the workflow will automatically retry that specific step based on your defined policy. Implement resumable streams for durable sessions One of the most powerful features of the Workflow DevKit is the ability to resume a session even if the user closes their browser or the server restarts. This is achieved by separating the stream from the API handler. The stream lives in the workflow's persistence layer (like Redis in production or a local file in development). Client-Side Reconnection To support this, your API must return the `runId`. The client can then use this ID to check for an existing session and reconnect to the stream. ```typescript // API Handler: chat/route.ts export async function POST(req: Request) { const { messages } = await req.json(); const run = await start(codeWorkflow, messages); return new Response(run.stream, { headers: { "x-workflow-id": run.runId } }); } ``` On the frontend, you can use a transport layer that checks for a stored `runId` and calls a dedicated "resume" endpoint rather than starting a fresh chat. This ensures that even if an agent task takes 10 minutes, the user can always see the progress. Master the long-running agent with Sleep and Webhooks Standard serverless functions have strict timeouts (often 30 seconds to 15 minutes). AI agents often need to perform tasks over days or wait for human input. The Workflow DevKit handles this by "suspending" the execution. When you call `sleep`, the function literally stops running and consumes zero resources until the timer expires. Adding a Sleep Tool You can expose this capability to the LLM as a tool. This allows the agent to schedule its own future tasks. ```typescript import { sleep } from "workflow"; export const sleepTool = { execute: async ({ duration }) => { // No "use step" needed because sleep is a built-in step await sleep(duration); return { status: "awoken" }; } }; ``` Human-in-the-loop with Webhooks For tasks requiring approval (like deploying code to production), you can generate a webhook. The workflow will pause indefinitely until that webhook is triggered. ```typescript import { createWebhook, awaitWebhook } from "workflow"; // Inside a workflow const hook = await createWebhook(); console.log(`Please approve here: ${hook.url}`); const result = await awaitWebhook(hook); // Workflow resumes only after the human clicks the link ``` Essential Syntax and Best Practices Syntax Notes - **Directive Placement**: The `"use workflow"` directive must be at the top of the function body or file to trigger the specialized compiler. - **Determinism**: Never use `Math.random()` or `new Date()` directly inside the orchestration function. If you need these values, wrap them in a step so the value is cached and consistent during replays. - **Implicit Streams**: Use `getWritable()` within your steps to write data back to the UI. This ensures that data packets are correctly sequenced even across retries. Tips & Gotchas - **Version Compatibility**: If you update your code while a workflow is running, the Workflow DevKit can perform compatibility checks. If the step signatures have changed significantly, you may need to cancel the run or use the upcoming "upgrade" feature to migrate the state. - **Local Debugging**: Use the `npx workflow web` command. This launches a local dashboard where you can inspect every step's input and output, manually trigger webhooks, and visualize the event log. - **Sandbox State**: While the workflow manages the *orchestration* state, external systems like a Vercel Sandbox maintain their own state. Ensure your steps are idempotent; if a file-writing step is retried, it should overwrite the file rather than appending to it, unless appending is the intended behavior.