Laravel AI SDK

Overview: The Unified AI Strategy for Laravel Building AI features often feels like a fragmented journey. Developers usually jump between specialized APIs for text, separate services for images, and complex libraries for audio transcription. The Laravel AI SDK changes this by providing a unified, first-party toolkit that handles the heavy lifting of AI integration. It treats AI as a core application concern, much like how Laravel handles databases or queues. By abstracting the differences between providers like OpenAI, Anthropic, and Gemini, it allows you to write cleaner, more maintainable code that isn't locked into a single vendor's API. Taylor Otwell designed the SDK to feel "Laravel-esque." This means leaning into conventions like class-based agents, fluent API chains, and deep integration with the existing Laravel ecosystem. Whether you need to summarize an issue in a project management tool, generate realistic speech via ElevenLabs, or perform semantic search on a mountain of PDFs, the SDK provides the scaffolding to do it efficiently. It moves AI from being an experimental add-on to a standard part of the modern developer's workflow. Prerequisites and Environment Setup Before you begin building, ensure you have a standard Laravel environment ready. You should be comfortable with PHP, Composer, and basic Laravel concepts like controllers and service providers. You will also need API keys from at least one AI provider. While the SDK supports local models via Ollama, production applications typically require keys for OpenAI or Anthropic. To get started, install the package via Composer: ```bash composer require laravel/ai ``` After installation, publish the configuration and migrations: ```bash php artisan vendor:publish --tag="ai-config" php artisan vendor:publish --tag="ai-migrations" php artisan migrate ``` The configuration file (`config/ai.php`) allows you to define your default providers. You can set different defaults for different modalities—for instance, using Claude for text and DALL-E for images. This flexibility is a core strength of the SDK. Key Libraries & Tools * **Laravel AI SDK**: The primary toolkit for interacting with LLMs, image generators, and audio services. * **Prism**: A community package by TJ Miller that serves as the query builder layer for the SDK's text generation. * **ElevenLabs**: Integrated for high-quality text-to-speech capabilities. * **Ollama**: Enables running local models for development and testing without incurring API costs. * **Laravel Boost**: A local MCP server that provides AI agents with context about your specific Laravel codebase. * **PostgreSQL with PGVector**: Used for storing and searching vector embeddings locally. Code Walkthrough: Implementing Agents and Tools 1. Creating an Agent The Agent class is the heart of the SDK. It encapsulates the identity of your AI. Instead of passing long strings of instructions in every controller, you define them once in a reusable class. You can generate one using the Artisan command: ```bash php artisan make:agent SalesCoachAgent ``` In the generated class, you define the system prompt and the models to use. The `instructions` method is where you set the "personality" and guardrails for the agent. ```php namespace App\Agents; use Laravel\AI\Agent; class SalesCoachAgent extends Agent { public function instructions(): string { return "You are an expert sales coach. Analyze the provided transcript and offer three actionable improvements."; } } ``` 2. Using Structured Output One of the most powerful features is getting the AI to return data in a specific format rather than a messy string. The SDK uses a JSON Schema builder to ensure the model follows your rules. This makes it possible to save AI responses directly into your database without fragile regex parsing. ```php use App\Agents\SalesCoachAgent; use Laravel\AI\Schema; $agent = new SalesCoachAgent(); $response = $agent->predict( prompt: "Analyze the call from yesterday.", schema: Schema::object([ 'sentiment' => Schema::string()->description('Overall tone of the customer'), 'score' => Schema::integer()->description('Score from 1-10'), 'follow_up_needed' => Schema::boolean(), ]) ); // Access data directly as an array echo $response['sentiment']; ``` 3. Integrating Tools (Function Calling) Tools allow your AI to actually *do* things. You can give an agent the ability to search the web, fetch a URL, or even query your own database. The SDK comes with several provider tools built-in, but you can also write your own custom tools by extending the `Tool` class and implementing a `handle` method. ```php use Laravel\AI\Tools\WebSearch; class ResearcherAgent extends Agent { public function tools(): array { return [ new WebSearch(), ]; } } ``` When you prompt this agent, it will realize it needs more info, call the `WebSearch` tool, and then use the results to finish its answer. This turns a static LLM into a dynamic assistant. Syntax Notes: Attributes and Traits The Laravel AI SDK makes heavy use of PHP attributes to simplify configuration. These attributes allow you to stay updated with the latest model advancements without changing your code logic. * **`#[UseCheapestModel]`**: Instructs the SDK to use the most cost-effective model for a specific provider (e.g., GPT-4o-mini or Claude Haiku). This is perfect for simple tasks like summarization. * **`#[UseSmartestModel]`**: Forces the use of the flagship model (e.g., Claude 3.7 Sonnet) for tasks requiring high reasoning capabilities. * **`RemembersConversations` trait**: Adding this to your agent automatically manages database storage for chat history, ensuring the AI remembers previous messages without you manually passing a growing array of context. Practical Examples: The 'Larvis' Workflow A practical application of this tech is building a voice-enabled assistant like "Larvis." The workflow demonstrates the multi-modal nature of the SDK: 1. **Transcription**: The user uploads an audio file of their question. The SDK uses a `transcribe` method (typically via Whisper) to convert audio to text. 2. **Context Retrieval**: The agent fetches relevant local documents (like Markdown files) and injects them into the prompt to provide specific knowledge the LLM wasn't trained on. 3. **Inference**: The agent generates a text response based on the transcription and the local documents. 4. **Speech Synthesis**: The text response is passed to the `audio` method, using ElevenLabs to generate a high-quality voice response that is sent back to the user. This entire pipeline, which would previously take dozens of different library integrations, can now be handled in a single Laravel controller using less than 50 lines of code. Tips & Gotchas * **Context Bloat**: Be careful not to attach too many tools or files to every request. Every tool definition and message in a conversation history consumes tokens, which increases latency and cost. Use the `RemembersConversations` settings to prune old messages. * **Failover Logic**: In production, always define a fallback provider. If OpenAI is experiencing downtime or you hit a rate limit, the SDK can automatically switch to Anthropic to keep your app running. * **Local Development**: Use Ollama for your daily coding to save money. You can switch your local `.env` to point the AI provider to `http://localhost:11434` to test your logic for free. * **Async Processing**: For long-running tasks like transcribing a massive video file or generating a complex image, use the `queue` method. This offloads the work to your Laravel worker and prevents your web request from timing out.

Overview: The Shift to Agentic Development In the current software development landscape, we are moving beyond simple Large Language Models (LLM) wrappers toward sophisticated, autonomous entities known as AI agents. Unlike traditional chatbots that merely respond to prompts, these agents can use tools, access external data, and make decisions to execute complex business workflows. Redberry, a veteran Laravel partner, has formalized this process through LarAgent, an open-source tool designed to bring agentic capabilities directly into the PHP ecosystem. This approach matters because it allows developers to automate non-deterministic tasks—decisions that can't be hard-coded with simple if/else logic—while staying within a framework they already know and trust. Prerequisites To effectively build agentic systems with the tools discussed, you should have a solid grasp of the following: * **Modern PHP & Laravel**: Proficiency in service providers, configuration management, and the Laravel ecosystem. * **LLM Fundamentals**: Understanding of system prompts, temperature settings, and the difference between deterministic and non-deterministic outputs. * **API Integration**: Experience connecting with third-party services, as agents rely heavily on tool-calling to interact with the world. * **Vector Databases & RAG**: A basic understanding of Retrieval Augmented Generation (RAG) for providing agents with custom context. Key Libraries & Tools * **LarAgent**: An open-source package that provides the primitives for building agents in Laravel, including instruction management and tool-calling orchestration. * **Laravel AI SDK**: A first-party toolset from the Laravel team focused on standardizing AI interactions across different providers. * **MCP Client for Laravel**: A specialized package allowing Laravel applications to connect to Model Context Protocol (MCP) servers, giving agents access to an unlimited array of pre-built tools. * **Model Agnostic Layers**: Architectural patterns that allow switching between providers like OpenAI, Anthropic, or local models via configuration. The Anatomy of an AI Agent Sprint Building an agent isn't a linear coding task; it's a process of experimentation. A typical five-week proof of concept (PoC) focuses on time-boxing the non-deterministic nature of the project. Week 1: Discovery and Mapping Before writing code, you must map the business process. The goal is to identify which parts are deterministic (best handled by standard code) and which require an agent. If you can write a rule-based logic for a decision, you should. AI is reserved for the gaps where rules fail. Weeks 2-3: The First Prototype Using LarAgent, developers define the agent's instructions and the tools it can access. A "tool" in this context is often a PHP class or a specific API endpoint the agent can trigger. ```php // Defining a basic agent in LarAgent $agent = LarAgent::make('SupportBot') ->instructions('Assist users with order tracking.') ->tools([ OrderTrackingTool::class, InventoryCheckTool::class ]); ``` During this phase, you establish a benchmark data set. This is a collection of inputs and expected outcomes used to measure the agent's performance. Weeks 4-5: Iteration and Accuracy Initial success rates for agents often hover around 60-70%. The final weeks involve refining prompts, adjusting the orchestration of multiple agents, and tweaking tool definitions to push accuracy toward a production-ready 98%. This often involves "human-in-the-loop" design, ensuring a person reviews critical agent decisions. Syntax Notes & Orchestration Patterns One notable pattern in agentic development is the move away from a single, massive agent toward **multi-agent orchestration**. Instead of asking one agent to "manage an entire warehouse," you might have a "Receiver Agent," a "Stock Agent," and a "Dispatcher Agent." In LarAgent, this is handled through configuration-level model selection. Because different models excel at different tasks, you might use a smaller, faster model for simple categorization and a larger model for complex reasoning. ```php // Configuration-based model selection 'agents' => [ 'categorizer' => [ 'model' => 'gpt-4o-mini', 'temperature' => 0, ], 'analyzer' => [ 'model' => 'claude-3-5-sonnet', 'temperature' => 0.5, ], ] ``` Practical Examples * **Automated Test Case Generation**: Agents can scan project requirements and draft comprehensive test suites, which human developers then verify and approve. * **Legacy System Interfacing**: Using agents to interpret data from legacy systems that lack modern APIs, acting as a conversational or structured bridge between old and new tech. * **Regulated Industry Workflows**: In finance or healthcare, agents can pre-process documents and flag anomalies, significantly reducing manual labor while keeping a human as the final authority. Tips & Gotchas * **Avoid Tool Overload**: Exposing too many tools (more than 10) can overwhelm the LLM, leading to "hallucinations" or incorrect tool selection. Keep the agent's toolkit focused. * **Deterministic First**: Never use AI for something that can be solved with a simple database query or a standard function. It is more expensive and less reliable. * **Benchmark Early**: You cannot improve what you cannot measure. Build your test data set in week one so you have a baseline for every iteration. * **Legacy Blockers**: When integrating with ancient systems, expect blockers. Discovery should prioritize credential and API access to avoid stalling the sprint.

The New Standard in AI-Assisted Development The landscape of AI-driven coding shifted overnight with the simultaneous release of Claude Opus 4.6 and GPT-5.3 Codex. For developers, the choice between Anthropic and OpenAI often comes down to a trade-off between speed and architectural depth. Historically, Claude dominated the speed game, while Codex provided the "plan B" for complex logic. This latest iteration changes that dynamic, forcing a re-evaluation of which model belongs in a professional workflow. Key Features and Architectural Alignment Testing these models requires a modern stack. Using the fresh Laravel AI SDK, both models demonstrated an impressive ability to ingest new technical skills via Laravel Boost. This "skill-based" learning allows the LLMs to utilize documentation they weren't originally trained on. GPT-5.3 Codex stood out by automatically generating automated tests and implementing robust error handling without explicit prompting. Claude Opus 4.6, meanwhile, focused on a polished user experience and cleaner, more immediate UI implementations. Analysis: The Speed-to-Quality Ratio The most shocking revelation in this head-to-head is the closing speed gap. GPT-5.3 Codex formerly dragged behind Opus, often taking twice as long to generate files. Now, Codex matches Opus second-for-second in simple tasks and remains within 20-30% of its speed on massive, multi-phase projects. However, Codex occasionally makes questionable executive decisions, such as downgrading an image model to Gemini 2.5 Flash to save tokens—a move that resulted in typos and lower quality. Opus is more faithful to the default high-quality settings but tends to take shortcuts, missing hard-coded URLs and failing to update documentation during complex refactors. Final Verdict and Recommendation If you value "production-ready" code that anticipates future edge cases, GPT-5.3 Codex is the superior architect. It provides significantly more depth in forward-thinking patterns and rigorous testing. Claude Opus 4.6 remains the king of the frontend and rapid prototyping, offering a better "terminal UX" and cleaner coding style. For most senior developers, the recommendation is clear: use Codex for the heavy lifting and backend logic, but let Opus handle the final UI polish. At nearly a quarter of the price of the Anthropic $100 Max plan, GPT-5.3 Codex currently offers the best value in the high-end LLM market.