Madura says programming with DSPy ends the era of prompt engineering

The landscape of Large Language Model (LLM) development is undergoing a fundamental shift away from "prompt engineering" toward a rigorous programming paradigm. DSPy represents this evolution, providing a declarative framework for building modular software where LLMs are treated as first-class citizens. Instead of manually tweaking strings to coax specific behaviors out of a model, developers define the intent of their program through typed interfaces and logical modules. Kevin Madura, a technical consultant at AlixPartners, argues that this transition is essential for enterprise-grade applications that require testability, robustness, and transferability across different models.

This tutorial explores how to use DSPy to decompose complex business logic into maintainable Python code. We will examine the core primitives that allow you to separate the structure of your program from the implementation details of the underlying LLM. By the end of this guide, you will understand how to build a multi-stage pipeline that can classify, route, and process various document types using optimized prompting strategies that the system generates for you.

Prerequisites

To follow this tutorial, you should have a baseline understanding of the following concepts and tools:

• Python Programming: Familiarity with classes, decorators, and asynchronous programming in Python.
• Pydantic: Knowledge of Pydantic for data validation and settings management, as it underpins much of DSPy's type hinting.
• LLM Basics: An understanding of how LLMs process tokens and the general concept of system prompts vs. user messages.
• Environment Setup: A working Python environment with an API key for a provider like OpenAI, Anthropic, or Google (or an aggregator like OpenRouter).

Key Libraries & Tools

• DSPy: The core declarative framework used to structure and optimize LLM programs.
• LightLLM: Used under the hood by DSPy to provide a unified interface for calling various model providers.
• Attachments: A utility library that simplifies working with disparate file types (PDFs, images) and converting them into LLM-friendly formats.
• Phoenix: An observability platform from Arize AI used for tracing and debugging LLM calls within the DSPy ecosystem.
• BAML: A domain-specific language for extracting structured data from LLMs, which can be used as an adapter within DSPy for better token efficiency.

Section 1: Signatures as Declarative Intent

The heartbeat of any DSPy program is the Signature. A signature defines what a task should accomplish without specifying how it should be prompted. This is a critical distinction: you are defining the inputs and outputs, and DSPy handles the transformation into a prompt.

Shorthand Signatures

For simple tasks, you can use a shorthand string notation. This is ideal for rapid prototyping:

import dspy

# A simple sentiment classifier shorthand
sentiment_predictor = dspy.Predict("text -> sentiment:int")
response = sentiment_predictor(text="The service was absolute garbage.")
print(response.sentiment)

In this example, text -> sentiment:int tells DSPy that the input field is named text and the output field is an integer named sentiment.

Class-based Signatures

For more complex enterprise logic, class-based signatures allow you to provide docstrings and field descriptions that the model uses to understand the context. These descriptions essentially function as "mini-prompts" embedded within your code structure.

class DocumentClassifier(dspy.Signature):
    """Classify the type of document based on visual and text content."""

    document_images = dspy.InputField(desc="Images of the first few pages of the document")
    document_type = dspy.OutputField(desc="One of: SEC_FILING, PATENT, CONTRACT, OTHER")

# Usage
classifier = dspy.Predict(DocumentClassifier)

Section 2: Building Logic with Modules

Modules are the organizational units of DSPy, analogous to layers in a neural network. A module wraps one or more signatures and can include custom control flow, database calls, or other Python logic. Every module inherits from dspy.Module and implements an __init__ method to define its components and a forward method for the execution logic.

class SupportAnalyzer(dspy.Module):
    def __init__(self):
        super().__init__()
        self.categorize = dspy.ChainOfThought("message -> category")
        self.sentiment = dspy.Predict("message -> sentiment:int")

    def forward(self, message):
        category = self.categorize(message=message).category
        sentiment = self.sentiment(message=message).sentiment
        
        # Add hard-coded business logic
        is_urgent = (sentiment < 3) or (category == "billing")
        
        return dspy.Prediction(category=category, sentiment=sentiment, urgent=is_urgent)

By using dspy.ChainOfThought instead of dspy.Predict, you automatically instruct the model to reason through the problem before providing the final answer, which is often more accurate for nuanced classification tasks.

Section 3: Adapters and Token Efficiency

While signatures define the intent, Adapters determine how that intent is formatted for the LLM. By default, DSPy uses a JSON adapter, but this can be inefficient for complex nested objects. Kevin Madura highlights that using alternative formats like BAML can improve performance by 5-10% because they are more intuitive for models to parse and use fewer tokens.

from dspy.adapters import ChatAdapter, JSONAdapter
from baml_adapter import BAMLAdapter # Hypothetical specialized adapter

# Switching adapters is a one-line change that doesn't break your program logic
with dspy.context(adapter=BAMLAdapter()):
    response = my_module(input_data=data)

Adapters live between the Signature and the LLM call, acting as the "translator" that turns your Python objects into the final string sent over the wire.

Section 4: The Power of Optimizers

The most distinctive feature of DSPy is the Optimizer (formerly called Teleprompters). Optimizers are algorithms that tune the prompts in your program to maximize a specific Metric. This is "AI building AI": the system tries different prompt variations and few-shot examples, measures them against your ground truth data, and keeps the version that performs best.

The Optimization Flow

1. Define a Dataset: You need 10 to 100 examples of inputs and expected outputs.
2. Define a Metric: This can be a simple equality check or a "LLM-as-a-judge" metric that evaluates subjective quality.
3. Run the Optimizer: Algorithms like MIPRO (Multi-objective In-context Prompt Optimization) will iteratively refine your program.

from dspy.telepropmt import MIPRO

# Setup the optimizer
optimizer = MIPRO(metric=my_accuracy_metric, num_candidates=10)

# Compile the program (this is where the 'training' happens)
optimized_program = optimizer.compile(SupportAnalyzer(), trainset=my_dataset)

# Save the optimized state
optimized_program.save("optimized_support_v1.json")

This compiled object contains the highly tuned prompts that the optimizer discovered. You can then load this program in production, ensuring that your small, cheap model (like GPT-4o mini) performs nearly as well as a larger, expensive model.

Syntax Notes

• Dot Notation: DSPy predictions return objects that allow for easy access via dot notation (e.g., response.sentiment).
• Context Managers: Use dspy.context or dspy.settings.configure to switch models or adapters globally or within a specific block of code. This is invaluable for "model mixing" where you use a cheap model for classification and a powerful model for reasoning.
• Type Hinting: Always use Python type hints in signatures (text:str -> summary:str). DSPy uses these to validate the LLM's response before it ever reaches your application logic.

Practical Examples

• Document Routing: A pipeline that takes a PDF, uses an image-capable model (Gemini 2.0 Flash) to classify the layout, and then routes it to a specialized summarizer module if it's a contract, or an extraction module if it's an SEC filing.
• Boundary Detection: In legal tech, identifying where the "Main Agreement" ends and "Schedule A" begins. By passing page-level classifications into a DSPy module, the system can determine logical document boundaries with high precision.
• Cost Reduction: Taking a complex reasoning task that currently requires GPT-4o and using DSPy optimizers to find a prompt strategy that allows Claude 3 Haiku to achieve the same accuracy at 1/10th the cost.

Tips & Gotchas

• Caching: DSPy caches LLM responses by default. If you change your code but the output doesn't change, check if you're hitting the cache. Changing a single space in a signature string will bust the cache.
• Field Naming: The names of your input and output fields are prompts. If you name a field output1, the model will struggle. If you name it summarized_legal_clause, the model's performance will naturally improve.
• The Optimizer is Not Magic: An optimizer cannot fix a fundamentally broken program logic. Build your program first, ensure it works on a handful of examples manually, and then use the optimizer to squeeze out the final 10-20% of performance.
• Observability: Always use a tool like Phoenix or the dspy.inspect_history(n=1) command during development to see exactly what strings are being sent to the LLM. DSPy adds a lot of "boilerplate" to your prompts that you need to be aware of.