Python

The shift from text extraction to visual document intelligence Traditional Retrieval-Augmented Generation (RAG) pipelines rely on a fractured architecture. To process a complex PDF, you must first disassemble it: text is stripped into strings, tables are reconstructed through OCR, and images are isolated into sub-directories. This process, while standard, destroys the spatial context of the document. When we segregate these entities, we lose the relationship between a figure and its caption, or the alignment of data in a non-standard table. It is like disassembling a family and expecting a stranger to identify they belong together. ColPali represents a fundamental shift by treating every document page as an image rather than a collection of characters. Instead of running expensive and often error-prone OCR passes, we generate embeddings directly from the visual representation of the page. This approach is particularly effective for convoluted data like insurance policies, government forms, or technical manuals where text is often embedded within graphics. By keeping the document whole, we preserve the visual semantics that human readers use to navigate complex information. ColPali architecture and the mechanics of late interaction At the heart of this vision-based retrieval is the concept of late interaction. Unlike traditional models that compress an entire chunk of text into a single vector, ColPali breaks a page into a grid of patches—typically 32x32. Each patch is processed through a vision-based encoder to generate its own embedding vector. If a document has 10 pages and each page has 15 patches, the system manages 150 vectors. When a user submits a text query, the model tokenizes the text and generates vectors for each token. The "late interaction" occurs when we perform a dot product between every query token vector and every image patch vector stored in the database. We calculate a maximum similarity score for each token against the patches, then sum these maximums to derive a total similarity score for the page. This ensures that a page is retrieved only if all parts of the user's question find strong matches across the various patches of that image. It effectively solves the problem of finding specific information buried in a sea of similar terms across a large corpus. Setting up the environment and vector storage To implement this, we require a vector database that supports multi-vector configurations and specific comparators. Qdrant is uniquely suited for this task because it allows us to define a collection with a `multivector_config` using the `max_sim` (maximum similarity) comparator. This is essential for executing the late interaction logic during search. Prerequisites and libraries To follow this implementation, you will need Python 3.10+ and Docker to run the Qdrant instance locally. The primary libraries used include: * **ColPali-engine**: For loading the pre-trained vision-retrieval models. * **Qdrant-client**: To interface with the vector database. * **Pillow (PIL)**: For image processing and RGB conversion. * **Strands Agent**: A lightweight framework to orchestrate the agentic workflow. ```python from colpali_engine.models import ColPali from qdrant_client import QdrantClient, models Initialize Qdrant local instance client = QdrantClient(host="localhost", port=6333) Create a collection with MaxSim comparator client.create_collection( collection_name="document_vision", vectors_config=models.VectorParams( size=128, distance=models.Distance.COSINE, multivector_config=models.MultiVectorConfig( comparator=models.MultiVectorComparator.MAX_SIM ) ) ) ``` Logical code walkthrough for vision-based RAG The implementation follows a three-stage pipeline: data ingestion, semantic retrieval, and agentic response generation. Stage 1: Document to image conversion Before embedding, we must convert PDF pages into a standard image format. This step ensures that the vision model receives consistent input regardless of the original document's source. We store these images in a list with metadata including `page_number` and `document_id` to allow for easy reconstruction after retrieval. ```python def convert_pdf_to_images(pdf_path): # Uses pdf2image to transform pages into RGB tensors images = [] # ... logic to iterate pages ... return images ``` Stage 2: Embedding and ingestion We pass these images through the ColPali pre-processor and model. Note that the batch size should be kept small—typically 2 to 4—if running on a consumer-grade laptop to avoid memory crashes. The resulting embeddings are then upserted into Qdrant. Stage 3: Retrieval and multimodal response When a query is received, we generate its multi-vector representation and query Qdrant. The database returns the top 'k' most relevant images. Because these are images, we cannot use a standard text-based LLM for the final answer. We need a multimodal model like Claude 3.5 Sonnet via Amazon Bedrock or a local model via Ollama to interpret the visual chunks and generate a response. Creating agentic workflows with Strands To make this system interactive, we wrap the retrieval and generation logic into an agent using the Strands Agent framework. Strands is a model-first SDK that prioritizes reasoning over complex prompt engineering. It treats an agent as a combination of a model and a set of tools. By defining our retrieval logic as a custom tool, we allow the agent to decide when and how to search the vector database. ```python from strands import Agent, tool @tool def retrieve_documents(query: str): # Logic to search Qdrant and return image paths return matched_image_paths Initialize the agent with Bedrock and tools agent = Agent( model_id="anthropic.claude-3-5-sonnet-20241022-v2:0", tools=[retrieve_documents, image_reader, speak] ) ``` In this setup, the `image_reader` tool handles the multimodal interpretation, while the `speak` tool provides the final voice synthesis. This turns a standard search query into a conversational experience where the agent "looks" at the document and "speaks" the findings back to the user. Practical applications and performance considerations While ColPali is powerful, it is not a universal replacement for traditional RAG. It is computationally heavy during the ingestion phase because storing 32x32 vectors per page requires more storage than a single text embedding. However, the retrieval speed remains high due to optimized indexing techniques like Hierarchical Navigable Small World (HNSW), which prunes the search space effectively. This architecture shines in industries like insurance and aerospace. For instance, IKEA assembly manuals contain almost no text, relying instead on emojis and diagrams. A traditional RAG system would find zero matches for a text query about a specific screw in an IKEA PDF. A vision-based system, however, can find the visual pattern of that screw across the patches and identify the correct page. Start with cost-effective text-based RAG for simple documents, but switch to vision-based retrieval when the context is visual or the data is highly convoluted.

Breaking Free from Fragile Code Hardcoded logic is the silent killer of maintainable software. When you bake specific behaviors directly into a class, you create a rigid structure that resists change. If your Python data pipeline only knows how to load from a CSV file because the `pd.read_csv` call is buried inside a method, you are stuck. The moment a requirement shifts—say, you need to pull from a SQL database or an S3 bucket—you have to perform surgery on the class itself. This violates the Open-Closed Principle and makes unit testing a nightmare. You cannot test the pipeline logic in isolation because the database connection or file system dependency is "baked in." Dependency Injection (DI) solves this by shifting the responsibility of creating dependencies from the object that uses them to the code that calls it. Instead of a class looking for its tools, you provide the tools upon initialization. This simple shift in perspective turns brittle, monolithic blocks of code into a collection of swappable, modular components. Prerequisites and Toolkit To implement these patterns effectively, you should be comfortable with Python 3.10+ fundamentals, specifically classes and type hinting. Familiarity with functional programming concepts like first-class functions and closures will help when we move into manual injection techniques. Key Libraries & Tools - **Typing Module**: Uses `Callable`, `Protocol`, and `Any` to define interfaces. - **FastAPI**: A modern web framework that includes a built-in dependency injection system. - **Thesys C1**: A generative UI API (featured sponsor) that demonstrates how external services are integrated into modern backends. Refactoring to Manual Injection We start by extracting hardcoded methods into standalone functions or objects. By passing these functions as arguments, we transform standard methods into higher-order functions. ```python from typing import Callable def load_data_from_csv() -> list[dict]: return [{"name": "Arjan", "id": 1}] class DataPipeline: def run(self, loader: Callable[[], list[dict]]): data = loader() print(f"Processing {data}") Usage pipeline = DataPipeline() pipeline.run(loader=load_data_from_csv) ``` While functional injection is elegant for simple scripts, a class-based approach using Protocols offers more robust architectural guardrails. Protocols allow for structural subtyping—you define the *shape* of an object (e.g., it must have a `.load()` method) without requiring it to inherit from a specific base class. This keeps your pipeline decoupled from the concrete implementation of the loader. ```python from typing import Protocol class Loader(Protocol): def load(self) -> list[dict]: ... class CSVLoader: def load(self) -> list[dict]: return [{"data": "from_csv"}] class DataPipeline: def __init__(self, loader: Loader): self.loader = loader def run(self): data = self.loader.load() # process data... ``` Building a Custom DI Container In larger systems, manual wiring in the `main()` function becomes verbose. A DI Container acts as a registry for your dependencies. It manages the lifecycle of objects, deciding whether to return a new instance or a cached singleton. ```python class Container: def __init__(self): self.providers = {} self.singletons = {} def register(self, name, provider, is_singleton=False): self.providers[name] = (provider, is_singleton) def resolve(self, name): if name in self.singletons: return self.singletons[name] provider, is_singleton = self.providers[name] instance = provider() if is_singleton: self.singletons[name] = instance return instance Wiring it up container = Container() container.register("loader", CSVLoader, is_singleton=True) container.register("pipeline", lambda: DataPipeline(container.resolve("loader"))) pipeline = container.resolve("pipeline") pipeline.run() ``` This container allows you to centralize your configuration. You could even swap providers based on environment variables or a JSON config file, allowing the application to change behavior without changing a single line of business logic code. Syntax Notes and Conventions Python is uniquely suited for DI because functions are first-class objects. You don't always need a heavy framework. Using `lambda` functions for delayed execution is a common pattern when a dependency requires runtime arguments (like a filename) that the container doesn't know about yet. Additionally, the use of `typing.Protocol` is preferred over `abc.ABC` because it promotes loose coupling; any class that happens to have the right method names satisfies the protocol. Practical Examples and Frameworks FastAPI demonstrates the peak of DI utility. It uses a `Depends()` function to handle database sessions. This ensures that every route gets a fresh session that is automatically closed after the request, keeping the endpoint code clean and focused only on the logic of the API. DI is also essential in Machine Learning pipelines. You might want to swap an `IncompleteDataTransformer` for a `StandardScaler` during an experiment. By injecting these as components, you can run multiple versions of a pipeline simply by changing the injection script. Tips and Gotchas Avoid over-engineering. If you are writing a 50-line script, a DI Container is overkill. Just pass the function. A common mistake is "Interface Bloat," where you define protocols for everything even when there is only ever one implementation. Only introduce abstraction when you actually need to swap the behavior—usually for testing or supporting different storage backends. Finally, remember that Python does not enforce type hints at runtime. If you inject the wrong object, it will only fail when the method is called, so back your DI architecture with a solid suite of unit tests.

The Problem with Python-Centric AI Pipelines Python is the undisputed king of AI prototyping, but it remains trapped in a "container cage." Developers currently face a mountain of infrastructure friction: writing Dockerfiles, managing remote GPUs, and stringing together fragile HTTP calls just to run a new model from Hugging Face. This complexity creates a massive bottleneck between research and deployment. Yusuf Olokoba and his team at Muna are solving this by treating AI inference as a compilation problem. Instead of deploying environments, they convert Python functions into tiny, self-contained native binaries. This approach allows a model like Google’s Gemma 2 270M to run anywhere—from local Apple silicon to edge devices—without a Docker container in sight. Prerequisites and Tooling To follow this workflow, you should be comfortable with Python syntax, basic C++ types, and the concept of Foreign Function Interface (FFI). Key tools mentioned include: * **PyTorch 2.0**: Specifically the Torch FX graph tracer. * **OpenAI Client**: The target interface for model consumption. * **Node.js**: Used here to demonstrate cross-language execution. Tracing and Type Propagation The compiler pipeline begins with **tracing**, which captures the function's logic as an Intermediate Representation (IR). While Muna initially explored using LLMs to generate traces via structured outputs, they eventually moved to analyzing the Abstract Syntax Tree (AST) for speed. The critical challenge is Python's dynamic nature versus the static requirements of C++ or Rust. Muna uses **type propagation** to solve this. If a function concatenates a string prefix with an input string, the compiler identifies the C++ output must be `std::string` and constrains the generated code accordingly. Code Walkthrough: From Python to Binary ```python def get_embeddings(text: list[str]): prompts = [f"query: {s}" for s in text] tokens = tokenizer(prompts) return model(tokens) ``` The compiler traces this logic and uses LLMs to generate equivalent native code. By using LLMs to write the elementary operations (like string concatenation or matrix multiplication), the compiler avoids the manual labor of hand-coding every possible Python operation in C++. ```cpp // Generated C++ snippet std::vector<std::string> prompts; for (auto& s : text) { prompts.push_back("query: " + s); } ``` Once compiled into a `.so` or `.dll` file, you can load it into Node.js using FFI: ```javascript const nativeLib = ffi.Library('./embedding_model.so', { 'get_embeddings': ['pointer', ['pointer']] }); ``` Tips and Gotchas * **Hybrid Inference**: Small models like Gemma 2 270M are ideal for local execution to reduce latency and cloud costs. * **LLM Verification**: Always fence LLM-generated compiler code with automated testing to ensure the native output matches the Python reference exactly. * **FFI Scaffolding**: Standardizing the FFI layer allows you to swap model binaries behind a familiar OpenAI-style `client.embeddings.create` interface without changing application code.

Overview of Event Sourcing Most applications function by overwriting state. When a player picks up a sword in a game, a database record changes from two to three. This approach is efficient but destructive; it discards the history of how that state was reached. Event Sourcing flips this paradigm. Instead of storing the final balance or the current inventory count, you store a sequence of immutable events—facts that have happened in the past. By replaying these events from the beginning, you can reconstruct the state at any point in time. This provides an inherent audit log, simplifies debugging by allowing "time travel," and enables the creation of multiple "projections" or views of the same data without altering the source of truth. It is the same fundamental logic that powers Git and Blockchain. Prerequisites To follow this implementation, you should have a solid grasp of Python 3.10+ fundamentals, specifically **Object-Oriented Programming (OOP)**. Familiarity with Dataclasses and Enums is essential, as these provide the structure for immutable events. A basic understanding of **Dependency Injection** will also help when connecting the inventory logic to the underlying event store. Key Libraries & Tools - **enum**: Used to define distinct, readable event types like `ITEM_ADDED` and `ITEM_REMOVED`. - **dataclasses**: Provides a concise way to create event objects, specifically using `frozen=True` to ensure immutability. - **collections.Counter**: A specialized dictionary subclass for counting hashable objects, used here to aggregate inventory totals. - **functools.cache**: Implements a simple memoization strategy to avoid replaying the entire event history on every read request. - **Flox**: A tool for creating reproducible development environments, ensuring consistent package management across different machines. Code Walkthrough: Building the Core System Step 1: Defining Immutable Events We start by defining what an event looks like. It must contain the type of action and the data associated with it. ```python from dataclasses import dataclass, field from datetime import datetime from enum import Enum, auto class EventType(Enum): ITEM_ADDED = auto() ITEM_REMOVED = auto() @dataclass(frozen=True) class Event: type: EventType data: str timestamp: datetime = field(default_factory=datetime.now) ``` The `frozen=True` parameter is vital. Events represent the past; they cannot be changed once they occur. We use a `default_factory` for the timestamp to ensure each event is accurately placed in the timeline. Step 2: The Event Store and Caching The Event Store is a simple append-only list. To prevent performance degradation as the list grows, we apply a cache to the state reconstruction method. ```python from functools import cache from collections import Counter class Inventory: def __init__(self, store): self.store = store @cache def get_items(self): counts = Counter() for event in self.store.get_all_events(): if event.type == EventType.ITEM_ADDED: counts[event.data] += 1 elif event.type == EventType.ITEM_REMOVED: counts[event.data] -= 1 return {k: v for k, v in counts.items() if v > 0} def _invalidate_cache(self): self.get_items.cache_clear() ``` When we add an item, we append an event to the store and trigger `_invalidate_cache()`. The next time `get_items()` is called, it recalculates and recaches the state. Step 3: Advanced Projections Projections allow us to ask different questions of our data. For example, we can determine which items were collected most frequently, regardless of whether they are still in the inventory. ```python def get_most_collected(store): events = store.get_all_events() added_items = [e.data for e in events if e.type == EventType.ITEM_ADDED] return Counter(added_items).most_common(3) ``` Syntax Notes This implementation relies on **Generic Type Variables (`TypeVar`)** when evolving the system to handle complex objects rather than just strings. Using `typing.Generic[T]` allows the `Event` and `EventStore` classes to remain flexible, supporting any data structure while maintaining type safety. The use of the **decorator pattern** via `@cache` demonstrates a clean way to separate performance concerns from business logic. Practical Examples - **Financial Systems**: Storing every transaction (credit/debit) instead of just the balance to provide a perfect audit trail. - **E-commerce**: Tracking how long items sit in a cart before being removed to analyze user hesitation. - **Gaming**: Building a replay system by storing player inputs as events to recreate the match exactly. Tips & Gotchas - **Schema Evolution**: If you change the structure of your `Item` object later, your old events might break. You must plan for "upcasting" (transforming old events into the new format) or versioning your event schemas. - **Snapshotting**: For systems with millions of events, replaying from zero is too slow even with local caching. Periodically save a "snapshot" of the state so you only have to replay events from the last snapshot forward. - **Avoid for CRUD**: If your application only requires basic create, read, update, and delete operations without any need for history, event sourcing will introduce unnecessary complexity.

Refactoring From Messy to Idiomatic Writing code that works is only the first step. True software craftsmanship lies in making that code readable, maintainable, and idiomatic. In Python, we call this writing Pythonic code. Writing Pythonic code means using the language's built-in strengths rather than fighting against them. We want to avoid overengineering, reduce boilerplate, and write logic that feels natural. This guide steps through how to refactor a clunky, rigid fitness tracking script into clean, beautiful Python. We will replace unnecessary classes with clean functions, implement safe resource management, and apply modern standard library tools. Prerequisites and Toolkit To follow this tutorial, you should understand basic Python syntax, functions, and file operations. We will use several built-in modules to clean up our code: * dataclasses: Simplifies class creation for data storage. * pathlib: Provides an object-oriented approach to handling file paths. * logging: Offers a robust way to track events instead of using print statements. The Code Walkthrough Let us look at how to structure our program. We start by removing a class that has no state and replacing it with focused functions. Then, we use a data class to structure our inputs and an iterator to stream data safely. ```python from dataclasses import dataclass, field from datetime import datetime from collections.abc import Iterator from pathlib import Path import logging Centralize paths as constants using Path objects FOOD_FILE = Path("food.csv") ACTIVITY_FILE = Path("activities.csv") Setup logging configuration logging.basicConfig( level=logging.INFO, format="%(asctime)s - %(levelname)s - %(message)s" ) def today() -> str: return datetime.now().strftime("%Y-%m-%d") @dataclass class Entry: description: str calories: int date: str = field(default_factory=today) def append_entry(file_path: Path, entry: Entry) -> None: # Using a context manager ensures safe file closing with open(file_path, "a") as f: f.write(f"{entry.date},{entry.description},{entry.calories}\n") logging.info(f"Appended entry: {entry.description} ({entry.calories} cal)") def read_entries(file_path: Path) -> Iterator[Entry]: # Easier to Ask Forgiveness than Permission (EAFP) try: with open(file_path, "r") as f: for line in f: parts = line.strip().split(",") if len(parts) == 3: yield Entry(parts[1], int(parts[2]), parts[0]) except FileNotFoundError: logging.warning(f"{file_path} not found. Starting fresh.") def run_day_summary(date: str) -> None: food_list = list(read_entries(FOOD_FILE)) activity_list = list(read_entries(ACTIVITY_FILE)) # List comprehensions filter and aggregate data cleanly food_total = sum(e.calories for e in food_list if e.date == date) activity_total = sum(e.calories for e in activity_list if e.date == date) net_calories = food_total - activity_total print(f"--- Summary for {date} ---") print(f"Food intake: {food_total} calories") print(f"Activity burn: {activity_total} calories") print(f"Net: {net_calories} calories") def main() -> None: # Clear entry point avoiding global namespace pollution banana = Entry(description="banana", calories=100) append_entry(FOOD_FILE, banana) running = Entry(description="running", calories=300) append_entry(ACTIVITY_FILE, running) run_day_summary(today()) if __name__ == "__main__": main() ``` This refactored script is modular. We replaced manual file closing with context managers. We replaced repetitive manual parsing with a reusable generator function that yields clean data objects. Syntax and Best Practices This refactoring highlights several Pythonic paradigms. First, we favor **EAFP** (Easier to Ask Forgiveness than Permission) over **LBYL** (Look Before You Leap). Instead of checking if a file exists before opening it, we try to read it and catch the `FileNotFoundError`. Second, we use **default factories** in our data class. If you write `date: str = today()`, Python evaluates that default value exactly once when it loads the module. Every entry created thereafter gets that same frozen timestamp. By using `default_factory=today`, Python executes our function every single time we instantiate a new `Entry`. Finally, we use an **Iterator** via the `yield` keyword. Yielding records one by one keeps memory consumption minimal, even when handling large files. Tips and Pitfalls Avoid creating classes that do not hold state. If a class only contains a constructor and a few methods that do not modify `self`, delete the class. Write pure functions instead. Do not use print statements for operational tracking. Prints clutter standard output and make debugging difficult. Standardize on the built-in logging library to gain timestamps and severity levels automatically.

Overview of Reliable Unit Testing Unit testing validates the behavior of small, isolated code fragments, typically individual functions or methods. These tests serve as a critical safety net during refactoring, ensuring that modifications in one area don't inadvertently break unrelated logic. Beyond catching bugs, well-crafted tests act as a live specification of how your system should behave. While Python includes a built-in `unittest` module, the industry standard has shifted toward pytest due to its readable syntax and powerful feature set. Prerequisites To follow this guide, you should have a solid grasp of Python fundamentals, including classes and decorators. Familiarity with the `pip` package manager and basic command-line operations is necessary. You should also understand the basics of HTTP requests, as our examples involve simulating API interactions. Key Libraries & Tools - **pytest**: A framework that simplifies writing small tests while scaling to support complex functional testing. - **httpx**: A next-generation HTTP client for Python used in our examples to fetch weather data. - **unittest.mock**: A standard library module that allows you to replace parts of your system under test with mock objects. Code Walkthrough: Handling External Dependencies Testing code that relies on external APIs, such as a `WeatherService`, is challenging because you cannot perform real HTTP requests during a unit test. You must replace the network call with a controlled response. Implementation with Monkey Patching Monkey patching dynamically replaces a function at runtime. In the following example, we replace `httpx.get` with a fake version that returns pre-defined data. ```python import pytest import httpx from weather import WeatherService def test_get_temperature_with_patch(monkeypatch): def fake_get(url, params=None): class FakeResponse: def raise_for_status(self): return None def json(self): return {"current": {"temp": 19}} return FakeResponse() monkeypatch.setattr(httpx, "get", fake_get) service = WeatherService(api_key="test_key") assert service.get_temperature("Amsterdam") == 19 ``` Implementation with MagicMock The `MagicMock` class from `unittest.mock` offers a cleaner alternative to manual fake classes. It allows you to define return values and verify how many times a method was called. ```python from unittest.mock import MagicMock, patch def test_with_mock(): mock_response = MagicMock() mock_response.json.return_value = {"current": {"temp": 25}} with patch("httpx.get", return_value=mock_response) as mock_get: service = WeatherService(api_key="test_key") temp = service.get_temperature("Utrecht") assert temp == 25 mock_get.assert_called_once() ``` Syntax Notes: Pytest Features - **Fixtures**: Use the `@pytest.fixture` decorator to define reusable setup code. Passing the fixture name as an argument to a test function automatically injects the object. - **Parametrization**: Use `@pytest.mark.parametrize` to run the same test logic against multiple sets of data without duplicating code. - **Exception Testing**: Use `with pytest.raises(ExceptionType):` to verify that your code handles errors correctly. Refactoring for Testability Direct dependencies on libraries like httpx make testing rigid. By using **Dependency Injection**, you pass a client object into the `WeatherService` constructor. This allows you to swap the real HTTP client for a mock client during testing without ever touching monkey patches. Good design and testability go hand in hand; if a function is hard to test, it usually needs a better architectural approach. Tips & Gotchas - **Python Path**: Ensure your `pyproject.toml` includes the correct `pythonpath`. Without it, pytest may fail to find your local modules. - **Assert Quantity**: Aim for one logical assertion per test to maintain clarity. If a test fails, you should know exactly why immediately. - **Skip & XFail**: Use `pytest.mark.skip` for temporary bypasses and `pytest.mark.xfail` for known bugs that are being tracked but not yet fixed.

Overview Object instantiation often starts simple but quickly descends into a chaotic "monster constructor." When a class requires numerous optional parameters, flags, or nested structures, standard initialization becomes fragile and unreadable. The Builder Pattern solves this by separating the construction of a complex object from its representation. It allows you to build an object step-by-step, ensuring the final product is both valid and, ideally, immutable. Prerequisites To follow this guide, you should have a solid grasp of Python fundamentals, including classes and methods. Familiarity with Data%20Classes and the concept of immutability will help you understand why we often separate the builder from the final product. Key Libraries & Tools * **dataclasses**: Used for creating clean, concise data models with built-in methods. * **typing**: Essential for implementing Self-typing to enable fluent API method chaining. * **http.server**: A built-in Python module used in the infrastructure bonus to preview generated content. Code Walkthrough Step 1: The Product First, define the core object. We use a frozen data class to ensure that once the builder "finishes" the object, it cannot be modified accidentally. ```python from dataclasses import dataclass, field @dataclass(frozen=True) class HTMLPage: title: str body: str metadata: dict[str, str] = field(default_factory=dict) def render(self) -> str: meta_tags = "".join([f'<meta name="{k}" content="{v}">' for k, v in self.metadata.items()]) return f"<html><head>{meta_tags}<title>{self.title}</title></head><body>{self.body}</body></html>" ``` Step 2: The Builder The builder maintains the state during the construction phase. By returning `self` in each method, we enable a fluent API. ```python from typing import Self class HTMLBuilder: def __init__(self): self.title = "" self.body_content = [] self.metadata = {} def add_title(self, title: str) -> Self: self.title = title return self def add_heading(self, text: str) -> Self: self.body_content.append(f"<h1>{text}</h1>") return self def build(self) -> HTMLPage: return HTMLPage(title=self.title, body="".join(self.body_content), metadata=self.metadata) ``` Syntax Notes Notice the use of `typing.Self`. This allows methods to return the instance itself, enabling the "dot-chaining" syntax (e.g., `builder.add_title("Hi").add_heading("Welcome")`). This pattern transforms procedural code into a more declarative, readable style. Practical Examples You encounter the Builder Pattern frequently in established libraries. Pandas uses it for styling data frames, and Matplotlib employs it to assemble charts layer by layer before calling `plt.show()`. It is the gold standard for generating HTML, SQL queries, or complex JSON configurations. Tips & Gotchas Avoid the builder for simple objects with only two or three fields; it adds unnecessary boilerplate. The primary risk is forgetting the final `.build()` call, which results in holding a builder instance instead of the desired product. Use this pattern when your object reaches five or more optional fields.