Joe Reeve builds viral museum AI in two hours via vibe coding

Building a sophisticated AI application used to require a massive engineering effort, but Joe Reeve from ElevenLabs demonstrated a shift in the development paradigm. By utilizing a technique he calls vibe coding, Reeve constructed a viral "talk to statues" app in just two hours on a Sunday afternoon. This approach prioritizes rapid prototyping, high-level intent, and the seamless stitching of robust APIs over traditional, granular software engineering.

The technique relies on the developer acting as an orchestrator of complex models rather than a writer of boilerplate code. By using Cursor, an AI-native code editor, Reeve was able to "one-shot" the application’s core logic. The goal isn't just to write code faster, but to experiment with interaction patterns that were previously too expensive or time-consuming to explore. For developers, this means the "glue" between services—and the story you tell with that glue—becomes the most valuable asset.

Core Architecture of the Statue Voice Pipeline

The technical backbone of the application involves a multi-stage pipeline that transitions from visual input to interactive voice. The process begins when a user takes a photo of a statue. This image is processed through OpenAI to perform "deep research," identifying the subject and generating a detailed historical persona. This isn't just a factual summary; the model identifies what the voice of that specific historical figure should sound like based on their origin, era, and materials.

Once the persona is established, the data flows into the ElevenLabs Voice Design API. Unlike static voice selection, this API accepts a text-based description of a voice (e.g., "a weathered Egyptian Pharaoh with a deep, authoritative resonance") and generates a unique, matching audio profile on the fly. Finally, an ElevenLabs Agents is initialized with this voice and the research data, allowing the user to begin a phone-call-style conversation with the statue—all within roughly 30 seconds of snapping the photo.

Prerequisites and Integration Tools

To build similar conversational AI tools, developers should be comfortable with the following stack and concepts:

• Modern Web Frameworks: Proficiency in Next.js or similar frameworks for the frontend interface.
• API Orchestration: Understanding how to handle asynchronous requests across multiple providers.
• Vector Databases: Familiarity with Supabase or similar tools for managing user state and knowledge retrieval.
• Prompt Engineering: The ability to write precise system prompts that define agent personality and constraints.

Key tools mentioned in this workflow include Cursor for AI-assisted coding, ElevenLabs for audio generation and agentic orchestration, and OpenAI for the vision-to-text intelligence layer.

Navigating Voice Interface Challenges

While the technology allows for rapid deployment, voice as a primary interface introduces unique UX friction. One major hurdle is information density. Users can skim a diagram or a page of text in seconds, but voice is linear. Reeve notes that when an agent provides a long-winded response, users often feel frustrated by the inability to "skim listen."

// Conceptual snippet for handling interruptions in a voice agent flow
const handleUserInterruption = (timestamp) => {
  const currentTranscript = agent.getTranscript();
  // Edit the transcript to forget what the AI was about to say
  const shortenedTranscript = currentTranscript.slice(0, timestamp);
  agent.updateContext(shortenedTranscript);
  agent.stopAudioPlayback();
  agent.listenForNewInput();
};

Another significant issue is social politeness. Humans are conditioned not to interrupt, which can make interacting with a verbose AI agent feel awkward. Developers are now experimenting with "multimodal cues," such as visual indicators that show the agent wants to speak or a UI that displays the agent's internal thought process in real-time, allowing the user to interact with the data visually while continuing the conversation.

Practical Scaling and Production Readiness

A common critique of vibe coding is the difficulty of moving from a prototype to a production-ready system. Reeve argues that because the heavy lifting—agent management, voice synthesis, and research—is handled by scalable third-party APIs, the engineering challenge shifts to user management and "evals" (evaluations).

For museums like the British Museum or the Science Museum, scaling isn't just about traffic; it’s about curatorial control. Instead of relying on raw AI research, production versions involve giving curators a dashboard to edit system prompts and upload specific knowledge base files. This ensures the AI reflects the museum's official narrative rather than "hallucinating" historical facts from the open web. The future of these interactions likely lies in embedding the hardware directly—putting the microphone and speakers inside the statue itself—to remove the barrier of the smartphone screen entirely.