University of Toronto

The Quest for the Holy Grail of Chess AI For decades, chess engines like Stockfish have thoroughly outplayed humans. Yet, these systems lack the vocabulary to explain *why* a move works. They output cold numbers, not narrative. Conversely, large language models (LLMs) weave excellent narratives but struggle with spatial reasoning, regularly hallucinating illegal moves. Bridging this gap is what Munich-based developers at TNG Technology Consulting call the "holy grail" of chess programming. By combining LLM reasoning with precise programmatic chess tools, developer Stephan Steinfurt has built an AI agent capable of generating high-quality chess explanation videos. These videos are produced automatically every night and uploaded directly to YouTube. Giving the Model a Board and Legal Boundaries To overcome the limits of standard LLMs, Steinfurt's team wrapped Gemini 1.5 Pro with a suite of custom Python tools. The model does not merely guess moves; it interacts with an active virtual chessboard. * **The Legal Moves Tool**: This acts as a strict guardrail, preventing the LLM from ever suggesting illegal piece coordinates. * **The Checks, Captures, and Threats Tool**: This mimics beginner heuristics, forcing the AI to evaluate immediate tactical opportunities. * **Interactive Sandbox**: The agent can make moves, evaluate the resulting engine positions, and retract them to explore branching paths. With these tools, the agent acts as an analytical engine. It chooses which squares to highlight, where to draw visual guide arrows, and how to structure a compelling narrative around blunders and brilliant sacrifices. Translating Board State Into Human Excitement Once the agent completes its deep tactical analysis, it exports the game state to a proprietary video-generation pipeline. The narration relies on ElevenLabs for high-fidelity text-to-speech. By using advanced SSML-style audio tags like `<excited>`, the system injects human-like enthusiasm into the voiceover, avoiding the sterile, robotic tones of older chess software. Rather than chasing cheap viral tricks like exploding digital kings, the system focuses purely on chess quality. It can even draw on specialized engines like Maya—a neural network trained by the University of Toronto to mimic human play at specific Elo ratings—to evaluate how a human player, rather than an optimal computer, would handle a position. Scaling Chess Education for Everyone While elite chess streamers explain grandmaster games, they rarely have time to analyze amateur play. This automated pipeline democratizes the experience, making it possible to generate custom video summaries for everyday players. The system produces highly detailed analysis for roughly 20 to 30 cents per video, proving that automated, high-quality content production is both technically viable and remarkably cost-effective.