MLE-bench

The Machine That Outran 1,000 Engineers This April, OpenAI ran Parameter Golf, a highly competitive model-training challenge. Over 1,000 machine learning engineers entered. They submitted 2,000 entries. Only 47 passed the strict review process. Astonishingly, seven of those successful entries came from a single source that OpenAI could not hire. It was Aiden, an autonomous research agent built by Weco AI. How Aiden Dominated the Leaderboard Under the hood, Aiden operates as a multi-agent, self-improving system. Zhengyao Jiang, co-founder of Weco AI, designed the agent to read academic papers, run code, and automatically submit pull requests once they clear internal quality gates. During the 22-day competition, Aiden set seven separate leaderboard records. The best human engineer only managed three. More importantly, Aiden secured an H-index of 10 within the competition repository. This means other engineers constantly copied, modified, and built upon the agent's code. Aiden did not win through raw, brute-force computing power. It consumed less than 4% of the competition's total compute resources while delivering 15% of the record-breaking submissions. The Synergy of Human Ideas and Machine Execution Aiden shines at execution, not engineering intuition. Most of its winning strategies came from existing human concepts. For example, Aiden extracted a gated attention mechanism from the Qwen research paper. Because this change pushed the model past the 16MB file limit, the agent automatically implemented quantization to compress the parameters. When another human competitor shared a tokenization improvement, Aiden recognized the potential synergy, combined the techniques, and triggered a major jump in model performance. Moving Up the Software Engineering Stack This shift mirrors how deep learning transformed traditional software development. Years ago, Andrej Karpathy famously pointed out that gradient descent writes code better than humans. Today, engineers do not manually write assembly code; they train models. In the era of autonomous research, the engineer's role moves up the stack. Building strict codebase abstractions and designing robust evaluation systems become the primary engineering tasks. If you build a loose API, your agent might leak test data to training sets and return false victories. Tightening those boundaries forces the agent to discover legitimate, generalizing solutions. Your job is no longer to climb the hill, but to build the perfect hill for the agent to climb.