MILO Automates Harness Discovery for Agentic Systems: Outperforms State-of-the-Art
MILO is a new framework that automates the discovery and optimization of agent harnesses by evolving both harnesses and their search strategies. It significantly outperforms existing state-of-the-art methods across multiple benchmarks, reducing human effort and resource usage.
Researchers have introduced MILO (Meta-evolutionary Island Orchestration), a new framework designed to automate the exploration and optimization of harnesses that coordinate AI models in agentic systems. Harnesses, which control execution and interactions with the environment, are crucial for achieving strong long-horizon performance but traditionally require significant human design effort. MILO addresses the combinatorial complexity of this task by simultaneously evolving both harnesses and the strategies used to search for them.
How MILO Works
MILO employs three main components: - Hierarchical lineage memory across island-based trees, which leverages both successful and rejected mutations to guide the search process. - Per-island mutator agents capable of rewriting entire harnesses using global search history and detailed feedback from parent harnesses. - An orchestrator that dynamically adapts the search by adjusting lineage structure, assigning mutators, and updating the search curriculum.
Performance and Benchmark Results
MILO-discovered harnesses outperform eight state-of-the-art harnesses and six search methods using both leading proprietary (Opus 4.8) and open-weight (gpt-oss-120b) models. On three major benchmarks—Terminal-Bench 2.1, PaperBench, and DeepSWE—MILO improves performance over initial harnesses by 12.0%, 28.3%, and 10.3% respectively, compared to best prior automated search gains of 4.5%, 18.3%, and 0%. On Terminal-Bench 2.1, MILO achieves an 86.1 ± 2.0% resolution rate, surpassing the official leaderboard's top score (83.8 ± 2.3%) and using 26% fewer tokens than its initial harness.
What Developers Should Do
If you are building or researching agentic systems, consider employing MILO to automate and accelerate harness design. This can reduce manual workload and potentially uncover configurations that outperform hand-designed alternatives. Review detailed methodology, results, and implementation considerations in the official publication.