Inspired by the power of human collective intelligence, where great achievements arise from the collaboration of diverse minds, we believe the same principle applies to AI. Individual models possess unique strengths and biases, which we view as valuable resources for collective problem-solving.

AB-MCTS (Adaptive Branching Monte Carlo Tree Search) harnesses these individualities, allowing multiple models to cooperate and engage in effective trial-and-error, solving challenging problems for any single AI.

Our initial results on the ARC-AGI-2 benchmark are promising, with AB-MCTS combining o4-mini + Gemini-2.5-Pro + R1-0528, current frontier AI models, significantly outperforming individual models by a substantial margin.

This research builds on our 2024 work on evolutionary model merge, shifting focus from “mixing to create” to “mixing to use” existing, powerful AIs.

At Sakana AI, we remain committed to pioneering novel AI systems by applying nature-inspired principles such as evolution and collective intelligence.

We believe this work represents a step toward a future where AI systems collaboratively tackle complex challenges, much like a team of human experts, unlocking new problem-solving capabilities and moving beyond single-model limitations.

Algorithm (TreeQuest): github.com/SakanaAI/tre...

The Continuous Thought Machine (CTM) incorporates neuron-level temporal processing and neural synchronization, moving beyond current AI limitations.

Interactive Paper (with web-demo): pub.sakana.ai/ctm/
Full Paper: arxiv.org/abs/2505.05522
GitHub Project: github.com/SakanaAI/con...

1/ Transformer-Squared: Self-adaptive LLMs

Paper: openreview.net/forum?id=dh4...

Transformer-Squared adapts its weights on the fly for each query, achieving strong performance across tasks and enabling parameter-efficient life-long learning.

2/ Agent Skill Acquisition for Large Language Models via CycleQD

Paper: openreview.net/forum?id=Kvd...

CycleQD is an ecological niche-inspired model-merging approach that achieves great performance on computer science tasks while retaining language capabilities.

3/ An Evolved Universal Transformer Memory

Paper: openreview.net/forum?id=s1k...

Neural Attention Memory Models (NAMMs) is an evolved memory system trained to improve performance and efficiency on language transformers, and zero-shot transferring to vision and RL foundation models.

4/ TAID: Temporally Adaptive Interpolated Distillation for Efficient Knowledge Transfer in Language Models

Paper: openreview.net/forum?id=cqs...

TAID is a novel knowledge distillation method that uses a time-dependent intermediate distribution addressing common challenges in distilling LLMs.