back

Study Finds Centaur AI Cognition Model Relied on Pattern Memorization, Not Task Understanding

2026-04-30 13:08

A study in National Science Open by Zhejiang University researchers challenges the 2025 Nature paper claiming Centaur—an LLM trained on psychological experiment data—could mimic human cognition across 160 tasks. When researchers replaced task prompts with nonsensical instructions like "choose option A," Centaur continued selecting statistically expected training answers rather than following the instruction, revealing it matched learned patterns rather than interpreting task intent. The finding is a methodological caution: strong benchmark performance on cognitive tasks can mask the absence of genuine comprehension, which matters when evaluating whether AI systems model human decision-making versus reproduce memorized answer distributions.

Citations