Insights from the AnalogyAI team
We introduce FM-Bench, the Football Management Benchmark: an AI agent runs a football club for 20 in-game years under hidden information, cumulative consequences, a counter-adaptive market and a board that can fire it — alone against a scripted world, and head-to-head against 14 other models.
We introduce Grid2Matrix (G2M), a diagnostic benchmark that exposes a critical gap in Vision-Language Models: the inability to faithfully read out fine-grained visual structure, a failure we term Digital Agnosia.
We introduce VULCA-BENCH, a multicultural art-critique benchmark revealing that frontier VLMs suffer a 31–40 percentage-point drop from surface visual perception to deep cultural interpretation.
We introduce CFE-Bench, a multimodal STEM reasoning benchmark built from authentic university exams with instructor-verified solutions. Frontier models top out around 60%.