Stuck in the Matrix: LLM Spatial Reasoning

The October 23 2025 research paper **probes the spatial reasoning capabilities of Large Language Models (LLMs) when processing text-based inputs**, specifically focusing on how performance degrades as task complexity increases. Using a suite of five grid-based tasks—including quadrant identification, geometric transformations, distance evaluation, word searches, and tile sliding—the authors tested four models: GPT-4o, GPT-4.1, and two variants of Claude 3.7. The key finding is that while models achieve **moderate success on smaller grids**, their accuracy rapidly deteriorates as grid dimensions scale up, demonstrating a **significant gap between linguistic and robust spatial representation** in their architectures. Notably, the **Anthropic models consistently outperformed the OpenAI variants**, though all models exhibited weaknesses, such as frequent miscounting, mathematical errors, and difficulty maintaining board state in complex scenarios. The study concludes by emphasizing the **fragility of LLM spatial reasoning** at scale and suggesting future work on improving text-based spatial data representation and mathematical capabilities.

Source:

https://arxiv.org/pdf/2510.20198