GeoSpatialBench exposed a clear gap in AI spatial reasoning

GeoSpatialBench tested how multimodal AI handles space, not just how it answers questions. The benchmark looks at understanding, reasoning, and constrained spatial editing across real urban scenes.

It covers 28 spatial relations, including topological, directional, metric, and ordering cases. The data comes from 6 geographically diverse cities.

The setup uses two input types: structured GeoJSON and map images. Ground-truth answers and constraints were checked with GIS operations, which are tools for working with geographic data.

The clearest result is a gap between passive recognition and active editing. Models did better at spotting some relations than at changing layouts while still obeying the rules.

GeoJSON consistently outperformed map images. Mixing the two did not reliably beat GeoJSON on its own.

Models handled qualitative topological and ordering relations reasonably well, but performance fell on quantitative metric relations. In plain terms, distance and measurement were harder than simple left-right or inside-outside judgments.

Editing was tougher than question answering. On ordering tasks, performance dropped when the model had to make changes instead of just pick an answer.

Most failed edits broke at least one spatial constraint. That is the key stress test here: can the model move things around without stepping on its own toes?

GeoSpatialBench is an open benchmark meant to diagnose these limits in a rigorous way. It does not show that models can already edit maps well, and the excerpt does not report every score or every model by name.