Researchers released a new benchmark to test AI queries on document databases
The new AptMQL-Bench provides a reliable way to evaluate how well AI models convert natural language into database commands.
A research team recently introduced AptMQL-Bench, a new testing tool designed to see if artificial intelligence can effectively talk to document-oriented databases like MongoDB. These databases are common in modern tech, but they are often hard for non-experts to query without knowing specific technical languages.
Previously, efforts to test AI on these systems relied on simple conversions from SQL, the standard language for older, table-based databases. However, these older benchmarks often failed to work correctly, dropped significant amounts of data, or created slow, inefficient queries.
Realistic text-to-MQL generation remains challenging.
A better way to test
The new benchmark consists of 21 databases and over 3,000 requests. Instead of just copying old methods, the researchers built a pipeline using AI agents to design database structures based on how people actually access the information.
This approach ensures that the data stays intact and that the system performs well as it grows. The researchers hope this will push developers to build smarter interfaces that let anyone pull information from complex databases just by asking.
Current performance
When tested on the high-end Claude Opus 4.5, the model achieved a 57.38% success rate without outside help. That number rose to 70.34% when the model was given access to extra information.
Is it time to start talking to your databases in plain English instead of code? Even with these improvements, the system is not perfect. The researchers noted that generating these queries remains a difficult task for current AI, especially when handling complex data structures.
In this story
Source: arXiv cs.CL