Researchers developed a way to read massive documents with standard AI models

Large language models often struggle to process extremely long texts because their memory capacity is limited. A new research paper introduces Periscope, a technique that helps these models handle long documents without needing extra training or massive hardware upgrades.

Imagine you have a long book, but your AI can only read one chapter at a time. Instead of trying to force the whole book into its memory, Periscope cuts the text into smaller chunks and arranges them on a grid. The AI then looks at these specific sections to find the information needed.

By using this grid-based approach, a model can effectively search through millions of tokens. A 27B model using this method can handle a context of 4.5 million tokens while using only one 80GB GPU. Without this, a traditional system would require significantly more memory.

Tests on platforms like LongBench v2 and InfiniteBench show that this method often performs better than traditional reading approaches. It acts like a librarian who uses an index to find the right page rather than reading the entire library from cover to cover.

This technique works by creating an evidence map that identifies which parts of a document contain the relevant answers. It effectively replaces the need for a single, long pass through the data, which often causes models to lose accuracy.

Could this finally let us upload our entire personal archives to an AI assistant without slowing down our computers?

However, Periscope is specifically designed for tasks where the AI must find information within a large dataset. It does not change the core architecture of the model and was tested on document-based benchmarks rather than general conversation or creative writing tasks.