Researchers achieved real-time video generation on Trainium hardware

The Neuron Science team at Amazon collaborated with Reactor to optimize how generative AI models create video. They focused on running autoregressive diffusion models on AWS Trainium hardware. These models create video frames one by one, similar to how large language models predict the next word.

Streaming video requires intense processing. Each frame must be ready before the playback timer hits to avoid jerky motion or lag. The teams chose a model called Rolling Forcing to test their improvements. It can generate 16 frames per second, which is the standard for smooth video playback.

Generic software tools struggle with the unusual memory patterns required for this task. The teams used the Neuron Kernel Interface (NKI) to write custom code that communicates directly with the hardware. This allowed them to cut the processing time of specific memory operations from seconds down to milliseconds.

They also implemented a hybrid sharding strategy to distribute the heavy calculations. Self-attention, the mechanism that allows models to track relationships between different parts of a video, was split across multiple cores to prevent memory bottlenecks.

"For real-time workloads where every millisecond matters, that direct hardware access is what makes production-grade performance achievable."

Does this move us closer to seeing fully interactive, AI-generated worlds in our daily video games or creative software?

This approach currently works for specific model architectures and requires deep customization of hardware kernels. The researchers did not report if these techniques provide similar speed boosts across all types of generative video models or smaller hardware configurations.