Researchers found that AI training memory causes models to forget old information
New research shows how common optimizers inadvertently erase previously learned knowledge while trying to improve performance.
A new research paper identifies a surprising reason why language models lose what they have already learned during fine-tuning. Even when a model is being updated to improve, it can paradoxically discard old answers it previously mastered.
Modern AI training relies on optimizers like Adam, which use momentum to guide learning. Momentum keeps a record of past updates to make training smoother. However, the study shows that this historical record can act against the model’s interests.
Together, these findings reveal how an optimiser's memory can erode learned behaviour even as its current gradient acts to preserve it.
Imagine you are studying for a second exam while trying to keep the information from your first one. If you focus too hard on your old notes, you might get confused, but if you ignore them completely, you forget everything. This AI research shows that the optimizer is essentially doing both at once.
The researchers discovered that the damage often comes from older gradients stored in the momentum memory. Recent updates actually try to protect the old information, but the accumulated history from the beginning of the new task pushes the model in the wrong direction.
The study suggests that clearing or adjusting this momentum history can help models retain their past capabilities better. Does this mean our AI models are currently tripping over their own memories as they learn new skills?
This research focused specifically on how optimizers interact with model memory during training. It does not account for all potential causes of forgetting in large models, and it specifically analyzed how momentum contributes to the decline of answer accuracy.
Source: arXiv cs.CL