Skip to content
Cyberttak.
· 3 min read

Pilots, doctors and students lost skills after leaning on automation, an essay on AI says

Macquarie University's Michael Richardson argues that AI learns from finished work, not the messy process behind it, and that people can suffer the same gap.

By CyberttakPublished · 23:54

In brief

AI copies thinking's leftovers, not thinking

A Macquarie professor says AI trains on finished work, not the process behind it. He cites pilots, clinicians and students who lost skills. About ten minutes of AI help was enough in one study.

A pilot's hands on the control yoke of an airliner cockpit, with the flight instrument panel ahead and a cloudy view through the windshield
A pilot's hands on the control yoke of an airliner cockpit, with the flight instrument panel ahead and a cloudy view through the windshieldCredit: AI-generated · GPT Image 2

An essay in The Conversation argues that today's AI systems learn from the leftovers of thinking, not from thinking itself. It was written by Michael Richardson, a professor in the School of Psychological Sciences at Macquarie University. He says the essay draws on a new paper he co-wrote, though that paper was not available to check.

The core idea is simple. A painting in a gallery, a studio recording or a published book is a finished product. The bad takes, messy drafts and dead ends are gone. Richardson calls what remains a trace of the real process.

AI models are trained on those traces. So they produce traces too: output shaped like the result of thought, but not necessarily made the way thought is made.

Reasoning models and agents

Some newer models print out their steps. Richardson says that printed reasoning is just another trace, and it often does not match the computation that actually produced the answer.

Agentic systems, where software wrapped around a language model runs a loop and feeds results back, get slightly closer. They add some real trial and error. Even so, he says, the chain of thought in the middle is still a trace.

He sees the gap as widest in creative work. Painters and musicians describe being changed by accidents, like paint running or an instrument sounding off. AI models never meet that kind of friction.

The human version

The essay says people face a similar problem when they lean on automation. In 1997, an American Airlines captain warned that pilots were becoming "children of the magenta line", following the flight computer so closely that manual flying skills decayed.

It also cites AI-assisted colonoscopy. Clinicians detected fewer polyps without the software than they had before adopting it. In one classroom example, high school maths students with unrestricted generative AI beat their peers while they had the tool, then scored lower once it was removed. One study found about ten minutes of AI help was enough to make people perform worse alone and give up sooner.

Richardson's suggested answer is to bring more process into the picture, including building AI that works in more human-like ways.

These are the author's arguments, not independent reporting. Every copy found traces back to the same article, and the author discloses funding from a Macquarie University 2025 education grant. The linked paper was not verified, and the available text cuts off before the full recommendations.

Sources