As AI Learns to Think and Reason, Will We Forget How To?

Of all AI dystopias imagined in science fiction, perhaps the most terrifying wasn't the militant rise of Skynet or the coldly calculated oppression of HAL-9000. It was the seemingly benign future depicted in Pixar's WALL-E—humans lounging in floating chairs, every need met by machines. What makes it truly unsettling isn't the presence of tyrannical AI overlords, but their complete absence. All it needs is our willingness to offload tasks, one after another, until we scarcely do anything for ourselves. We might dismiss the idea with 'That'll never be me,' but the decline doesn't arrive overnight.

With the 12 Days of openai coming to an end last Friday—capped off by the preview of their new frontier model o3—there’ll be a fresh wave of commentary predicting both utopia and dystopia due to AI over the coming weeks. Some two years ago, I would occasionally dwell on these articles, but nowadays I mostly see them as reflections of each author’s baseline optimism or pessimism. Still, whatever one’s viewpoint, the unprecedented capabilities of o3 remain largely undeniable.

The model has reportedly achieved benchmark-shattering feats that almost make it feel like AGI (Artificial General Intelligence). While there are many definitions of AGI floating around, what I mean by AGI is an AI that can learn or perform any intellectual task a typical human can, rather than being limited to a narrow domain. For instance, o3 scored 96.7% on the AIME 2024—a rather challenging math contest for top high-schoolers in the US, where a near-97% score exceeds even most elite contestants. It also attained 87.7% on GPQA Diamond, a measure of advanced scientific reasoning that typically demands “out-of-the-box” thinking. Beyond these, o3 also shows sophisticated problem-solving across multi-step logical benchmarks, well beyond simple question-and-answer tasks. For many if not most people, this signals the arrival of a new golden age of frictionless productivity and creativity, with mental drudgery handed off to AI. This enthusiasm is further supported by o3 demonstrating human-surpassing capabilities on the ARC-AGI benchmark (87.5% score). For context, ARC-AGI is designed to test an AI’s general intelligence across puzzle-like tasks requiring flexible reasoning rather than rote memorization. A high score there hints at something closer to broad reasoning skills.

While many are dazzled by those large numbers in those metrics, the performance I find most striking is o3’s leap on EpochAI’s Frontier Math—from a previous 2% solve-rate to 25.2%. Frontier Math is geared toward deep, puzzle-like reasoning in mathematics, meant to distinguish true “problem-solving intelligence” from mere pattern matching. The reason why this feels particularly interesting to me is because for the first time, I’m personally convinced that these models might be doing more than regurgitating data. While these models can produce step-by-step solutions that superficially resemble human reasoning, we should be cautious about equating their chain-of-thought outputs with genuine understanding. Just as a calculator showing intermediate steps doesn't truly 'understand' mathematics, these models may be executing sophisticated pattern matching rather than reasoning in any human-like way. They may have finally evolved beyond a “stochastic parrot”—a term coined by Emily Bender et.al to describe large language models that produce fluent text without genuine understanding. And implicit in that realization is an unsettling risk: this could be, most likely, the beginning of most of humanity gradually starting to lose our own reasoning skills as we hand these tasks over to AI.

This is the rather insidious danger: we'll let AI take over our thinking and reasoning, and in the process progressively cease to practice the very mental tasks that we trained the AI to do. I believe this danger is far more damaging than any potential fear of some rogue AI taking over the world, or an unaligned AI enabling some deranged individual, or the fear mongering around AI enabling some rogue nation.Each time we, as a collective, let technology handle a task, we quietly surrender abilities that we had learned and optimized over several hundred generations—one small convenience at a time. For example, I've personally observed stark differences in arithmetic prowess between kids raised in Asian systems (where calculators are discouraged) and those in Western settings (where calculators are routine). This pattern of technology reshaping our cognitive habits extends far beyond mathematics.

“The real danger isn't AI learning to think like us or even better than us, but humans willfully choosing not to.”

Beyond personal anecdotes, subtle though it may be, such analogous effect has been observed with several other areas. The so-called “Google effect” suggests that instant info-retrieval reduces our motivation to commit facts to memory. GPS removes our need to map cities in our mind, in stark contrast to London taxi drivers who famously develop extensive mental street layouts (and corresponding brain changes). Yes, typing speeds up writing, but handwriting can foster deeper retention and comprehension. Of course, not all psychological findings replicate seamlessly, and AGI proponents often downplay talk of “cognitive degradation.” But each of us recognizes the basic truth: use it or lose it. Neglect a language, and you’ll stumble through even simple sentences. Skip mental math for a year, and calculating a tip can feel oddly draining. When we stop exercising a muscle—mental or physical—it atrophies.

So, what if we begin to outsource genuine reasoning altogether? With o3 (and its successors) increasingly excelling at high-level tasks, our slope toward dependency will grow steeper, delegating complex math or scientific queries will seems logical and the prudent thing to do. These “thinking” AIs will become standard in every domain—drafting legal briefs, hypothesizing scientific theories, or analyzing market trends— humanity’s role will likely shrink to merely giving prompts, reading final outputs, and trusting in the AI’s prowess. Fast-forward a decade, and imagine a week-long internet outage or not having access to your devices that have these AIs on the edge.

Deprived of our digital assistants, we might experience something far beyond the discomfort of social-media withdrawal—a genuine inability to perform basic cognitive tasks. I've already started hearing anecdotal instances of students struggling to write or think clearly without ChatGPT. While these cases are currently rare, they portend our likely future: each small surrender of agency might seem rational in isolation, but collectively they represent a steady march toward intellectual dependence. From writing summaries to navigating streets to making decisions, we risk normalizing a complete dependence on AI systems. Like the humans in WALL-E, we might find ourselves not in floating chairs controlled by malicious machines, but willingly settled into them through our own gradual surrender of cognitive capabilities, one convenience at a time.

Let me be clear: this cautionary perspective isn’t some Luddite manifesto, nor am I that old cynic from The Simpsons yelling at passing clouds. The o3 model represents an extraordinary leap in AI research, likely to revolutionize fields from medicine to software engineering. I’m certainly not calling for us to halt this progress—nor could I even if I wanted to. The future won’t stand still, but by recognizing how our cognitive muscles might atrophy under constant automation, we can consciously protect our independence of thought. There’s even poetic irony in how AI helped refine parts of this text (Claude 3.5 sonnet, to be precise).

Yet as Plato warned, “The worst of all deceptions is self-deception.” Much like how most modern travelers rely on GPS while a few still navigate by the stars or paper maps, perhaps a committed minority will preserve these fundamental human faculties. Just as those traditional navigators keep alive an ancient art, some of us must sustain the art of reasoning. By deliberately choosing to exercise our minds alongside AI, we ensure that technology remains what it should be—a powerful ally. And when AI merely amplifies our intellect, rather than substitutes for it, we preserve the very essence of what makes us human.