Beyond Prompt AI Studio

Where AI comes from

The neural path: how deep learning broke through

"The idea before the technology: the origins of AI" showed how AI initially worked with hard-coded rules. In parallel, a small research direction pursued a completely different idea: systems that learn from examples instead of following rules. This path took much longer to win out.

Four things worth remembering

Try it yourself: symbolic vs. neural

Symbolic (rule-based)Neural (learning)

How does the system learn?

> Humans program rules and expert knowledge directly into the system (see module 30).

Reliable for clearly defined cases, but helpless in unexpected situations.

A second idea: learning from examples instead of programming rules

As early as 1958, Frank Rosenblatt introduced the "perceptron" - the basic building block of neural networks, loosely modeled on the structure of nerve cells. Instead of hard-coding expert knowledge as rules (as in the symbolic AI of "The idea before the technology: the origins of AI"), such a network was meant to learn patterns directly from example data.

An early disappointment

In 1969, Marvin Minsky and Seymour Papert showed mathematically, in their book "Perceptrons", that simple perceptrons fundamentally couldn't solve certain simple problems (the so-called XOR problem). That slowed research on neural networks for roughly a decade - its own small setback within the broader AI story.

Backpropagation makes deeper networks trainable

The comeback moment came in 1986: a team around David Rumelhart, Geoffrey Hinton, and Ronald Williams popularized "backpropagation", a method that let multi-layer ("deep") neural networks learn reliably too. But the idea remained practically limited for decades - it simply lacked the compute and data volume to be fully exploited.

2012: the ImageNet moment

The real breakthrough came only in 2012: a deep neural network called AlexNet (Alex Krizhevsky, Ilya Sutskever, Geoffrey Hinton) won the ImageNet image-recognition competition by a historic margin - made possible by graphics cards (GPUs) as compute power and huge, newly available amounts of image data. From here, "deep learning" became the dominant approach.

Why this matters for you as a decision-maker

The decisive shift wasn't a brilliant new idea, but the coming-together of a decades-old method with enough data and enough compute. The same lesson still applies today: progress in AI often comes from scaling proven approaches, not from constant new breakthroughs - a useful yardstick against "revolutionary" vendor claims (see "Spotting AI hype vs. real value").

Key takeaways

  • Alongside rule-based AI (see "The idea before the technology: the origins of AI"), the idea of neural, learning networks existed as early as the 1950s.
  • Early neural networks failed at simple problems (XOR) - that slowed research for a decade.
  • Backpropagation (1986) solved the training problem in principle, but stayed limited by a lack of compute for a long time.
  • The breakthrough came only in 2012 (ImageNet/AlexNet) - through enough data and GPU compute, not a new core idea.
  • Progress in AI often comes from scaling proven methods, not from constant new breakthroughs.

Quick check: did it land?

1 / 3

What slowed research on neural networks for roughly a decade starting in 1969?

Want to understand how today's AI landscape applies to your business?