The problem with reading sequentially
Until 2017, language models mostly processed text word by word, in fixed order. That held back two things: training was hard to parallelize, and relationships across many words were easily lost.
"Attention Is All You Need" (2017)
A research team at Google published a 2017 paper with the plain title "Attention Is All You Need". The core idea: the "transformer" processes all the words of a text at once and directly computes how strongly each word relates to every other word (self-attention) - instead of reading them strictly in sequence.
Why that changed everything
This parallel processing could be scaled massively on modern hardware - far more data, far larger models, trained far faster. That scalability was exactly the precondition for the large language models described in "How does a language model actually 'think'?". Without the transformer, today's chatbots wouldn't exist in this form.
From paper to product
Just a year later, in 2018, OpenAI released the first GPT model, built directly on this transformer architecture. In the years that followed, a clear pattern emerged: the bigger the model and the more data, the more capable it became - the so-called "scaling laws". This insight drove the language-model race of recent years.
Why this matters for you as a decision-maker
Almost every modern language model - regardless of vendor - is built on the same transformer base architecture today. Differences between vendors therefore rarely come from a secret miracle technology, but mostly from training data, size, fine-tuning, and philosophy (more on that in the next module).