Beyond Prompt AI Studio
Understanding AGI

The gap to today: what today's models still lack

"What is AGI, really?" showed that on the level-based scale, the best models today sit at level 2, "broad assistance" - they help with many tasks, but under close human guidance. This module makes concrete what separates level 2 from level 3, "generalized competence".

Four examples - worth remembering

Try it yourself: within vs. outside the training distribution

Within the distributionOutside the distribution

A coding assistant gets a new task.

> A standard refactor, the kind that appears thousands of times in training data.

Reliably solved.

Gap 1: generalization outside the training distribution

Today's models excel at recognizing patterns similar to what they've seen in large quantities. On tasks clearly outside that pattern, performance drops unpredictably - not gradually, but often abruptly. A coding assistant that reliably handles standard refactors can produce plausible-sounding but wrong suggestions for an unusual, rarely documented architecture decision, without noticing it itself.

Gap 2: continual learning without forgetting

A human learns from every new experience without losing prior knowledge. Today's models are essentially frozen after training: new knowledge either arrives through an expensive, infrequent retraining process, or gets fed in as context for the current conversation only (see "How does a language model actually 'think'?") - gone again the next day. Retraining models so they learn without overwriting older knowledge remains an unsolved research problem.

Gap 3: reliability across long task chains

"Agentic AI: from chatbot to autonomous coworker" already explained the error chain: failure probabilities compound over many steps. An agent that's 95% reliable per step ends up with roughly a one-in-three success probability after 20 steps. This gap is a main reason autonomous agents today mostly stay limited to short, tightly scoped task chains.

Gap 4: grounding in the physical world

"World models: on the path to physical understanding" describes the sim-to-real gap: a system that shines in simulation often behaves surprisingly fragile in the unpredictable real world. A language model can describe every step of a recipe precisely without ever having understood how much force it takes to crush an egg.

Gap 5: reliable self-assessment

"Where AI hits its limits" showed that a model often sounds just as confident giving a wrong answer as a right one. Knowing what you don't know is its own capability - and today's models mostly lack it. That makes automated oversight harder: a system that doesn't reliably signal its own uncertainty is hard to monitor automatically (see "Human in the loop: when AI needs approval").

Why this matters for you as a decision-maker

These five gaps aren't a reason to avoid AI - they show where human oversight is still needed today. Knowing them lets you target where AI already works reliably (narrow, tightly scoped tasks close to the training data) instead of waiting for a generality that doesn't exist yet.

The key points

  • On the level scale from "What is AGI, really?", the best models today sit at level 2, "broad assistance" - not level 3, "generalized competence".
  • Five concrete gaps separate today's state from AGI: generalization outside the training distribution, continual learning without forgetting, reliability across long task chains, physical grounding, and reliable self-assessment.
  • The error chain in long tasks is especially practical: even small per-step error rates compound over many steps into low overall success rates.
  • Today's models are essentially frozen after training - new knowledge only comes through retraining or the current context, not through ongoing learning.
  • These gaps aren't a reason to avoid AI - they show where human oversight is still needed today and where AI already works reliably.

Quick check: did that make sense?

1 / 3

Which level from "What is AGI, really?" do the best models today sit at, according to this module?

Want to find out where AI already works reliably in your business - and where human oversight is still needed?