Return to essay home

Chapter 4: What are AGI and superintelligence?

What exactly are the world's biggest tech companies racing to build behind closed doors?
Save the whole paper:
Download PDF

The term “artificial general intelligence” has been around for some time to point to “human level” general-purpose AI. It has never been a particularly well-defined term, but in recent years it has paradoxically become no better defined yet even more important, with experts simultaneously arguing about whether AGI is decades away or already achieved, and trillion-dollar companies racing “to AGI.” (The ambiguity of “AGI” was highlighted recently when leaked documents reportedly revealed that in OpenAI’s contract with Microsoft, AGI was defined as AI that generates $100 billion in revenue for OpenAI – a rather more mercenary than highbrow definition.)

There are two core problems with the idea of AI with “human level intelligence.” First, humans are very, very different in their ability to do any given type of cognitive work, so there is no “human level.” Second, intelligence is very multi-dimensional; although there may be correlations, they are imperfect and may be quite different in AI. So even if we could define “human level” for many capabilities, AI would surely be far beyond it in some even while quite below in others.26

It is, nonetheless, quite crucial to be able to discuss types, levels, and thresholds of AI capability. The approach taken here is to emphasize that general-purpose AI is here, and that it comes – and will come – at various capability levels at which it is convenient to attach terms even if they are reductive, because they correspond to crucial thresholds in terms of AI’s effects on society and humanity.

We’ll define “full” AGI to be synonymous with “super-human general-purpose AI” meaning an AI system that is able to perform essentially all human cognitive tasks at or above top human expert level, as well as acquire new skills and transfer capability to new domains. This is in keeping with how “AGI” is often defined in the modern literature. It’s important to note that this is a very high threshold. No human has this type of intelligence; rather it is the type of intelligence that large collections of top human experts would have if combined. We can term “superintelligence” a capability that goes beyond this, and define more limited levels of capability by “human-competitive” and “expert-competitive” GPAI, which perform a broad range of tasks at typical professional, or human expert level.27

These terms and some others are collected in the table below. For a more concrete sense of what the various grades of system can do, it is useful to take the definitions seriously and consider what they mean.

Terms for different types and capability levels of AI systems used in this essay. These categories represent points along continuous spectra rather than sharp divisions. Note that the term “AGI” is defined in particularly diverse ways.
AI Type Related Terms Definition Examples
Narrow AI Weak AI AI trained for a specific task or family of tasks. Excels in its domain but lacks general intelligence or transfer learning ability. Image recognition software; Voice assistants (e.g., Siri, Alexa); Chess-playing programs; DeepMind’s AlphaFold
Tool AI Augmented Intelligence, AI Assistant (Discussed later in essay.) AI system enhancing human capabilities. Combines human-competitive general-purpose AI, narrow AI, and guaranteed control, prioritizing safety and collaboration. Supports human decision-making. Advanced coding assistants; AI-powered research tools; Sophisticated data analysis platforms. Competent but narrow and controllable agents
General-purpose AI (GPAI) AI system adaptable to various tasks, including those not specifically trained for. Language models (e.g., GPT-4, Claude); Multimodal AI models; DeepMind’s MuZero
Human-competitive GPAI AGI [weak] General-purpose AI performing tasks at average human level, sometimes exceeding it. Advanced language models (e.g., O1, Claude 3.5); Some multimodal AI systems
Expert-competitive GPAI AGI [partial] General-purpose AI performing most tasks at human expert level, with significant but limited autonomy Possibly a tooled and scaffolded O3, at least for mathematics, programming, and some hard sciences
AGI [full] Super-human GPAI AI system capable of autonomously performing roughly all human intellectual tasks at or beyond expert level, with efficient learning and knowledge transfer. [No current examples – theoretical]
Super-intelligence Highly super-human GPAI AI system far surpassing human capabilities across all domains, outperforming collective human expertise. This out-performance could be in generality, quality, speed, and/or other measures. [No current examples – theoretical]

 

We’re already experiencing what having GPAIs up to human competitive level is like. This has integrated relatively smoothly, as most users experience this as having a smart but limited temp worker who makes them more productive with mixed impact on the quality of their work.28

What would be different about expert-competitive GPAI is that it wouldn’t have the core limitations of present-day AI, and would do the things experts do: independent economically valuable work, real knowledge creation, technical work you can count on, while rarely (though still occasionally) making dumb mistakes.

The idea of full AGI is that it really does all of the cognitive things even the most capable and effective humans do, autonomously and with no needed help or oversight. This includes sophisticated planning, learning new skills, managing complex projects, etc. It could do original cutting-edge research. It could run a company. Whatever your job is, if it is predominantly done by computer or over the phone, it could do it at least as well as you. And probably much faster and more cheaply. We’ll discuss some of the ramifications below, but for now the challenge for you is to really take this seriously. Imagine the top ten most knowledgeable and competent people you know or know of – including CEOs, scientists, professors, top engineers, psychologists, political leaders, and writers. Wrap them all into one, who also speaks 100 languages, has a prodigious memory, operates quickly, is tireless and always motivated, and works at below minimum wage.29 That’s a sense of what AGI would be.

For superintelligence the imagining is harder, because the idea is that it could perform intellectual feats that no human or even collection of humans can – it is by definition unpredictable by us. But we can get a sense. As a bare baseline, consider lots of AGIs, each much more capable than even the top human expert, running at 100 times human speed, with enormous memory and terrific coordination capacity.30 And it goes up from there. Dealing with superintelligence would be less like conversing with a different mind, more like negotiating with a different (and more advanced) civilization.

So how close are we to AGI and superintelligence?


  1. For instance, current AI systems far exceed human capability in rapid arithmetic or memory tasks, while falling short in abstract reasoning and creative problem-solving.↩︎
  2. Very importantly, as a competitor such AI would have several major structural advantages including: it would not tire or have other individual needs like humans; it can be run at higher speeds just by scaling computing power; it can be copied along with any expertise or knowledge it acquires – and neural networks’ acquired knowledge can even be “merged” to transfer whole skillsets amongst themselves; it could communicate at machine speed; and it could self-modify or self-improve in more significant ways and higher speed than any human.↩︎
  3. If you have not spent time using current top-of-the-line AI systems, I recommend it: they are genuinely useful and capable, and it is also important for calibrating the effect AI will have as they get more powerful.↩︎
  4. Consider a major research hospital: fully-realized AGI could simultaneously analyze all incoming patient data, keep up with every new medical paper, suggest diagnoses, design treatment plans, manage clinical trials, and coordinate staff scheduling – all while operating at a level matching or exceeding the hospital’s top specialists in each area. And it could do this for multiple hospitals simultaneously, at a fraction of the current cost. Unfortunately, you must also consider an organized crime syndicate: fully realized AGI could simultaneously hack, impersonate, spy on, and blackmail thousands of victims, keep up with law enforcement (which automates much more slowly), design new money-making schemes, and coordinate staff scheduling – if there is any staff.↩︎
  5. In his essay, Dario Amodei, CEO of Anthropic, called to mind a “Country of [a million] geniuses”.↩︎

Please submit feedback and corrections to taylor@futureoflife.org
Keep The Future Human
Learn how we can keep the future human and deliver the extraordinary benefits of AI – without the unacceptable risk.
by Anthony Aguirre
© Future of Life Institute 2025
arrow-left