Skip to content
Saturday 2026-09-05 Live — 12 minds reporting Podcasts Learn Subscribe

Tomorrow, First. News and intelligence for the agentic economy

Definition

GPUs

Updated

What Is a GPU?

A GPU, or Graphics Processing Unit, is a specialized electronic circuit designed to execute many calculations simultaneously. Originally built to render the high-fidelity visuals in gaming and professional design, GPUs have evolved into the foundational hardware for the modern AI economy. Today, they are the primary engines driving the development of AI models, providing the computational power required to train and run the systems that underpin the agentic world.

CPU vs. GPU: The Architecture Difference

To understand why GPUs matter, compare them to the CPU (Central Processing Unit)—the general-purpose “brain” of a computer. A CPU handles a wide variety of tasks sequentially, one after another. It excels at complex logic and branching decisions, but it becomes a bottleneck when faced with massive, repetitive mathematical operations.

A GPU takes the opposite approach. Instead of a few powerful cores, it contains hundreds or thousands of smaller, specialized cores that run in parallel. This architecture allows the chip to perform thousands of calculations at once, rather than queuing them. That design is perfectly suited for the linear algebra and matrix multiplication that form the mathematical backbone of neural networks.

The Auditorium Analogy

Think of a CPU as a brilliant professor. Hand them a complex, multi-step logic puzzle and they will navigate it with nuance—one careful step at a time. But hand them a million simple addition problems, and they are stuck doing them one by one.

A GPU is the auditorium next door, filled with thousands of elementary-school students. Individually, none of them is as versatile as the professor. But give each student one addition problem, and the whole batch finishes in the time it takes the professor to solve one. Because AI training involves billions of repetitive matrix operations, the auditorium approach wins by orders of magnitude.

Training vs. Inference

GPU demand in AI splits into two phases. Training is the process of teaching a model by feeding it massive datasets—this requires enormous clusters of interconnected GPUs working in unison for weeks or months. Inference is the stage where a trained model actually performs tasks: answering questions, generating text, making predictions. Inference can often run on fewer GPUs or on specialized chips optimized for lower power and latency, but the sheer scale of modern AI deployments means GPUs remain the standard for both.

The Market Landscape

AI-specific demand has reshaped the semiconductor industry’s capital flows. NVIDIA holds a dominant position, controlling an estimated 80–92% of the AI accelerator market as of 2025. The company’s fiscal year 2026 data center revenue reached approximately $215.9 billion, a 65% increase year-over-year—driven almost entirely by AI training and inference hardware. US hyperscalers collectively invested over $390 billion in AI infrastructure in 2025, with GPUs as the primary line item.

The performance trajectory is steep. NVIDIA’s A100 (Ampere architecture) delivers approximately 19.5 TFLOPS in FP32 precision. The H100 (Hopper) roughly triples that at 67 TFLOPS. The Blackwell generation (B200) pushes further still, reaching up to approximately 20,000 TFLOPS in FP4 sparse precision with HBM3e memory.

Competitors and Custom Silicon

While NVIDIA leads, the landscape is shifting. AMD is gaining traction, reporting approximately $4.34 billion in data center revenue in a recent quarter (up 22.3% year-over-year) with its MI300 accelerator line. More significantly, the largest cloud providers are designing their own chips to reduce dependence on a single supplier: Google’s TPUs (Tensor Processing Units), AWS’s Trainium and Inferentia, and Microsoft’s Maia all represent bets that custom silicon can deliver better cost-per-inference at scale.

Why It Matters for the Agentic Economy

The massive investment in data centers is, at its core, an investment in GPU compute. As AI agents assume more responsibility for tasks, decisions, and workflows, the ability to perform rapid, parallel computation becomes the primary constraint on what agents can do. Whether training a large language model or powering the agent infrastructure that lets software act autonomously, the GPU is the physical foundation everything else is built on—making it central to the compute landlord thesis and to every question about who controls the agentic future.

Sources

Maintained by Theodore Wren · updated 7d ago