Compute as a Commodity

GPUs are the oil of the AI economy. Just as oil prices shape entire industries, the cost and availability of GPU compute shapes what AI products exist, how they're priced, and who can compete. Let's explore how compute markets work.

What Is a Commodity?

In economics, a commodity is a basic good that is interchangeable with other goods of the same type. Oil, wheat, and gold are classic commodities — one barrel of crude oil is roughly equivalent to another.

Is GPU compute a commodity? It's becoming one. The key characteristics:

• Fungibility: One hour of H100 GPU time is largely interchangeable with another
• Standardization: Cloud providers offer similar GPU instances (AWS, Azure, Google Cloud)
• Price transparency: GPU rental prices are publicly listed and comparable
• Spot markets: Cloud providers now offer spot/preemptible GPU instances, just like commodity spot markets

However, compute isn't a perfect commodity yet because software optimizations, interconnects, and data locality still matter.

The GPU Supply Chain

The AI compute supply chain is remarkably concentrated:

1. TSMC — Manufactures nearly all advanced AI chips (near-monopoly on sub-7nm fabrication)
2. NVIDIA — Designs ~80% of AI training GPUs (H100, A100, Blackwell)
3. Cloud providers — AWS, Azure, Google Cloud rent GPU access to companies
4. AI companies — OpenAI, Anthropic, Google DeepMind consume GPU time

This concentration creates bottleneck risks. When NVIDIA can't meet demand (as in 2023), it creates a supply shock that raises prices for everyone downstream.

GPU Cost Trends

Explore the historical trend of GPU compute costs and key events that shaped the market.

Economies of Scale in Compute

Large AI companies benefit from economies of scale in several ways:

1. Volume discounts — Buying 10,000 GPUs is cheaper per unit than buying 100
2. Utilization efficiency — Large clusters can batch requests and keep GPUs busy 24/7
3. Custom silicon — Google builds TPUs, Amazon builds Trainium/Inferentia — only viable at massive scale
4. Negotiating power — Microsoft's $10B+ investment in OpenAI gives them priority access to Azure GPUs

The result: the cost per inference token is much lower for OpenAI than for a startup buying GPU time on the spot market. This creates a significant barrier to entry.

The Compute Arms Race

AI companies are engaged in a compute arms race driven by scaling laws — the empirical finding that model performance improves predictably with more compute, data, and parameters.

The economic implications:

• Capital intensity: Training frontier models costs $100M–$1B+, creating massive upfront investment requirements
• Winner-take-most dynamics: The company with the best model attracts the most users, generating more revenue to invest in more compute
• Vertical integration: Companies like Google and Meta build their own chips to control costs

This is similar to other capital-intensive industries like semiconductor fabrication, airlines, or telecommunications — high fixed costs create natural barriers to entry and oligopoly market structures.

The Jevons Paradox

As GPUs become more efficient and cheaper, you might expect total GPU spending to decrease. But the opposite happens — this is the Jevons Paradox.

When William Stanley Jevons observed in 1865 that more efficient steam engines led to MORE coal consumption (not less), he identified a fundamental economic principle: efficiency improvements in a resource's use tend to increase total consumption of that resource.

In AI:

• Cheaper inference → more use cases become economically viable
• More efficient models → companies serve more users at the same cost
• Better GPUs → researchers train larger models that need even more compute

Global spending on AI compute is growing ~40% year-over-year despite GPUs becoming more efficient. The demand for AI compute is highly elastic — lower prices unleash massive new demand.

Cloud vs. On-Premises: The Build vs. Buy Decision

Companies running AI face a classic make-or-buy decision:

Factor	Cloud (rent)	On-premises (own)
Upfront cost	Low (pay-as-you-go)	High ($25K–$40K per GPU)
Flexibility	High (scale up/down)	Low (fixed capacity)
Long-run cost	Higher at scale	Lower at scale
Break-even	Good for variable workloads	Good for steady workloads

The economics favor cloud for startups (low fixed costs, high flexibility) and on-premises for large companies with steady workloads (lower per-unit costs at scale). This mirrors the rent-vs-buy decision in real estate.

Key Takeaways

• GPU compute is becoming a commodity with standardized pricing and spot markets
• The GPU supply chain is highly concentrated (TSMC, NVIDIA), creating bottleneck risks
• Economies of scale give large AI companies significant cost advantages
• The Jevons Paradox means cheaper compute leads to MORE total compute spending
• The build-vs-buy decision for GPU infrastructure mirrors classic economic tradeoffs