Where the Energy and Water Go in a Data Center

When you send a prompt, the chip that writes the answer is only part of the story. The building around it also uses power: to keep spare machines ready, to move electricity around and to remove heat. And the power plant that feeds the building has its own footprint.

This lesson follows the energy and water from the power plant to your answer, so you can see why the "full" cost of a prompt is bigger than the chip alone.

If you want the hardware side in more depth (how chips work together, and how air and liquid cooling work), see AI Hardware Explained. This lesson focuses on what those things cost in energy, water and carbon.

What You'll Learn

• Why the chips are only part of a prompt's energy cost
• What "overhead" means in a data center, and how it is measured
• The difference between water used on site and water used to make electricity
• Why the same prompt can have very different carbon emissions in different places

The full chain behind one answer

Each step adds to the total. Studies that count only the AI chip give the smallest numbers. Studies that count the whole chain give larger ones.

The chips are only part of it

Google's 2025 study gives a clear example. It measured the median Gemini text prompt two ways:

• Counting only the active AI chips: about 0.10 Wh.
• Counting the whole system: about 0.24 Wh.

The full count adds the supporting processors and memory, machines kept idle so the service can handle sudden spikes, and the building's cooling and power systems. That more than doubled the number. This is why you should be careful with any figure that counts the chip alone.

Overhead and PUE

Data centers measure building overhead with a score called Power Usage Effectiveness (PUE).

• PUE = total energy used by the building ÷ energy used by the computers.
• A PUE of 1.0 would mean every unit of energy goes to computing, with zero overhead. That is not possible in practice.
• A PUE of 1.5 means that for every 1 unit used by computers, another 0.5 unit goes to cooling, power conversion and other systems.

Large, modern sites are designed to keep PUE low. Older or smaller server rooms often have much more overhead. When a company reports per-prompt energy, check whether it includes this overhead. Google's 0.24 Wh figure does.

Two kinds of water

Water use in AI comes in two main kinds.

Two more words help when you read reports:

• Withdrawal is water taken from a river, lake or underground source. Some of it goes back.
• Consumption is water that does not go back, mostly because it evaporates.

Evaporative cooling consumes water. That matters most in dry places, where the same water is needed by farms, homes and nature. This is why the location of a data center matters as much as its size.

Data centers can also use less water by choosing other cooling methods, such as closed loops that recycle the same water or cooling with outside air when the weather allows. But there is a trade-off: some low-water methods use more electricity. Engineers have to balance water against energy.

Why carbon depends on where you are

Electricity is not equally clean everywhere. A grid running mostly on wind, solar, hydro or nuclear power produces far less carbon per Wh than one running mostly on coal or gas.

A 2021 study by Google and UC Berkeley researchers found that the same AI training job could produce 5 to 10 times as much carbon in one place as in another, depending on which data center and grid it ran on, even within one country. The energy was similar. The carbon was not.

That leads to three practical points:

• Energy and carbon are not the same. A prompt can use the same energy in two places and have very different emissions.
• Time of day matters too. Solar power is only available in daylight, so the same grid can be cleaner at noon than at night.
• Company claims about "clean energy" vary. Some buy clean power for every hour they use. Others buy enough clean energy over a year to match their total, even if they use fossil power at some hours. Both are common, and they are not the same thing.

How much power does a whole site use?

Single prompts are small. Whole sites are not. The International Energy Agency (IEA) estimates that a typical AI-focused data center uses as much electricity as about 100,000 households, and that the largest ones under construction will use about 20 times more than that.

That scale is why data centers now matter to power companies, governments and local communities, which is the topic of lesson 4.

Key Takeaways

• The AI chip is only part of a prompt's energy use. Supporting machines, idle capacity and cooling can more than double it.
• PUE measures building overhead. Lower is better, and 1.0 is the theoretical limit.
• On-site water cools the data center. Off-site water is used at power plants and is often left out of company figures.
• Water use matters most in dry places, and some low-water cooling uses more electricity.
• Carbon depends on the grid. The same job can emit up to 5 to 10 times as much carbon in one place as in another.

Sources

• Google, Measuring the environmental impact of delivering AI at Google scale (2025)
• Patterson et al., Carbon Emissions and Large Neural Network Training (2021)
• International Energy Agency, Energy and AI: Executive summary (2025)
• Li, Yang, Islam and Ren, Making AI Less "Thirsty" (2023)