Inference Tokens-per-Watt Calculator: Measure AI Energy Efficiency
Tokens per watt measures AI productivity: how much useful model output — tokens — a chip, server, or data center produces for every unit of electrical power it draws. It has become a key metric for evaluating the true efficiency and ROI of AI infrastructure, sometimes described as running an "AI token factory," because it captures real-world output rather than just theoretical compute. A facility that generates more tokens per watt delivers more product per dollar of electricity, which is the number that ultimately decides whether an inference business scales profitably. This calculator computes tokens per watt at both the chip and the facility boundary, folds in data-center PUE overhead, and translates the result into cost per million tokens — the line item that shows up on the cloud bill. Enter your throughput, power draw, PUE, electricity rate, and monthly volume to see live results.
1 ÷ (power × PUE ÷ tps ÷ 3600)
watt-hours to generate 1k tokens
at selected electricity rate
4,000 kWh at 50,000M tokens
serving cost at volume
Results update live as you type. For planning and field-check estimates — always verify against applicable standards and equipment ratings.
How we calculate this →The core formula
Tokens per watt is calculated at two boundaries — the chip (or server) and the facility — because each tells you something different about where energy is being spent.
Chip / server level: Tokens per Watt = Throughput (tokens per second) ÷ Power draw (watts)
Facility level: Tokens per Watt-Hour = Tokens generated in one hour ÷ Total facility power draw (watts), where facility power includes PUE overhead (cooling, power delivery, power conversion, and building losses).
The chip-level formula isolates the silicon: how efficiently the accelerator turns electricity into model output. The facility-level formula steps back to the meter — the grid power actually pulled to keep that chip alive, including every watt of cooling, UPS, and distribution loss along the way. Measuring at both boundaries is what makes tokens-per-watt a complete efficiency picture rather than a single optimistic number.
A worked example
Consider a node running an open-source model on an NVIDIA H100-class accelerator:
- Throughput: 2,000 tokens/sec - GPU power draw (TDP): 700 watts - Chip-level result: 2,000 ÷ 700 = 2.86 tokens per watt
Now apply a data-center PUE of 1.35: total facility power becomes 700 × 1.35 = 945 watts.
- Facility-level result: 2,000 ÷ 945 = 2.11 tokens per watt at the facility boundary
That gap — 2.86 down to 2.11 — is why PUE matters, and why tokens-per-watt is measured at both boundaries. A chip can look superbly efficient in isolation while a poorly run facility throws away a third of the energy before it ever reaches the workload. Expressed per watt-hour (the unit this calculator uses for energy cost accounting), the facility figure of 2.11 tokens/watt becomes 2.11 × 3,600 ≈ 7,600 tokens per watt-hour.
Why tokens per watt matters
Beyond raw hardware. Metrics like TFLOPS-per-watt only measure theoretical math capacity — how many floating-point operations a chip can perform per unit of energy. Tokens-per-watt captures the whole inference stack: model architecture, quantization (e.g. FP8 vs FP16), batch size, and software efficiency. Those are the things that determine how much raw compute becomes useful output, and they can differ wildly between two systems with identical hardware.
The PUE blindspot. A data center can post a near-perfect PUE of 1.0 and still waste enormous amounts of energy if its servers sit idle or run inefficient software. PUE measures overhead, not productivity. Tokens-per-watt tracks actual useful work — the tokens delivered — so it exposes waste that PUE alone hides.
Economic scaling. Infrastructure operators use tokens-per-watt to link electricity cost directly to revenue per million tokens. Once you know how many watt-hours a million tokens consume, every input — GPU choice, model size, quantization, batching, electricity rate — rolls up into the inference total cost of ownership (TCO) that decides whether a serving business is profitable at a given price.
How this calculator works
The math starts with one question: how much energy does one token cost? Power draw (watts) times PUE gives total facility power attributable to this serving node. Dividing by tokens-per-second gives watt-seconds per token, and dividing by 3,600 converts to watt-hours per token — the natural unit for energy cost accounting. Flipping that gives tokens per watt-hour, the efficiency headline.
Cost per million tokens is the commercial translation: multiply kWh per token by one million, then by the electricity rate. At $0.08/kWh and a modest 5,000 tok/s serving rate at 1,200 W with a 1.2 PUE, energy cost alone is roughly $0.046 per million tokens — a fraction of the API prices cloud providers charge, but a real and growing line item as inference volumes scale.
Monthly cost scales directly with volume. At 50 billion tokens per month (50,000 M), that same setup draws about 320 MWh and costs roughly $25,600 in electricity alone — before amortized hardware, bandwidth, or labor. Understanding this per-token energy cost is the first step toward optimizing model serving: smaller models, quantization, speculative decoding, and batching all improve the tokens-per-watt figure and reduce cost at scale.
Frequently asked questions
What is tokens per watt?
Tokens per watt is the efficiency metric for AI inference: how many tokens of model output a serving system generates for every watt of power it draws. The core formula is throughput in tokens per second ÷ power draw in watts, so a higher number means more useful AI output per unit of energy. It is becoming the standard metric for comparing GPU generations, model sizes, and serving configurations because it directly determines the electricity cost of running an AI service at scale. The facility-level variant, tokens per watt-hour, goes one step further by multiplying the chip's power draw by PUE to account for cooling, power conversion, and building overhead — giving the efficiency that actually shows up on the data center's power bill.
How do you calculate tokens per watt?
At the chip, divide throughput in tokens per second by the power draw in watts: tokens/watt = tokens/sec ÷ watts. A GPU generating 2,000 tokens/sec at 700 W delivers 2,000 ÷ 700 ≈ 2.86 tokens per watt. At the facility boundary, multiply the chip power by PUE to capture total grid power, then divide: tokens/watt = tokens/sec ÷ (watts × PUE). With a 1.35 PUE that same node reports 2,000 ÷ (700 × 1.35) ≈ 2.11 tokens per watt. To express it per watt-hour — the unit used for energy cost accounting — multiply the facility tokens-per-watt figure by 3,600 (seconds per hour): 2.11 × 3,600 ≈ 7,600 tokens per watt-hour. Cost per million tokens is then 1,000,000 ÷ tokens-per-watt-hour ÷ 1,000 × the electricity rate in $/kWh.
What is a good tokens-per-watt value?
There is no single universal target — it depends heavily on the GPU generation, model size, and quantization you run. A modern H100-class accelerator serving a mid-size open-source model in FP8 might deliver roughly 2–3 tokens per watt at the chip and around 1.5–2.2 tokens per watt at the facility boundary once PUE overhead is included. Older generations or larger models running in FP16 will be lower; newer accelerators, smaller or distilled models, and lower-precision formats like FP8 or INT4 push it higher. The useful comparison is year-over-year improvement within your own stack, benchmarked against the facility-level number that ties directly to your power bill.
Why is tokens per watt better than TFLOPS per watt?
TFLOPS per watt measures raw arithmetic throughput — how many floating-point operations a chip performs per unit of energy. But a model is not paid for FLOPs; it is paid (and valued) for the tokens it generates. Two systems with identical TFLOPS-per-watt can deliver very different tokens-per-watt depending on model architecture, quantization, batching, speculative decoding, and memory bandwidth — the things that determine how much raw compute becomes useful output. Tokens per watt captures the entire inference stack (silicon capability plus model and serving efficiency), so it correlates directly with the electricity cost of running an AI service. TFLOPS per watt remains useful for comparing hardware generations; tokens per watt is the metric that connects to the bill.
Why is inference energy becoming a bigger deal than training energy?
Training a model is expensive but one-time. Inference — answering user queries — runs 24/7 for the life of the product. As AI assistants, copilots, and agents become embedded in daily workflows, the inference fleet can dwarf the training cluster. Industry estimates suggest the ratio of inference to training compute will approach 10:1 or higher for widely deployed models, making per-token energy efficiency the dominant operational cost lever.
How much does it cost in electricity to serve a million AI tokens?
At a modern GPU node efficiency of roughly 5,000 tokens/second at 1,200 W (with 1.2 PUE and $0.08/kWh), electricity runs about $0.04–$0.06 per million tokens. Less efficient configurations or higher electricity rates can push this to $0.10–$0.20 per million tokens. The gap between this raw electricity cost and the $1–$15/million-token prices cloud providers charge covers hardware amortization, networking, redundancy, and margin.