The Energy Cost of Training AI Models

Why training uses so much computing

Training is the process by which an AI system learns from examples. In machine learning, a model begins with adjustable internal values, called parameters, and a training process repeatedly adjusts them to reduce errors on example data. For small models, this can run on an ordinary computer in minutes. For the largest systems, such as large language models, it involves enormous numbers of calculations.

Three things drive the amount of computing needed:

1. Model size. Neural networks with more parameters require more calculations for each example.
2. Amount of training data. Larger datasets mean more passes of calculation. The role of training data in shaping a model is a separate topic, but quantity affects cost directly.
3. Number of attempts. Developers often run many experiments, tuning settings and testing variations, before producing a final model. Those earlier runs consume energy too and are easy to overlook.

The calculations are mostly repeated arithmetic, performed in parallel on specialized chips such as graphics processors or other accelerators. Thousands of these chips may work together for days or weeks on a single large run, housed in data centers with the power and cooling systems that implies.

From computing to energy

Energy is power multiplied by time. A chip draws a certain amount of power while running, and the energy used is that power times the hours it runs. Add the other hardware (memory, networking, host computers) and the facility overhead for cooling and power conversion, and you get the total.

Here is a purely hypothetical example to show the arithmetic, not a description of any real system. Suppose a training run used 1,000 accelerators, each drawing an average of 0.5 kilowatts, for 30 days.

• Power: 1,000 x 0.5 kW = 500 kW.
• Time: 30 days x 24 hours = 720 hours.
• Energy for the chips: 500 kW x 720 h = 360,000 kilowatt-hours.

If the facility had overhead that raised total consumption by, say, 20 percent, the total would be 360,000 x 1.2 = 432,000 kilowatt-hours. These figures are invented for illustration. Real runs vary greatly in size, hardware and duration, and the emissions would depend on the electricity source. The same kilowatt-hours produce very different emissions on a grid dominated by low-carbon sources compared with one relying on fossil fuels.

Why estimates are uncertain

Published figures for the energy of training particular models should be read with caution.

• Often not disclosed. Many developers do not publish the hardware, run time or number of experiments, so outsiders must estimate.
• Different boundaries. Some estimates count only the final training run. Others include experiments, data preparation, and the manufacturing footprint of the hardware.
• Efficiency changes quickly. New chips, software techniques and training methods can reduce the energy needed for a given capability. Meanwhile, models and datasets keep growing, so total demand can still rise.
• Different electricity mixes. Emissions depend on where and when the computing runs.
• Hardware utilization. Chips that sit partly idle waste energy; well-tuned systems waste less.

Because of this, comparisons between models, or between AI and other activities, are often rough. Beware of statements that give a single dramatic equivalent, such as comparing a model to a number of cars or households, without explaining the method.

Training is not the whole story

Training happens once for a given model version (plus later updates). Using the model, known as inference, happens every time someone sends a request. For a widely used service, the cumulative energy of inference can be large compared with the original training, though the balance depends on how popular the model is, how long it stays in use and how efficient the serving system is. Some analysts argue that inference now accounts for a major share of AI energy use, while others highlight training for the largest models. The honest answer is that it varies by system, and public data is limited.

Generative AI, which produces text, images, audio or video, is often more computationally demanding per request than traditional software such as a simple search, but how much more depends on the task and the model size. Image and video generation typically involve more computation than short text responses.

What can reduce the energy used

Several approaches are active areas of work, and their real-world impact varies:

• More efficient hardware. Chips designed for AI calculations can do more work per unit of energy.
• Smaller or specialized models. For many tasks, a smaller model may be good enough. Techniques such as compressing or distilling large models into smaller ones aim to keep much of the performance with less computing.
• Better algorithms and training methods. Improved techniques can reach the same quality with fewer calculations.
• Reusing existing models. Adapting a pretrained model to a new task usually takes far less computing than training from scratch.
• Careful experiment management. Avoiding wasteful reruns and stopping unpromising experiments early saves energy.
• Cleaner and better-timed electricity. Running jobs where and when low-carbon power is available can reduce emissions, although this depends on grid conditions and has practical limits.
• Efficient facilities. Cooling and power design affect overhead, as explained in our article on data centers.
• Using AI only where it helps. Not every problem needs a large model. A simple rule or a conventional method may do the job with far less computation.

Rebound and the bigger picture

Efficiency improvements do not guarantee lower total energy use. If AI becomes cheaper and easier to run, it may be used more widely, which can raise overall consumption. This is sometimes called a rebound effect. Whether it dominates is unknown.

It is also fair to consider potential benefits on the other side of the ledger. AI is used in some settings to optimize energy systems, forecast demand, support predictive maintenance or improve resource use in farming. Whether those benefits outweigh the energy AI itself consumes is hard to determine in general and should be evaluated case by case rather than assumed.

Questions to ask when you see a claim

1. Is the figure about training, everyday use or both?
2. Is it measured or estimated, and by whom?
3. Does it include experiments and hardware manufacturing?
4. What electricity source does it assume?
5. What is it being compared with, and is the comparison fair?

Where this leaves us

The energy needed to train large AI models is substantial for the biggest systems, but the exact amounts are often unknown outside the organizations that run them. The trend of the future is uncertain: efficiency gains and growing demand pull in opposite directions. Transparent reporting, clear methods and attention to both the costs and the benefits are the most useful ways to understand it. Discussions of responsible AI increasingly include environmental impact alongside fairness, safety and privacy.