Turing winner David Patterson: how to give AI a bad carbon footprint

Key Concepts:

• Carbon Footprint of AI
• Four M's of Emissions (Model, Machine, Mechanization, Maps)
• Power Usage Effectiveness (PUE)
• Life Cycle Analysis (LCA)
• Greenhouse Gas Protocol
• Location-Based vs. Market-Based Emissions Accounting
• Energy Attribute Certificates (EACs) - Bundled vs. Unbundled
• 24/7 Carbon-Free Energy
• Contrails
• Jevon's Paradox

1. How to Give AI a Bad Carbon Footprint (and How to Avoid It)

• Bad Approach:
  • Pick the biggest model (e.g., Meta's Llama 405B parameter model). The assumption is "bigger is better" based on scaling laws.
  • Use an older GPU (e.g., Volta). Older GPUs use less power while operating (e.g., 300W), but are less efficient overall.
  • Train in a local, inefficient data center with a high Power Usage Effectiveness (PUE) (e.g., PUE of 2, meaning half the energy is used for cooling and distribution).
  • Train in a location with high carbon intensity (e.g., West Virginia, which relies on coal).
• Good Approach:
  • Pick the most efficient model (e.g., Google's Gamma 327B). This model achieves similar quality to Llama with significantly fewer floating-point operations.
  • Use the latest GPU or TPU. Newer hardware is faster, reducing the overall training time and energy consumption.
  • Utilize efficient cloud data centers with low PUE (e.g., 1.1, meaning only 10% of energy is used for cooling and distribution).
  • Train in a location with low carbon intensity (e.g., the Northwest, which uses hydroelectric power).
• The Four M's of Emissions:
  • Model: Number of floating-point operations (FLOPs) required to train the model. Llama requires approximately 40 septillion (3.8 * 10^25) FLOPs, while Gamma requires about 2 septillion.
  • Machine: Energy consumption per FLOP, determined by the hardware used.
  • Mechanization: Power Usage Effectiveness (PUE) of the data center.
  • Maps: Carbon intensity of the energy grid in the training location (kilograms of CO2 per megawatt-hour).
• Quantitative Comparison:
  • Training Llama in West Virginia with an older GPU and inefficient data center results in approximately 240,000 metric tons of CO2 emissions.
  • Training Gamma in the Northwest with a newer TPU and efficient data center results in approximately 150 metric tons of CO2 emissions.
  • The "bad" approach is approximately 1,600 times worse than the "good" approach.

2. Fallacies in Carbon Footprint Accounting

• Fallacy 1: Newer Hardware is Worse:
  • A life cycle analysis (LCA) of AI accelerators (TPUs) from cradle to grave (materials, manufacturing, transportation, operation, recycling) reveals that newer TPUs are significantly better.
  • Google's study compared five TPUs, breaking them into versatile (V4I, V5I, 6E) and powerful TPUs.
  • Versatile TPUs show a factor of three improvement in carbon emissions over two generations.
  • Powerful TPUs show a factor of 1.2-1.3 improvement.
  • The Greenhouse Gas Protocol defines location-based (utility emissions) and market-based (purchased clean energy) accounting methods.
  • Even with both accounting methods, newer TPUs demonstrate a dramatic reduction in emissions.
• Fallacy 2: Unbundled Energy Attribute Certificates (EACs) are Equivalent to Direct Clean Energy Use:
  • Allegory: Max wins an organic food competition by buying cheap organic breakfast receipts from San Diego, rather than eating organic food locally.
  • Unbundled EACs allow companies to purchase clean energy certificates from anywhere in the same continent, regardless of where their energy is consumed.
  • Academics have found that unbundled EACs do not effectively encourage local utilities to invest in carbon-free energy.
  • Financial Times article reveals that major hyperscalers (Amazon, Microsoft, Facebook) use a significant fraction of unbundled EACs. Amazon almost 60%, Microsoft more than half. Google is at 0%.
  • Microsoft can claim zero operational carbon footprint using unbundled EACs, but this does not reflect actual environmental impact.
  • Google advocates for 24/7 carbon-free energy, ensuring that carbon-free energy is used locally and continuously. 160 organizations have embraced it.
• Fallacy 3: Data Centers and AI are Major Consumers of Global Electricity:
  • The International Energy Agency (IEA) estimates that data centers accounted for 1.2-1.5% of global electricity consumption in 2022.
  • Continued economic growth, electric cars, and air conditioners will have a much larger impact on electricity consumption.
  • Air conditioners alone will consume more electricity than data centers.
  • AI is a small fraction of data center energy consumption.
  • While AI's global impact is limited, it can have a significant local impact if data centers are concentrated in one area.

3. AI's Potential Upsides: Contrail Reduction

• Aviation accounts for 2.5% of worldwide emissions, with contrails contributing over a third of these emissions.
• Google uses AI to steer planes to avoid humid regions, reducing contrail formation.
• Experiments with American Airlines showed a 0.3% increase in fuel consumption but a significant reduction in contrails.
• Widespread adoption of this technology could save 180 million tons of CO2, five to ten times the emissions of the top four hyperscalers.

4. Personal Actions and Advice

• Google Flights provides carbon emission estimates for flights.
• Wireless chargers are less energy-efficient than wired chargers.
• "Vampire power" from chargers left plugged in consumes unnecessary energy.
• Advice for developers:
  • Prioritize family and happiness over wealth.
  • Find something you love to do.
  • Be optimistic.

5. Notable Quotes

• "Energy is physics, but emissions is accounting."
• "Bigger is better" (referring to model size, as a potentially flawed assumption).
• "I was wrong, you were right, I love you" (the nine magic words for a long relationship).

6. Synthesis/Conclusion

The carbon footprint of AI is a complex issue with many nuances. By optimizing the four M's of emissions (Model, Machine, Mechanization, Maps), using newer hardware, avoiding unbundled EACs, and focusing on accurate accounting, it is possible to significantly reduce the environmental impact of AI. Furthermore, AI can be used to address other environmental challenges, such as reducing contrails from airplanes. It's crucial to have accurate numbers and consider both the upsides and downsides of AI to work on the right problems and avoid making mistakes.