Gen AI: climate saviour or atmospheric carbon spewing vortex?
One of the advantages of living more years in the world is having a decent set of heuristics that help you decide just how worried to get about a potentially worrying thing. But my set of heuristics have been coming up empty recently on the topic of ‘just how concerned should I be about the increasing demands for energy consumption from the remarkably power hungry foundational models?’.
People are getting quite factional about it already and I’ve seen camps forming on both sides of the argument:
using AI extensively will be more efficient than humans doing the work and hence, while widespread use of Generative AI will put more carbon into the atmosphere, the net effect will be absolute carbon saving. All hail AI!
the increased processing load of augmenting everything under the sun with carbon-heavy calls to ChatGPT et al will both place added demands on our finite supplies of electricity and water (particularly in poorer countries where the local populations have less agency and political clout) and pump more carbon into the atmosphere. It’s a disaster!
So this week I decided to go behind the sensational headlines and see what I could pull together for myself.
What follows is the notes I’m now building my useful new heuristic on. Not a complete answer but hopefully a useful starting point for further investigation as your personal interests and passions guide you.
Electricity, carbon and embodied carbon
First up, I needed to do some jargon untangling. Why do we talk about the carbon footprint of AI? As with so many phrases we lean on, this is shorthand!
Carbon is often used as a broad term to refer to the impact of all types of emissions and activities on global warming.
For example, 1 ton of methane has the same warming effect as about 84 tons of CO2 over 20 years, so we normalize it to 84 tons CO2eq. We may shorten even further to just carbon, which is a term often used to refer to all Green House Gases.
Green Software Practitioner website, April 2024
Today, using electricity (almost) always produces carbon emissions. Running software and particularly running compute-intensive software like AI therefore produces carbon emissions both in training and in inference.
In addition to the carbon (emissions) due to using electricity to run your software, there is also the concept of embodied carbon (which has been coming up a fair bit recently in the discussions around the vehicle fleet transition to EVs).
Rather than a measure of how many carbon atoms in the device in question, it is the amount of carbon emitted during the creation and disposal of the device.
Just how much embodied carbon can be attributed to the rise in AI is again difficult to pin down. Mainly I think because it’s a really tricky thing to measure well. For instance, part of the embodied carbon of a GPU will be determined by the carbon intensity (electricity source mix) of the electricity used to produce it. The impression I got from some fairly persistently searching and reading, is that it’s not unreasonable to expect that the embodied carbon of a typical end user device is the large majority of total CO2 emissions over the device lifecycle.
So it’s likely that the embodied carbon of the GPU / TPU servers in the big cloud warehouses is also a significant percentage. For me, the takeaway is to be a good steward of any hardware you use and work hard to increase utilisation and extend the useful lifespan.
So keep in mind that the numbers in the next section deliver an incomplete picture because they only look at operational electricity usage. And this is often true in discussion you read about the carbon footprint of AI.
Operational electricity usage
As the human species seeks to ‘electrify everything’ and along the way clean up the carbon emissions of electricity generation, there are a lot of energy uses competing for that green electricity pool. How big is the demand from AI in relative terms?
Again not a trivial stat to uncover but the Electricity 2024 - analysis and forecast to 2026 from the International Energy Agency has some useful detail.
We estimate that data centres, cryptocurrencies, and artificial intelligence (AI) consumed about 460 TWh of electricity worldwide in 2022, almost 2% of total global electricity demand.
Electricity consumption from data centres, artificial intelligence (AI) and the cryptocurrency sector could double by 2026.
Could double. So rise to around 4%? Not to be sneezed at but not enormous.
Although remember, that isn’t taking into account embodied carbon in the servers needed to accomodate that potential doubling of demand. NVIDIA and others are working hard to make their servers more energy efficient, potentially at the risk of making them less carbon efficient.
As an aside, it’s quite striking how often Ireland comes up in studies and comparisons of greening your AI energy usage. Data centres made up around 16% of total electricity usage in Ireland in 2022.
Not all AI is Gen AI
One argument I have seen repeated many times is that AI will provide the solution to climate change so we shouldn’t be concerned about any extra carbon burden on the way to that outcome. This has niggled at me for a while because I just couldn’t see how Generative AI - the most power hungry version of AI - was going to help us with climate change.
While I imagine some folk are actually thinking along the lines of
LLM → AGI → climate solution from super intelligence
many of these types of conversation actually seem to stem from category confusion.
I have read convincing arguments to the effect that AI will help with climate change through better grid control and better weather forecasting. Basically we can get more clean energy and more effective utilisation of that clean energy by using extensive real time data sets and machine learning to optimise distribution and creation.
This seems very plausible to me (although funding the real time data collection needed is a story for another day and a heinously complicated one politically) but as far as I can tell, the ML models that are likely to be useful in these types of problems are not the kind of AI that is massively driving up power consumption. Grid optimisation and weather forecasting are almost certainly best addressed by task-specific models trained only on directly relevant data and focussed on solving a particular problem. Meanwhile it is ‘general purpose’ models trained on internet scale data that are spiking current and forecast energy usage.
Exact numbers are super hard to come by but I had a crack at benchmarking the carbon burden of training AlphaFold, a task specific model not wholly unlike what might be trained for weather forecasting and GPT4, a foundation model aimed at being generally useful. The carbon footprint was different by two orders of magnitude. And that’s just on the training side of things (see next section).
So at the moment, my opinion is that there’s no justification for the ‘AI will solve climate change so why worry?’ stance.
Inference is the new power guzzler
It’s important when making your carbon budgets to be aware that Generative AI models have flipped the paradigm where the training phase took the lions share of resources (money, carbon, water) and the inference phase was comparatively insignificant. With the current transformer architecture, foundation models have weighty training requirements AND weighty costs for inference.
There’s a positive to pull out of this point though. While most individuals and companies can have zero impact on how ‘climate aware’ the training of foundation models is, any development team using calls to foundation model in their software can make design choices that can reduce the inference side carbon impact.
A useful primer that I discovered during my reading is this introductory training course from The Green Software Foundation. As well as offering techniques you can apply within your application design to build carbon aware software, it summarises key concepts and cuts through some fairly confusing jargon.
If you’re designing and building software I highly recommend taking a read. If nothing else, you’ll gain a great vocabulary for talking and thinking about climate aware, whole of lifecycle software design.
De-carbonisation commitments and new tracking tools from the mega vendors
While I hope those of my readers with the opportunity to shape application development will add the patterns of green software to their design toolkit, that’s going to be a pretty small drop in a very large bucket (but you still should! please!).
So I was really happily surprised to read about the on-the-face-of-it impressive commitments to reducing the carbon impact of their electricity consumption made by Microsoft, Google and AWS.
Documentation of this abounds including in beautiful technicolour annual reports driven by ESG requirements, but the devil can be in the detail here so to illustrate the point, let’s dive into Microsoft’s 100/100/0 commitment.
By 2030 Microsoft will have 100 percent of our electricity consumption, 100 percent of the time, matched by zero carbon energy purchases.
Made to measure: sustainability commitment progress and updates, July 2021
Fair enough that unless you build the grid and all the power sources contributing to it, there are limits to what you can do to keep your power carbon free.
Hopefully the Microsoft commitment means they will increasingly enter into power purchasing agreements (PPAs) that fund the development of new renewable energy resources. Some of this is definitely already happening.
PPAs are typically very long-term contracts. A renewable plant can find financing with one of these agreements since it already has had a buyer for its electricity for many years.
PPAs encourage something called additionality. Purchasing a PPA drives the creation of new renewable plants. PPAs are a solution that gets us towards a future where everyone has access to 100% renewable energy.
Green Software Practitioner website, April 2024
The other finicky detail is how far down the rabbit hole they are prepared to go with the granularity of matching. The gold standard seems to be 24/7 hourly matching which can require both spatial demand shifting and demand shaping
Google, who has been working on data centre efficiency for many years (and using it as a great PR oppo for pretty much as long) can now show some fairly impressive stats on data center power usage effectiveness (PUE) a ratio that describes how efficiently a computer data center uses energy; specifically, how much energy is used by the computing equipment (in contrast to cooling and other overhead that supports the equipment). (Wikipedia)
The text of the report reads “compared with five years ago, we now deliver approximately three times as much computer power with the same amount of electrical power.” Good stuff, even if driven by their own bottom line as much as by a climate imperative.
These commitments of course don’t address the embodied carbon in all the new servers and data centres. Which as we learned above, can be a very significant contributor to the ‘full lifecycle’ carbon impact. This is called out as a future focus area in the 81 page 2022 Environmental Sustainability Report from Microsoft.
Let’s keep watching. Because of course, these are all only statements of intent and we might see an erosion of commitment if Generative AI workloads make both electricity demand and the embodied carbon of all the new servers soar. One to watch and, if you have significant purchasing power, an area where making your focus on these factors known to your cloud vendor(s) is a way to have an impact.
Another way to vote with your feet is to become an early adopter of Azure Carbon Optimization, the AWS Customer Carbon Footprint Tool or Google’s Carbon Footprint. Disclaimer that I haven’t used any of these services but I’m thrilled to see them emerge, even if they are really crappy right now and mostly focussed on backwards facing reporting, because any level of transparency that promotes choice and action will facilitate more.
Hopefully tools that make it for easier for all of us to do things like spatial demand shifting (moving computation loads dynamically to cleaner power) will emerge in the near future.
Final thoughts
Understanding the true carbon footprint of the AI explosion is really, really hard. Mostly because it’s an irreducibly complex problem and partly because it’s not necessarily in any company’s best interests to be too transparent.
I learned a lot writing this cheatsheet for myself and at the same time feel like I barely scratched the surface. Understanding the AI impacts of water usage feels like a blindspot of equal magnitude.
Where I’ve landed at the moment is that I’m not super worried about the increasing carbon footprint of AI. That’s influenced by my own belief that widespread adoption will be slower than forecast. If we actually hit a ‘transformative AGI-esque’ breakthrough, I’ll definitely be thinking again. The manufacture and disposal of inorganic robot bodies, for instance, is not something I’ve dug into at all above.
I don’t see the need for specific legislation to investigate the environment impacts of AI either. Enforcing exisiting legislation feels like better bang for our collective buck.
But I will be championing carbon aware development and purchasing habits and thinking harder about embodied carbon which seems like the one that gets swept under the rug the most.
What did I miss? What do you think Would love to learn more from you all in the comments.