Are AI deployments worth their carbon footprint?

Climate change awareness is an incentive for the general public to question the sustainability of new technology, and AI is no stranger to such scrutiny. In the present text we focus on AI technologies based on machine learning. Rather than performing a specific task, machine learning algorithms are designed to identify patterns in data and use these to make predictions. These are the backbone of many of today’s AI implementations (including ChatGPT)

CHAPTER 1

A brief overview of machine learning

A machine learning model is a mathematical representation of a process that can make predictions or decisions based on input data. It is created by training an algorithm on a dataset, from which it learns patterns and relationships within the data. Once trained, the model is used to make predictions on new, unseen data. To train the model means using the dataset to evaluate parameters so as to minimise the discrepancy between the outcome predicted by the model and that of the data.

The number of parameters is generally a good indicator of the size of the model, and, as we shall discuss later, the energy required for the training and later deployment. Larger models are not necessarily more accurate, as they can be subject to overfitting and hence become unreliable when required to generalise to unseen data. Model pruning is a popular technique that is often used to decide which parameters are less significant in order to delete them and obtain a more efficient model without too much compromise in accuracy.

To better understand the gist of machine learning, let us take ChatGPT as a concrete example. Specifically, ChatGPT is a large language model; that is, a machine learning model designed to mimic and generate human language text. The training data is a large amount of text from the internet, which the model uses to predict ‘next words’ by learning language patterns and grammar. In addition to this, the model is also trained via human feedback so as to acquire a more conversational tone when required or to remove undesirable responses/biases. This part of the training is called fine tuning and requires plenty of human power.

How big a model is ChatGPT? As far as large language models, the answer is very large. ChatGPT is based on a model called GPT-3 with around 175 billion parameters [5]. GPT-4 (its successor) is even larger, though the exact number of parameters has not been disclosed. To put this value into context, language models used to enhance Google searches (such as BERT and XLnet) have around 340 million parameters.

“While the number of processors in a deployment is generally not an indicator of its efficiency, it is an important figure to be taken into account when considering carbon footprint …”

These figures clearly indicate that deploying large machine learning models requires a great deal of computational power in addition to storage for large data sets. AI processors are in fact highly specialised (such as TPUs) or designed to run many parallel computations simultaneously (such as multicore CPUs or GPUs). While the number of processors in a deployment is generally not an indicator of its efficiency, it is an important figure to be taken into account when considering carbon footprint and, of course, the cost of an implementation. It has been estimated by a study that ChatGPT has been trained on possibly 1000 GPUs over the course of a month. Had it been run on a single GPU, it would have required about 300 years to train [3].

CHAPTER 2

The environmental impact of AI deployment

The figures from the previous section suggest that training a machine learning model, and hence using it for inference, can have large energy consumption. There have been numerous studies on the carbon footprint of deploying AI models, each considering different parameters but suggesting similar outcomes. A study done by an AI company called Hugging Face [2] has considered the carbon footprint of a large language model called BLOOM during its whole lifecycle. The training itself, released an amount of CO2 similar to that emitted by 30 flights from London to New York. This amount nearly doubled if the footprint of the required hardware and other computing infrastructure was taken into account. Note that the supercomputer used to train BLOOM was located in France, so the power was supplied by nuclear energy. Thus, deploying a model in a country where coal is the main source of energy would result in a significantly larger figure. Also, Bloom used state-of-the art GPUs which resulted in a relatively efficient deployment.

“What the scientific literature is currently short of are systematic studies of the comparison between the emissions incurred by performing a task using an AI model and using normal computational methods.”

In addition to this, one needs to consider the carbon footprint of inference: while a single inference may not be too costly energy-wise, repeated queries by millions of users eventually add up. In fact, a study by researchers at Google and UC Berkeley [4] concluded that up to 3/5 of emissions from AI deployments could be attributed to the inference process. They also outline some guidelines for best practices in order to reduce the emissions due to training the model. These include trying to opt for the most efficient processors available, relying on data centres that are transparent about their carbon emissions, as well as a few suggestions to machine learning engineer to produce more efficient models. According to the study, these practices could reduce the energy required during training by 100 times and carbon dioxide emissions by 1000 times.

What the scientific literature is currently short of are systematic studies of the comparison between the emissions incurred by performing a task using an AI model and using normal computational methods. Admittedly, these studies can be hard to carry out rigorously. Consider, for example, the study of the 3D structure of proteins. Alphafold is an AI program developed by Deepmind to predict the protein structure from the sequence of amino acids. Prior to this, it would take years to come up with such a structure (the duration of an entire PhD degree, to quote Demis Hassabis), whereas Alphafold can perform the computation in a matter of hours. In the long term, we can expect Alphafold to be more efficient than a PhD student, but it is hard to prove this with rigorous data. Finally, it is important to note that we have mostly discussed carbon emissions. However, there are other aspects in which AI technologies can be costly to the planet (for example, chips such as GPUs may require rare metals, or some data centres may use fresh water as a cooling agent). A study of this aspect of the environmental impact of AI is also necessary to understand the full picture.

CHAPTER 3

How can AI deployment help with the climate crisis?

The success of current AI deployments is due to its ability to process and reason with data in a quantity and speed practically impossible to humans or ordinary computation models. Data is central to understanding many phenomena in the natural world, including the current climate crisis. As a consequence, it has often been asked how can AI help in the context of climate change and if its elevated energy requirements are worth sacrificing.

Below we list and briefly explain a few deployments of AI (specifically machine learning) that are relevant to the climate crisis. To be clear, we are not stating that AI will solve all climate related problems. The information below is merely some evidence of how AI can help us understand and predict climate patterns, cope with some of the consequences of climate change and make more informed decisions on actions to be taken.

Energy efficiency

In 2016, Deepmind used a machine learning model to optimise the cooling system of Google’s data centre. By using historical data from the centre itself, an ensemble of neural networks was trained in order to understand the recommended actions to be taken. As a result, Google manage to reduce their energy consumption by up to 40%. Such a machine learning model seems to have potential in similar contexts such as improving power plant conversion efficiency (getting more energy from the same unit of input), reducing semiconductor manufacturing energy and water usage, or helping manufacturing facilities increase throughput. Importantly, it is a big step towards making AI technologies more sustainable, given the reliance on data centres.

Interpreting satellite images

Studying the changes in glaciers and movement of icebergs is important to understand effects of climate change on the the oceans and other environments. Using a deep learning model, researchers from the Universities of Leeds, Newcastle (UK) and Tromso (NO) have found a way to identify and map icebergs from satellite images [1]. Prior to this AI deployment, this task was painstakingly slow, for it is hard to distinguish icebergs from big waves, clouds or other unwanted items in the picture. The deep learning model accelerated the required time by a factor of 10.000 and an accuracy of 99%. Similar methods can also be used by ecologists to map deforestation (though not explicitly mentioned, it seems that similar deep learning models might be used by Space Intelligence, a mapping company based in Edinburgh).

Waste management

According to the US Environmental Protection Agency, waste is responsible for 16% of greenhouse gas emissions. Greyparrot, a software startup based in London, has developed an AI system that helps identify and recover waste material that can be recycled. On a similar note to the image processing mentioned in the previous paragraph, a Dutch company called Ocean Cleanup [6] is using deep learning models to conduct quantitative and qualitative analysis of microplastics at sea. Currently, the most common monitoring method to quantify microplastics at sea requires physical sampling using surface trawling and sifting for beached microplastics, which are then followed by manual counting and laboratory analysis. Manual counting is time-consuming, operator-dependent, and expensive, so deploying this methods consistency on a large scale is practically unfeasible. AI can replace this with a more reliable method to detect waste, predict hotspots and organise logistics for the cleaning.

Weather forecasting

Between February 2022 and April 2023, a series of paper mainly by academics working for large tech companies such as Deepmind and NVIDIA reported significant progress in the quality of machine learning based weather forecasts. It has been claimed that these forecasts have surpassed the standard set by the Integrated Forecast System (IFS), which is a widely used numerical system. What’s more, making a forecast with these models requires only a single GPU, takes less than a minute, and consumes a tiny fraction of the energy required for an IFS forecast (compare this with the 1000 GPUs used in ChatGPT). Though these ML models still rely on the IFS for the training phase, the forecast they produce seem to be at least as accurate. Furthermore, evaluation of a range of case studies produces a fairly consistent picture: data-driven ML models are brilliant in predicting extreme events, but can lack the intensity predicted by the IFS. This is not a fundamental problem with ML models, but seems to stem from the training methodology.

Our key takeaway

AI technology and climate change are both part of our present lives and will continue to be for the foreseeable future. While the deployment of AI models can be costly to the environment, these technologies can also reduce the energy costs and massively speed up procedures both in industry and academic research. These deployments have been partly obscured by the large language models such as ChatGPT that have dominated the latest AI boom, though their impact on the world is just as strong if ‘imperceptible’ to the general public.

“AI policy and climate policy have roles to play both to minimise environmental harm as well as help humans to deal with the consequences of climate change.”

So are AI technologies worth their environmental impact? The general answer would be a straight yes, since, as seen in the previous section, climate related issues such as weather forecasting and waste management can be dealt with very efficiently (both from a time and energy consumption point of view) by machine learning models. This can be stated with confidence only by assuming the transparency of the developers - without this, it is very hard to estimate the impact of a deployment, and determine with certainty whether its environmental impact is justifiable. Hence, AI policy and climate policy have roles to play both to minimise environmental harm as well as help humans to deal with the consequences of climate change. Details of a deployment that need to be released by the companies in order to understand its environmental impact include model type and architecture, training data, methodology and location, retraining frequency and practices, as well as functionality and intended users.

In addition, while democratising AI is an important step forward, those with expertise in AI, particularly people in power at tech companies, should establish ethical principles to limit the technology’s use. For example, large language models on their own should not be used as a reliable source for factual information. Hence, educating the general public on both the potential and limitations of current AI is an integral part to ensure that we all get the full benefits of the technology.

Academic references

[1] Braakmann-Folgmann, A., Shepherd, A., Hogg, D., and Redmond, E. Mapping the extent of giant antarctic icebergs with deep learning. The Cryosphere 17, 11 (2023), 4675–4690.

[2] Luccioni, A. S., Viguier, S., and Ligozat, A.-L. Estimating the carbon footprint of bloom, a 176b parameter language model. J. Mach. Learn. Res. 24 (2022), 253:1–253:15.

[3] Narayanan, D., Shoeybi, M., Casper, J., LeGresley, P., Patwary, M., Korthikanti, V. A., Vainbrand, D., Kashinkunti, P., Bernauer, J., Catanzaro, B., Phanishayee, A., and Zaharia, M. A. Efficient large-scale language model training on gpu clusters using megatron-lm. SC21: International Conference for High Performance Computing, Networking, Storage and Analysis (2021), 1–14.

[4] Patterson, D. A., Gonzalez, J., Holzle, U., Le, Q. V., Liang, C., Mungu´ıa, L.-M., Rothchild, D., So, D. R., Texier, M., and Dean, J. The carbon footprint of machine learning training will plateau, then shrink. Computer 55 (2022), 18–28.

[5] Radford, A., and Narasimhan, K. Improving language understanding by generative pre-training.

[6] Royer, S.-J., Wolter, H., Delorme, A. E., Lebreton, L., and Poirion, O. B. Computer vision segmentation model—deep learning for categorizing microplastic debris. Frontiers in Environmental Science 12 (2024).