The artificial-intelligence (AI) industry is often compared to the oil industry: Once mined and refined, data, like oil, can be a highly lucrative commodity. Now it seems the metaphor may extend even further. Like its fossil-fuel counterpart, the process of deep learning has an outsize environmental impact.
In a new paper, researchers at the University of Massachusetts, Amherst, performed a life cycle assessment for training several common large AI models. They found that the process can emit more than 626,000 lbm of carbon dioxide equivalent—nearly five times the lifetime emissions of the average American car (and that includes manufacture of the car itself).
It’s a jarring quantification of something AI researchers have suspected for a long time. “While probably many of us have thought of this in an abstract, vague level, the figures really show the magnitude of the problem,” said Carlos Gómez-Rodríguez, a computer scientist at the University of A Coruña in Spain, who was not involved in the research. “Neither I nor other researchers I’ve discussed them with thought the environmental impact was that substantial.”
The paper specifically examines the model training process for natural-language processing (NLP), the subfield of AI that focuses on teaching machines to handle human language. In the last two years, the NLP community has reached several noteworthy performance milestones in machine translation, sentence completion, and other standard benchmarking tasks. OpenAI’s infamous GPT-2 model, as one example, excelled at writing convincing fake news articles.
But such advances have required training ever-larger models on sprawling data sets of sentences scraped from the Internet. The approach is computationally expensive—and highly energy intensive.
The researchers looked at four models in the field that have been responsible for the biggest leaps in performance: the Transformer, ELMo, BERT, and GPT-2. They trained each on a single GPU for up to a day to measure its power draw. They then used the number of training hours listed in the model’s original papers to calculate the total energy consumed over the complete training process. That number was converted into pounds of carbon dioxide equivalent based on the average energy mix in the US, which closely matches the energy mix used by Amazon’s AWS, the largest cloud services provider.
Read the full story here.
Find the paper here.
Don't miss our latest HSE content, delivered to your inbox twice monthly. Sign up for the HSE Now newsletter. If you are not logged in, you will receive a confirmation email that you will need to click on to confirm you want to receive the newsletter.
19 Jul 2020
- Austin, Texas
This one-day session will include a view of best practices for Carbon Management with a focus on fugitive methane reduction.
28 Apr 2020
- Brisbane, Australia
This one-day session features view on best practices for Carbon Management with a focus on fugitive methane reduction.
14 Sep 2020
- Bergen, Norway
The SPE Norway Subsurface Conference brings together the E&P industry to expand industry technical knowledge, connect with innovators and leaders and address technical challenges facing upstream oil and gas professionals.
7 - 10 Dec 2020
- Amsterdam, The Netherlands
Delivery for the Energy Challenge: Today and Tomorrow
4 Jun 2020 19:00
- Live, then On Demand
27 May 2020 20:00 BST
- Live, then On Demand
1 - 3 Dec 2020
- Banff, Alberta, Canada
Call for Papers
Submission Deadline - 31 May
HSE Now is a source for news and technical information affecting the health, safety, security, environment, and social responsibility discipline of the upstream oil and gas industry.
©2003-2020 Society of Petroleum Engineers, All Rights Reserved.