If you are old enough you remember posting to Usenet and the warning that would accompany each new submission:
This program posts news to thousands of machines throughout the entire civilized world. Your message will cost the net hundreds if not thousands of dollars to send everywhere. Please be sure you know what you are doing. Are you absolutely sure that you want to do this? [ny]
Maybe we meed something similar in LLM clients. Could be phrased in terms of how many pounds of atmospheric carbon the request will produce.
Individual LLM requests are vanishingly small in terms of environmental impact; inference providers use a lot of batching to do lots of work at once. Furthermore, LLMs and diffusion models are not the only ML workload. While generative AI tickles investors, most of the ML actually being deployed is more mundane things, like recommendation systems, classifiers, and the like; much of which is used for adtech purposes adversarial to that of users. If LLMs and diffusers were the only thing companies used ML for, but efficiency gains from new hardware remained constant, we'd still be at the 2017 baseline for environmental impact of data centers.
Likewise, I doubt that USENET warning was ever true beyond the first few years of the networks' lifetime. Certainly if everything was connected via dial-up, yes, a single message could incur hundreds of dollars of cost when you added the few seconds of line time it took to send up across the whole world. But that's accounting for a lot of Ma Bell markup. Most connections between sites and ISPs on USENET were done through private lines that ran at far faster speeds than what you could shove down copper phone wiring back then.
If what you're saying is true, why are we hearing about AI companies wanting to build nuclear power plants to power new data centers they think they need to build?
Are you saying all of that new capacity is needed to power non-LLM stuff like classifiers, adtech, etc? That seems unlikely.
Had you said that inference costs are tiny compared to the upfront cost of training the base model, I might have believed it. But even that isn't accurate -- there's a big upfront energy cost to train a model, but once it becomes popular like GPT-4, the inference energy cost over time is dramatically higher than the upfront training cost.
You mentioned batch computing as well, but how does that fit into the picture? I don't see how batching would reduce energy use. Does "doing lots of work at once" somehow reduce the total work / total energy expended?
> If what you're saying is true, why are we hearing about AI companies wanting to build nuclear power plants to power new data centers they think they need to build?
Well, partly because they (all but X, IIRC) have commitments to shift to carbon-neutral energy.
But also, from the article:
> ChatGPT is now estimated to be the fifth-most visited website in the world
That's ChatGPT today. They're looking ahead to 100x-ing (or 1,000,000x-ing) the usage as AI replaces more and more existing work.
I can run Llama 3 on my laptop, and we can measure the energy usage of my laptop--it maxes out at around 0.1 toasters. o3 is presumably a bit more energy intensive, but the reason it's using a lot of power is the >100MM daily users, not that a single user uses a lot of energy for a simple chat.
>If what you're saying is true, why are we hearing about AI companies wanting to build nuclear power plants to power new data centers they think they need to build?
Something to temper this, lots of these AI datacenter projects are being cancelled or put on hiatus because the demand isnt there.
But if someone wants to build a nuke reactor to power their datacenter, awesome. No downsides? We are concerned about energy consumption only because of its impact on the earth in terms of carbon footprint. If its nuclear, the problem has already been solved.
AI seems like it is speedrunning all the phases of the hype cycle.
"TD Cowen analysts Michael Elias, Cooper Belanger, and Gregory Williams wrote in the latest research note: “We continue to believe the lease cancellations and deferrals of capacity points to data center oversupply relative to its current demand forecast.”"
Because training costs are sky-high, and handling an individual request still uses a decent amount of energy even if it isn't as horrifying as training. Plus the amount of requests, and content in them, is going up with stuff like vibe coding.
They're incredibly unpopular because even when they're made revenue-neutral (meaning, everyone gets a refund check) people don't realize most of them would make money if they actually reduced their carbon.
They're incredibly unpopular because the ultraweathly use massive amounts of fossil fuels and thus lobby very, very hard against them...and make sure the public is often told just how evil they are and how expensive they'd hurt Johnny Everyday Worker, even car ownership, especially in a city (where much of the US populative lives) is not affordable to a large segment of the population.
If memory serves Jet A is not taxed at all federally in the case of for-profit corporations (while non-commercial users DO pay a tax!) and many states also either do not tax it or tax it very litte.
It's completely insane that we do not tax fuel usage for probably the most energy-intensive way to move people and/or goods and often that movement of people is entirely frivelous.
> They're incredibly unpopular because the ultraweathly use massive amounts of fossil fuels and thus lobby very, very hard against them...and make sure the public is often told just how evil they are and how expensive they'd hurt Johnny Everyday Worker, even car ownership, especially in a city (where much of the US populative lives) is not affordable to a large segment of the population.
Eh. It's not Bill Gates and Alice Walton. Sometimes the obvious answer is the real one: It's the fossil fuel industry.
> It's completely insane that we do not tax fuel usage for probably the most energy-intensive way to move people and/or goods and often that movement of people is entirely frivelous.
That one's just the arbitrage problem. Planes move around. If there is an international flight to a country that doesn't tax jet fuel (or taxes it less) then the plane is going to fly into LAX with enough fuel still in the tank to get back to the other jurisdiction and fill up again. Which actually increases fuel consumption because fuel is heavy and they otherwise wouldn't want to do that.
This is the same reason the EU doesn't tax jet fuel.
I find that message very curious because the message itself clearly does not cost much but the machines it is send on do. So the more messages that are send the less the messages will cost.
But "asking ChatGPT" is something people do so casually without there being any apparent clues of the costs associated with that. I guess that is true of pretty much everything online though.
Even driving your car around you at least are somewhat aware of the gas you are burning.
Batteries and EVs are the production side. Reducing demand is e.g. requiring you to drive fewer miles. You get a car that doesn't run on petroleum and CO2 goes down while vehicle miles can stay the same or go up.
I would bet most ppl drive around with very little awareness of how much it’s costing, either in money or environmental impact. Many people I’ve met seem to measure efficiency by how much it costs to fill up the tank.
Even more fundamentally, think about how much carbon running a cup of water from your faucet produces. No matter where you live, this is more carbon than an LLM prompt generates.
Or, even worse God forbid, think about how much carbon is produced to create a single bottle or carton of water. Then consider how casually people down bottles of water.
Can you source that? A lot of places have gravity fed reservoirs that are energy positive/neutral (LA, San Francisco and New York all rely on gravity fed reservoirs that were built before energy intensive pumping was practical). There are some costs but they are pretty small per gallon.
I’m assuming that calculation is amortizing the cost of running the water system (including wastewater treatment) adding the cost pumping it to the point that gravity can push it through the pipes. It’s never free.
That could be solved by charging more for the service. That is the only reason you are aware of the gas burning after all, you aren't conducting your own aq test you are noticing you are filling up twice a week at $50 a tank.
They're aware of the price they pay for the gas, not the emissions. I would wager that the mass ignorance of the impact of fossil fuels (and rubber on roads) that the broader population has is a significant reason why we're in this climate mess today.
I am 100% on the side of reducing pollutants — but this was never publicly seen as a major issue and I'm suspicious about the timing.
The oil industry is a conglomerate of degenerates spamming boomer logic all the way down to the workers. Their memes propagate throughout society and lead to the other boomer characteristic of rewriting personal and societal history.
The finger waggers now are being programmed to pretend they talked about tire particulates and the carheads are being programmed to pretend they never cared about 0-60. This another "We have always been at war with Eastasia", just like they all opposed the Iraq war from day 1 and didn't cancel the Dixie Chicks, et cetra.
This may have been discussed in specialist literature somewhere but even when I did ecology courses in university circa 2001ish, I never heard about tire particulates, while I did hear a lot about greenhouse gasses.
It's a concern but not a civilization ending concern like climate change. I low key resent these attempts to move goalposts to satisfy the writer's urge for negativity.
Consider that a bus has six to ten tires that each weigh around ten times more than a typical car tire. This is presented as the alternative to cars, is it even any different? Not implausible that it could actually be worse, especially if the bus isn't at full occupancy at all times.
Meanwhile the weight difference between EVs and petroleum cars is emphasized in the complaints, even though it isn't very large, while the much larger weight difference between any cars and buses is ignored. Because the point isn't to complain about tires, it's to complain about EVs.
And if the point actually was to complain about tires then you still wouldn't be talking about EVs, you would be talking about tires and how to make them shed less or construct them out of lower toxicity materials etc.
Last time I did the math, a Tesla Model Y only had 3x less tire emissions than a semi truck per distance traveled. City buses are on-par with a Tesla Model Y if you only care about mL/km tire wear.
The city bus uses tires with a harder rubber and dimensions such that the pressure at the road is less, plus its normal driving patterns have less wear than typical Tesla use.
To make those sorts of calculations easy, you can ignore all the pressure/usage/etc nonsense and just do basic math on tire dimensions (including min/max tread depth and width, not just radius, though I typically ignore siping and whatnot) and typical longevity. Volume lost per mile driven is basic high-school arithmetic, and the only real questions are regarding data quality and whether the self-imposed constraints (e.g., examining real-world wear rather than wear given optimal driving or something) are reasonable.
Well many of my fellow Americans would only accept an EV if it's gigantic, and even though I can't leave the house without seeing a Prius or a RAV4 hybrid, the news acts like it's gas versus electric as if Toyota hadn't solved this twenty years ago
There’s been decades of lies about climate change. And once the truth got out society was already massively dependent on it. For cars specifically it was a deliberate policy to make e.g. the US car-dependent. And once the truth got undeniable the cope was switched to people’s “carbon footprint” (British Petroleum). In fact there are rumors that the cope echoes to this day.
Zoom out enough and it becomes obviously unproductive to make “mass ignorance” the locus of attention.
Obviously people have zero awareness of or interest in their true impact on the environment. This extends to every facet of life and is not at all limited to AI use.
Do you really think the average person could within 2 orders of magnitude when estimating their carbon footprint for a year?
fairness to one polluter over another isn't the real issue - look at prop 65 in california; or if you're not used to this in CA, think of any time you've been on-call. alert fatigue is real and diminishes the urgency of the underlying message.
Yeah we do, it's basic epistemic hygiene. If you don't freak out about running your shower or microwave for a couple seconds or driving a few hundred feet you shouldn't be concerned about prompting an AI.
Except we do care about those things. We used to get tons of PSAs for carbon footprint. Turn off the lights when you leave a room, turn off your computer overnight, turn off the faucet when you're washing your hands. That type of thing.
Apologizing for AI boiling the oceans sounds like a lot of whataboutism.
I can picture an Elizabeth Holmesian cartoon clutching her diamond necklace.
"Oh, won't somebody think of the tech billionaires?!"
If you don't freak out about running your shower or microwave for a couple seconds or driving a few hundred feet
The basic premise of the modern tech industry is scale. It's not one person running a microwave for a couple of seconds, it's a few billion people running a microwave for the equivalent of decades.
You don’t need it for any pragmatic benefit because it won’t work. It doesn’t work for eating meat. It won’t work for AI.
The only purpose is to scapegoat the possible environmental or economic fallout. Might as well put it on individuals. Like what’s always done.
I’ve already seen it on the national broadcast. There some supposed experts were wagging their fingers about using AI for “fun”. Making silly images.
Meanwhile we’re gonna put AI to good use in arms races: more spam (automated applications, ads, ads, ads, abuse of services) and anti-spam. There’s gonna be so much economic activity. Disruptive.
The important part remains internalizing emission costs into the price of electricity. Fussing over individual users seems like a distraction to me. Rapid decarbonization of electricity is necessary regardless of who uses it. Demand will soar anyway as we electrify transportation, heating, and industry.
The projected electricity usage from all the planned data centers would cause an exponential increase in the amount of electricity they use. It's pretty fucking nuts and planning for it is a giant nightmare.
I agree but reducing consumption or increase of efficiency are still very important aspects of the energy transition. What is not consumed does not need to be generated.
If you internalize emissions costs into the price of electricity, reduced consumption will happen naturally. Precisely nobody likes higher energy bills, so there's a natural incentive to reduce consumption as long as you're paying for it.
I wonder how much households can really save here. Most "luxury" items using electricity don't really use much e.g. a modern laptop or modern smartphone. The stuff that does use a lot of electricity are things like your AC unit or your electric heater and electric stove. Seems there is little wiggle room there to me, people might end up just getting saddled with higher bills especially if slightly more efficient home appliances are out of reach (or not purchased by the renter at all). And for people who might get strongly affected out of their budget by these things for lack of income there are usually subsidies to help pay for their energy usage, which might further stymie market forces from changing behavior. Seems most high energy use consumers are high enough income where they won't be much affected by increased power costs like how we see them unaffected by water restrictions and higher fees for high water usage already.
Maybe that says the fees aren't yet high enough for high income people to change behavior, but I'm willing to bet they never truly will be due to the influence this subset of the population holds over politics.
Carbon taxes could be phased in over time, to give people a chance to make that decision over the course of natural appliances update lifecycles.
Even if rich people don’t consume much more energy than poor people (I have no idea, just engaging with your idea as stated), they must be buying something with their money… carbon taxes should raise the price of goods with lots of embodied carbon.
If they aren’t consuming much energy and am they aren’t buying stuff with much embodied carbon… I dunno, I guess that’s the goal, right?
It's not about households anyway, it's about transportation and industrial usage. Larger companies have enough scale that they can afford to invest in efficiency.
Some of these would benefit from changes (e.g. electric heating -> heat pump). Others would be better off with other changes. E.g. too much cooling? Consider better awnings, stronger blinds, or even IR rejecting films.
As for the stove, how much it uses is directly related to the kind of cooking you do, and for how long.
Even pricing CO2 output from burning fossil gas, plus a % for upstream leaks, and the same for car combustion will go a long way.
Mind you people won't like that since we're so used to using the atmosphere as a free sewer. The idea of having to pay for our pollution isn't palatable since the gasses are mostly invisible.
Though it's sad that we're talking about market solutions rather than outright bans for the majority of applications like we did for leaded gas.
Outright bans are a non-starter because it requires an infrastructure transition. You couldn't possibly replace every car with an electric one overnight, we can't make them that fast. But if you price carbon then it would cause every new car to be electric, or at least a plug-in hybrid that runs in electric mode 95% of the time. And the people who drive a lot of miles would switch to electric first, which would make a big difference right away.
Meanwhile the people with a 10 year old car they drive 5000 miles a year will keep it until it's a 20 year old car, at which point they'll buy another 10 year old car, but by then that one will run on electricity.
Then you could theoretically ban it, but by then do you even need to?
Sometimes I see chatter about using solar or nuclear or whatever power for data centers, thereby making them "clean," and it's frustrating that there isn't always the acknowledgement that the clean energy could displace other dirty generation.
Even with things like orphaned natural gas that gets flared otherwise - rescuing the energy is great but we could use it for many things, not just LLMs or bitcoin mining!
> the clean energy could displace other dirty generation.
If you would have built 10GW of solar or nuclear to replace other generation and instead the data center operators provide funding to build 20GW so that 10GW can go to data centers, the alternative wasn't replacing any of the other dirty generation. And the economies of scale may give the non-carbon alternatives a better cost advantage so you can build even more.
There’s no rule that increased demand will necessarily stimulate green energy production, only that it will stimulate energy production. And getting people to care about climate gets tougher, not easier, when energy demand goes up.
Indeed. However the problem with LLMs is that vast amounts of VC money are being thrown at them, in the [misplaced] hope of great returns. This results in a resource mis-allocation of biblical proportions, of which unnecessary carbon emissions are a part.
> In 2017, AI began to change everything. Data centers started getting built with energy-intensive hardware designed for AI, which led them to double their electricity consumption by 2023.
As we all know, the generative AI boom only really kicked into high gear in November 2022 with ChatGPT. That's five years of "AI" growth between 2017 and 2022 which presumably was mostly not generative AI.
When companies make ESG claims, sensible measurement and open traceability should always be the first proof they must provide. Without these, and validation from a credible independent entity such as a non-profit or government agency, all ESG claims from companies are merely PR puff pieces to keep the public at bay (especially in "AI").
Environmental, social, and governance (ESG) is shorthand for an investing principle that prioritizes environmental issues, social issues, and corporate governance.
Shameless plug . . . I run a startup who is working to help this https://neuralwatt.com We are starting with an os level (as in no model changes/no developer changes required) component which uses RL to run AI with a ~25% energy efficiency improvement w/out sacrificing UX. Feel free to dm me if you are interested in chatting either about problems you face with energy and ai or if you'd like to learn more.
Not just overconsumption, but also waste due to supply chain fragility. If you can induce demand anywhere then supply has to do crazy things to keep up.
> unprecedented and comprehensive look at how much energy the AI industry uses
Not sure about comprehensive claim here if end-to-end query chains were not considered.
For example the mobile wireless node (that're being used by the majority of the users) contribution to the energy consumption are totally ignored. The wireless power amplifier or PA for both sides of users and base-stations are notorious for their inefficiency being only less than than 50% in practice although in theory can be around 80%. Almost all of the current AI applications are cloud based not local-first thus the end users energy consumption and contribution are necessary.
I ponder this a lot, but the interface of "MIT technology Review" is unbearably overdesigned, its got that annoying narrow smartphone format where you can't zoom out, and then all these fancy graphics. Can't we have crisp, easy-to-read HTML? The format annoyed me so much I didn't read the article because this kind of design makes me doubt the source. Alas
I work in DCO, thats Data Center Operations if you’re not aware. I’ve tried explaining the amount of power used to my elderly mom; it isn’t easy! But here’s my best take:
The racks I am personally responsible for consume 17.2kW. That’s consistent across the year; sure things dip a bit when applications are shut down, but in general 17.2kW is the number. Presuming a national average of 1.2kW per home, each rack of equipment I oversee could potentially power 14 houses. I am responsible for hundreds of these racks, while my larger organization has many thousands of these racks in many locations worldwide.
I’ve found no other way to let the scale of this sink in. When put this way she is very clear: the price isn’t worth it to humanity. Being able to get, say, Door Dash, is pretty neat! But not at the cost of all our hoarded treasure and certainly not at the cost of the environment on the only planet we have access to.
The work done by AI will only ever benefit the people at the top. Because to be frank: they won’t share. Because the very wealthy have hoarding disorder.
It seems like you are having an emotional response to not understanding the general energy picture. For example, an A320 aloft uses the same energy as two thousand of your hypothetical racks (2.5 tons of kerosene per hour).
Each!
We are in no meaningful sense torching the biosphere to get AI.
With today’s AI systems, we still have very little visibility into their actual energy costs. As we push for larger models and faster responses, it’s worth asking whether we’re unintentionally accelerating energy use and emissions.
Finding the balance between innovation and long-term responsibility feels more important than ever.
I wouldn't be surprised if mankind will evolve similar to an organism and use 20% of all energy it produces on AI. Which is about 10x of what we use for software at the moment.
But then more AI also means more physical activity. When robots drive cars, we will have more cars driving around. When robots build houses, we will have more houses being built, etc. So energy usage will probably go up exponentially.
At the moment, the sun sends more energy to earth in an hour than humans use in a year. So the sun alone will be able to power this for the foreseeable future.
But the article says that energy use by AI is 48% more carbon intensive than the US average. So talk of solar power is a red herring -- that's not what it is running on now.
You said "for the foreseeable future", which I interpret as being about now.
Anyway I hope you're right, but so far global CO2 output is still growing. All the other energy has only come on top of carbon intensive energy, it hasn't replaced any of it. Every time we build more, we find new ways of spending that much energy and more.
Seeing 20 years into the future is quite possible in some aspects.
I remember how me and my friends discovered email in 1999 and were like "Yay, in the future we'll all do this instead of sending letters!". And it took about 20 years until letters were largely replaced by email and the web. And when the first videos appeared on the web, it was quite clear to us that they would replace DVDs.
Similar with the advent of self driving cars and solar energy I think.
The energy use by AI probably is just as, if not more, carbon intensive, but the article never says that. It talks about the energy use of the general data center.
> The carbon intensity of electricity used by data centers was 48% higher than the US average.
In case anyone is wondering why that is, it's because they put data centers in the places with the cheapest electricity. Which, in the US, is in places like Virginia and Ohio, where they burn fossil fuels.
If the people always talking about how cheap solar is want to fix this, find a way to make that cheapness actually make it into the customer's electric bill.
This assumes no technological adaptions towards efficiency. Consider yourself walking a mile and the energy expenditure. It isn't insignificant. Now imagine you have a bicycle. Some bicyclists will train and do century rides, a distance that were never possible merely walking for a day. But these are few bikers overall, most will not maximize capability to that extent but will still be taking advantage of the efficiency of the bike.
> When robots drive cars, we will have more cars driving around
This doesn't seem true. In SF, waymo with 300 cars does more rides than lyft with 45k drivers. If self driving cars interleave different tasks based on their routes I imagine they would be much more efficient per mile.
Seems like we are way too early in the adoption curve to tell. Currently the average number of passengers per trip is >1.0 across the whole fleet. Some day, I'd expect that to dip below 1.0, as people send an empty car to pick up the dog from the vet, or circle the block to avoid having to pay for parking, etc.
Thank you for this data point. It massively lowers the embodied carbon footprint (carbon from manufacturing, supply chain, transportation, etc.). Operational carbon is a solved problem; it is easy to measure and can be supplied from renewable sources.
> With more than 700 vehicles in its fleet - 300 of which operate in San Francisco - Waymo is the only U.S. firm that runs uncrewed robotaxis that collect fares.
>We’ve also incrementally grown our commercial fleet as we’ve welcomed more riders, with over 1,500 vehicles across San Francisco, Los Angeles, Phoenix, and Austin.
If waymo is doing more rides with 300 cars than 45k drivers on lyft, we can assume then that waymo cars are on the road serving customers at least 150x as long of time as a lyft driver. So yes it could really mean more cars are "around" even if the fleet is much smaller.
Most of the sunlight that hits a roof is already turned into heat. Whether you use that for calculations or not does not make a difference.
Not sure about the exact numbers, but I guess that at the moment normal roofs and solar panels absorb very roughly about the same percentage of sunlight.
So if in the future solar panels become more efficient, then yes, the amount of sunlight turned into heat could double.
Maybe that can be offset by covering other parts of earth with reflective materials or finding a way to send the heat back into the universe more effectively.
Building and running a nuclear reactor involves a lot of physical activity. And if the past is an indicator, we always move from physical activity to the flow of electrons.
The discussion about nuclear vs solar remind me of the discussions about spinning HDs versus solid state drives when they were new.
> When you ask an AI model to write you a joke or generate a video of a puppy, that query comes with a small but measurable energy toll and an associated amount of emissions spewed into the atmosphere. Given that each individual request often uses less energy than running a kitchen appliance for a few moments, it may seem insignificant.
> But as more of us turn to AI tools, these impacts start to add up. And increasingly, you don’t need to go looking to use AI: It’s being integrated into every corner of our digital lives.
Forward looking, I imagine this will be the biggest factor in increasing energy demands for AI: companies shoving it into products that nobody wants or needs.
> increasing energy demands for AI: companies shoving it into products that nobody wants or needs
I think of this a little every time Google gives me another result with the AI summary and no option for anyone to turn it off. Apparently worldwide there are 8+ billion searches every day.
In the short term perhaps, but even without carbon pricing the raw electricity prices will probably tamp down the enthusiasm. At someone point it’ll become cool for activist investors to demand to see ROI for AI features on earnings calls, and then the fat will get trimmed just like any other trend that goes too far.
I think the bigger underrated concern is if LLMs fall into an unfortunate bucket where they are in fact generally useful, but not in ways that help us decarbonize our energy supply (or that do, but not enough to offset their own energy usage).
Try to buy something that isn’t wrapped in three layers of plastic. Or that isn’t made of plastic itself. Then go to the checkout and see their “PSA” about how asking for a plastic bag to carry your plastic merchandise kills the planet.
I’m sorry. I’m being blocked by some mysterious force from understanding what “actual human” means. And I don’t know how to get you in contact with your car manufacturer. Would you like me to repeat my 20 step suggestion on how to troubleshoot “why does my shitty car put the A/C on freezer mode whenever “Indian Summer” tops the charts in Bulgaria”, but with more festive emojis?
This gives me the silly idea to go try to measure the power consumption of the local data center by measuring the magnetic field coming off the utility lines.
With all the issues and inefficiencies listed, there is a lot of room for improvement. I'm hopeful that just as the stat they give for data center energy not rising from 2005-2017, so to will the AI energy needs flatten in a few years. GPUs are not very efficient. Switching to more task specific hardware will provide more efficiency eventually. This is already happening a little with stuff like TPUs.
Oil might be able to carry more heat but it's more expensive to use.
Oil immersion is something nerds like to think is amazing but it's just a pain in the ass for negligible benefits. Imagine the annoyance of doing maintenance.
Wouldn't it be no different but your hands get a little oily? Say you take out a ram stick, oil goes into the empty dimm slot, but so what because its displaced again when you put in the new ram stick.
>you might think it’s like measuring a car’s fuel economy or a dishwasher’s energy rating: a knowable value with a shared methodology for calculating it. You’d be wrong.
But everyone knows fuel economy is everything but a knowable value. Everything from if it has rained in the past four hours to temperature to loading of the vehicle to the chemical composition of the fuel (HVO vs traditional), how worn are your tires? Are they installed the right way? Are your brakes lagging? The possibilities are endless. You could end up with twice the consumption.
By the way, copy-pasting from the website is terrible on desktop firefox, the site just lags every second, for a second.
fuel economy, like blood glucose levels, is impacted by many factors, but you can measure it over time. you might not be able to prescribe a course of action but you can make corrections to the course you're already on.
I wonder how the energy requirements are distributed between training and inference. Training should be extremely flexible, so one can only train when the sun shines and nobody uses the huge amount of solar power, or only when the wind turbines turn.
AFAICT the energy cost of training is still fairly low compared to cost of GPU's themselves so especially during a land grab it's important to drive as near as possible full utilization of the GPU's, energy be damned.
I doubt this is going to change.
That said, the flip side of energy cost being not a big factor is that you could probably eat the increase of energy cost by a factor of say 2 and this could possibly enable installation of short term (say 12h) battery storage to enable you to use only intermittent clean energy AND drive 100% utilization.
the numbers in the article are all over the place. I mean the article seems to try and some of the more general calculations on paper should work out but especially the image gen ones I can sorta disprove with my own experiences in local gen.
Even were it matches sorta (the 400 feet e-bike thing) that only works out for me because I use an AMD card. An NVIDIA card can have several times the generation speed at the same power draw so it all falls down again.
And the parameters they tried to standardize their figures with (the 1024x1024 thing) is also a bit meh because the SAME amount of pixels in a different aspect ratio can have huge variations in gen speed and thus power usage. for instance for most illustrious type checkpoints the speed is about 60% higher at aspect ratios other than 1024x1024. Its all a bit of a mess.
> There is a significant caveat to this math. These numbers cannot serve as a proxy for how much energy is required to power something like ChatGPT 4o.
Otherwise this is an excellent article critiquing the very real problem that is opacity of these companies regarding model sizes and deployments. Not having an honest accounting of computing deployed worldwide is a problem, and while it's true that we didn't really do this in the past (early versions of Google searches were undoubtedly inefficient!), it's not an excuse today.
I also wish this article talked about the compute trends. That is, compute per token is going significantly down, but that also means use of that compute can spread more. Where does that lead us?
> This leaves even those whose job it is to predict energy demands forced to assemble a puzzle with countless missing pieces, making it nearly impossible to plan for AI’s future impact on energy grids and emissions. Worse, the deals that utility companies make with the data centers will likely transfer the costs of the AI revolution to the rest of us, in the form of higher electricity bills.
... So don't? Explicitly shift the cost to the customer.
If I want to hook up to the energy grid with 3-phase power, I pay the utility to do it.
If a business wants more power and it isn't available, then the business can pay for it.
Then only businesses that really need it will be willing to step up to the plate.
No amount of "accounting" or "energy needs prediction" will guard against regulatory capture.
Might have missed it but was disappointed to see no mention of externalized costs like the scraping burden imposed on every IP-connected server. From discussions on HN this sounds quite substantial. And again, why exactly should the few AI companies reap all the value when other companies and individuals are incurring costs for it?
This series of articles is driving me insane. The authors or editors are using inappropriate units to shock readers: billions of gallons, millions of square feet. But they are not putting the figures into context that the reader can directly comprehend. Because if they said the Nevada data centers would use 2% as much water as the hay/alfalfa industry in Nevada then the entire article loses its shock value.
What’s the net energy footprint of an employee working in an office whose job was made redundant by AI? Of course that human will likely have another job, but what’s the math of a person who was doing tedium solved by AI and now can do something more productive that AI can’t necessarily do. In other words, let’s calculate the “economic output per energy unit expended.”
On that note, what’s the energy footprint of the return to office initiatives that many companies have initiated?
> Of course that human will likely have another job, but what’s the math of a person who was doing tedium solved by AI and now can do something more productive that AI can’t necessarily do
That’s a lot of big assumptions - that the job getting replaced was tedious in the first place, that those other “more productive” job exists, that the thing AI can’t necessarily do will stay that way long enough for it not to be taken over by AI as well, that the tediousness was not part of the point (e.g. art)…
Net energy change of people doing work on their desk versus browsing the internet versus playing games, you will likely not see difference at all. They're all at rest, more or less thinking something. People at home sofa always have metabolic processes running regardless of whether it produces additional value to some corporation
Weird I was assured that Bitcoin would be using all of the worlds electricity by now.
Which I already thought was odd, because London would need all that electricity to see through the giant mountain of poop piled up by all the horses the british use for transportation.
It's from 2021 so won't cover the 2022-onwards generative AI boom.
From the Wikipedia summary it sounds like it's about machine learning algorithms like classification, AlphaGo and concerns about ethics of training and bias.
Interesting, thanks for sharing! I share some concerns others have about this piece, but I’m most shocked about their finding that image generation is cheaper than text. As someone who’s gone down this rabbit hole multiple times, this runs against every single paper I’ve ever cited on the topic. Anyone know why? Maybe this is a recent change? It also doesn’t help that multimodal transformers are now blurring the lines between image and text, of course… this article doesn’t even handle that though, treating all image models as diffusion models.
"The carbon intensity of electricity used by data centers was 48% higher than the US average."
I'd be fine with as many data centers as they want if they stimulated production of clean energy to run them.
But that quote links to another article by the same author. Which says
"Notably, the sources for all this power are particularly “dirty.” Since so many data centers are located in coal-producing regions, like Virginia, the “carbon intensity” of the energy they use is 48% higher than the national average. The paper, which was published on arXiv and has not yet been peer-reviewed, found that 95% of data centers in the US are built in places with sources of electricity that are dirtier than the national average. "
"The average carbon intensity of the US data centers in our study (weighted by the energy they consumed) was 548 grams of CO2e per kilowatt hour (kWh), approximately 48% higher than the US national average of 369 gCO2e / kWh (26)."
which shows 375g/KWh (after converting from lb/MWh)
But the table they compare against shows.
VA 576g/KWh
TX 509g/KWh
CA 374g/KWh
and the EPA table shows
VA 268g/KWh
TX 372g/KWh
CA 207g/KWh
Which seem more likely to be true. The paper has California at only marginally better than the national average for renewables (Which I guess they needed to support their argument given the number of data centers there)
I like arxiv, It's a great place to see new ideas, the fields I look at have things that I can test myself to see if the idea actually works. I would not recommend it as a source of truth. Peer review still has a place.
If they were gathering emissions data from states themselves, they should have caclulated the average from that data, not pulled the average from another potentially completely different measure. Then their conclusions would have been valid regardless what weird scaling factor they bought in to their state calculations. The numbers might have been wrong but the proportion would have been accurate, and it is the proportion that is being highlighted.
there are still negative externalities to high renewable-energy usage (heat and water usage, which itself requires energy to purify once returned to the sewer, plus the environmental impact of building an enormous heat island in places where there was little industry previously).
Today Google launched a model, Gemma 3n, that performs about as good as SOTA models from 1-2 years ago that runs locally on a cell phone.
Training SOTA models will, like steel mills or other large industrial projects, require a lot of environmental footprint to produce. But my prediction is that over time the vast majority of use cases in the hands of users will be essentially run on device and be basically zero impact, both in monetary cost and environment.
http://archive.today/mnHb8
If you are old enough you remember posting to Usenet and the warning that would accompany each new submission:
This program posts news to thousands of machines throughout the entire civilized world. Your message will cost the net hundreds if not thousands of dollars to send everywhere. Please be sure you know what you are doing. Are you absolutely sure that you want to do this? [ny]
Maybe we meed something similar in LLM clients. Could be phrased in terms of how many pounds of atmospheric carbon the request will produce.
Individual LLM requests are vanishingly small in terms of environmental impact; inference providers use a lot of batching to do lots of work at once. Furthermore, LLMs and diffusion models are not the only ML workload. While generative AI tickles investors, most of the ML actually being deployed is more mundane things, like recommendation systems, classifiers, and the like; much of which is used for adtech purposes adversarial to that of users. If LLMs and diffusers were the only thing companies used ML for, but efficiency gains from new hardware remained constant, we'd still be at the 2017 baseline for environmental impact of data centers.
Likewise, I doubt that USENET warning was ever true beyond the first few years of the networks' lifetime. Certainly if everything was connected via dial-up, yes, a single message could incur hundreds of dollars of cost when you added the few seconds of line time it took to send up across the whole world. But that's accounting for a lot of Ma Bell markup. Most connections between sites and ISPs on USENET were done through private lines that ran at far faster speeds than what you could shove down copper phone wiring back then.
If what you're saying is true, why are we hearing about AI companies wanting to build nuclear power plants to power new data centers they think they need to build?
Are you saying all of that new capacity is needed to power non-LLM stuff like classifiers, adtech, etc? That seems unlikely.
Had you said that inference costs are tiny compared to the upfront cost of training the base model, I might have believed it. But even that isn't accurate -- there's a big upfront energy cost to train a model, but once it becomes popular like GPT-4, the inference energy cost over time is dramatically higher than the upfront training cost.
You mentioned batch computing as well, but how does that fit into the picture? I don't see how batching would reduce energy use. Does "doing lots of work at once" somehow reduce the total work / total energy expended?
> If what you're saying is true, why are we hearing about AI companies wanting to build nuclear power plants to power new data centers they think they need to build?
Well, partly because they (all but X, IIRC) have commitments to shift to carbon-neutral energy.
But also, from the article:
> ChatGPT is now estimated to be the fifth-most visited website in the world
That's ChatGPT today. They're looking ahead to 100x-ing (or 1,000,000x-ing) the usage as AI replaces more and more existing work.
I can run Llama 3 on my laptop, and we can measure the energy usage of my laptop--it maxes out at around 0.1 toasters. o3 is presumably a bit more energy intensive, but the reason it's using a lot of power is the >100MM daily users, not that a single user uses a lot of energy for a simple chat.
>If what you're saying is true, why are we hearing about AI companies wanting to build nuclear power plants to power new data centers they think they need to build?
Something to temper this, lots of these AI datacenter projects are being cancelled or put on hiatus because the demand isnt there.
But if someone wants to build a nuke reactor to power their datacenter, awesome. No downsides? We are concerned about energy consumption only because of its impact on the earth in terms of carbon footprint. If its nuclear, the problem has already been solved.
> Something to temper this, lots of these AI datacenter projects are being cancelled or put on hiatus because the demand isnt there.
Wait, any sources for that? Because everywhere I go, there seems to be this hype for more AI data centers. Some fresh air would be nice.
https://www.datacenterdynamics.com/en/news/microsoft-cancels...
AI seems like it is speedrunning all the phases of the hype cycle.
"TD Cowen analysts Michael Elias, Cooper Belanger, and Gregory Williams wrote in the latest research note: “We continue to believe the lease cancellations and deferrals of capacity points to data center oversupply relative to its current demand forecast.”"
Because training costs are sky-high, and handling an individual request still uses a decent amount of energy even if it isn't as horrifying as training. Plus the amount of requests, and content in them, is going up with stuff like vibe coding.
If you want to know more about energy consumption, see this 2 part series that goes into tons of nitty-gritty details: https://blog.giovanh.com/blog/2024/08/18/is-ai-eating-all-th...
The article says 80-90% of data center usage for AI is for inference, and is from a more reputable source than the random blog
The blog is citing specific studies for its claims. Is there an issue with those studies?
It's almost a year old at this point so at best it is horribly out of date
A lot of us live in a country where "rolling coal" is a thing. I fear your prompt may have an opposite of the intended effect.
Taxing anything that can pollute (methane, gasoline, diesel) would let The Hand sort it out
Carbon taxes are incredibly unpopular, because it makes easy and convenient things expensive.
They're incredibly unpopular because even when they're made revenue-neutral (meaning, everyone gets a refund check) people don't realize most of them would make money if they actually reduced their carbon.
They're incredibly unpopular because the ultraweathly use massive amounts of fossil fuels and thus lobby very, very hard against them...and make sure the public is often told just how evil they are and how expensive they'd hurt Johnny Everyday Worker, even car ownership, especially in a city (where much of the US populative lives) is not affordable to a large segment of the population.
If memory serves Jet A is not taxed at all federally in the case of for-profit corporations (while non-commercial users DO pay a tax!) and many states also either do not tax it or tax it very litte.
It's completely insane that we do not tax fuel usage for probably the most energy-intensive way to move people and/or goods and often that movement of people is entirely frivelous.
> They're incredibly unpopular because the ultraweathly use massive amounts of fossil fuels and thus lobby very, very hard against them...and make sure the public is often told just how evil they are and how expensive they'd hurt Johnny Everyday Worker, even car ownership, especially in a city (where much of the US populative lives) is not affordable to a large segment of the population.
Eh. It's not Bill Gates and Alice Walton. Sometimes the obvious answer is the real one: It's the fossil fuel industry.
> It's completely insane that we do not tax fuel usage for probably the most energy-intensive way to move people and/or goods and often that movement of people is entirely frivelous.
That one's just the arbitrage problem. Planes move around. If there is an international flight to a country that doesn't tax jet fuel (or taxes it less) then the plane is going to fly into LAX with enough fuel still in the tank to get back to the other jurisdiction and fill up again. Which actually increases fuel consumption because fuel is heavy and they otherwise wouldn't want to do that.
This is the same reason the EU doesn't tax jet fuel.
I find that message very curious because the message itself clearly does not cost much but the machines it is send on do. So the more messages that are send the less the messages will cost.
Only if that same warning is attached to literally everything else you do. It's unfair to single out AI for using energy.
But "asking ChatGPT" is something people do so casually without there being any apparent clues of the costs associated with that. I guess that is true of pretty much everything online though.
Even driving your car around you at least are somewhat aware of the gas you are burning.
"Civilization advances by extending the number of important operations which we can perform without thinking of them." — Alfred North Whitehead
Emissions should be fixed on the production side (decarbonization) not on the demand side (guilt/austerity).
While I agree in principle how does this work for fossil fuels? Is the idea that we should make extraction prohibitively expensive?
Scaling up battery production makes EVs more appealing on the demand side. How do you disincentivize fossil fuel production?
Carbon tax or something similar.
Batteries and EVs are the production side. Reducing demand is e.g. requiring you to drive fewer miles. You get a car that doesn't run on petroleum and CO2 goes down while vehicle miles can stay the same or go up.
Making it illegal is always an option, and one that many countries are considering
I would bet most ppl drive around with very little awareness of how much it’s costing, either in money or environmental impact. Many people I’ve met seem to measure efficiency by how much it costs to fill up the tank.
Even more fundamentally, think about how much carbon running a cup of water from your faucet produces. No matter where you live, this is more carbon than an LLM prompt generates.
Or, even worse God forbid, think about how much carbon is produced to create a single bottle or carton of water. Then consider how casually people down bottles of water.
Can you source that? A lot of places have gravity fed reservoirs that are energy positive/neutral (LA, San Francisco and New York all rely on gravity fed reservoirs that were built before energy intensive pumping was practical). There are some costs but they are pretty small per gallon.
I’m assuming that calculation is amortizing the cost of running the water system (including wastewater treatment) adding the cost pumping it to the point that gravity can push it through the pipes. It’s never free.
That could be solved by charging more for the service. That is the only reason you are aware of the gas burning after all, you aren't conducting your own aq test you are noticing you are filling up twice a week at $50 a tank.
They're aware of the price they pay for the gas, not the emissions. I would wager that the mass ignorance of the impact of fossil fuels (and rubber on roads) that the broader population has is a significant reason why we're in this climate mess today.
> rubber on roads
Funny how this suddenly became a thing after electrification became a thing. Need to find a new way to wag the finger after all.
It's always been a thing? I'm pro-electrification, BTW.
I am 100% on the side of reducing pollutants — but this was never publicly seen as a major issue and I'm suspicious about the timing.
The oil industry is a conglomerate of degenerates spamming boomer logic all the way down to the workers. Their memes propagate throughout society and lead to the other boomer characteristic of rewriting personal and societal history.
The finger waggers now are being programmed to pretend they talked about tire particulates and the carheads are being programmed to pretend they never cared about 0-60. This another "We have always been at war with Eastasia", just like they all opposed the Iraq war from day 1 and didn't cancel the Dixie Chicks, et cetra.
This may have been discussed in specialist literature somewhere but even when I did ecology courses in university circa 2001ish, I never heard about tire particulates, while I did hear a lot about greenhouse gasses.
It's a concern but not a civilization ending concern like climate change. I low key resent these attempts to move goalposts to satisfy the writer's urge for negativity.
It's pretty clearly a talking point.
Consider that a bus has six to ten tires that each weigh around ten times more than a typical car tire. This is presented as the alternative to cars, is it even any different? Not implausible that it could actually be worse, especially if the bus isn't at full occupancy at all times.
Meanwhile the weight difference between EVs and petroleum cars is emphasized in the complaints, even though it isn't very large, while the much larger weight difference between any cars and buses is ignored. Because the point isn't to complain about tires, it's to complain about EVs.
And if the point actually was to complain about tires then you still wouldn't be talking about EVs, you would be talking about tires and how to make them shed less or construct them out of lower toxicity materials etc.
Last time I did the math, a Tesla Model Y only had 3x less tire emissions than a semi truck per distance traveled. City buses are on-par with a Tesla Model Y if you only care about mL/km tire wear.
How is that math supposed to work when a city bus weighs almost ten times as much and has more and bigger tires?
The city bus uses tires with a harder rubber and dimensions such that the pressure at the road is less, plus its normal driving patterns have less wear than typical Tesla use.
To make those sorts of calculations easy, you can ignore all the pressure/usage/etc nonsense and just do basic math on tire dimensions (including min/max tread depth and width, not just radius, though I typically ignore siping and whatnot) and typical longevity. Volume lost per mile driven is basic high-school arithmetic, and the only real questions are regarding data quality and whether the self-imposed constraints (e.g., examining real-world wear rather than wear given optimal driving or something) are reasonable.
>It's always been a thing?
Is there a way to quantify this? My experience as well is that the tire particulate pollution has mostly been an anti-EV talking point.
Well many of my fellow Americans would only accept an EV if it's gigantic, and even though I can't leave the house without seeing a Prius or a RAV4 hybrid, the news acts like it's gas versus electric as if Toyota hadn't solved this twenty years ago
You would wager. Based on what?
There’s been decades of lies about climate change. And once the truth got out society was already massively dependent on it. For cars specifically it was a deliberate policy to make e.g. the US car-dependent. And once the truth got undeniable the cope was switched to people’s “carbon footprint” (British Petroleum). In fact there are rumors that the cope echoes to this day.
Zoom out enough and it becomes obviously unproductive to make “mass ignorance” the locus of attention.
https://simonwillison.net/2025/May/6/whats-the-carbon-footpr...
Obviously people have zero awareness of or interest in their true impact on the environment. This extends to every facet of life and is not at all limited to AI use.
Do you really think the average person could within 2 orders of magnitude when estimating their carbon footprint for a year?
fairness to one polluter over another isn't the real issue - look at prop 65 in california; or if you're not used to this in CA, think of any time you've been on-call. alert fatigue is real and diminishes the urgency of the underlying message.
It's unfair to single out AI for using energy.
Why? AI isn't a human being. We have no obligation to be "fair" to it.
Yeah we do, it's basic epistemic hygiene. If you don't freak out about running your shower or microwave for a couple seconds or driving a few hundred feet you shouldn't be concerned about prompting an AI.
Except we do care about those things. We used to get tons of PSAs for carbon footprint. Turn off the lights when you leave a room, turn off your computer overnight, turn off the faucet when you're washing your hands. That type of thing.
Lol If we do care about those things why did the PSAs stop? Problem solved?
Apologizing for AI boiling the oceans sounds like a lot of whataboutism.
I can picture an Elizabeth Holmesian cartoon clutching her diamond necklace.
"Oh, won't somebody think of the tech billionaires?!"
If you don't freak out about running your shower or microwave for a couple seconds or driving a few hundred feet
The basic premise of the modern tech industry is scale. It's not one person running a microwave for a couple of seconds, it's a few billion people running a microwave for the equivalent of decades.
You don’t need it for any pragmatic benefit because it won’t work. It doesn’t work for eating meat. It won’t work for AI.
The only purpose is to scapegoat the possible environmental or economic fallout. Might as well put it on individuals. Like what’s always done.
I’ve already seen it on the national broadcast. There some supposed experts were wagging their fingers about using AI for “fun”. Making silly images.
Meanwhile we’re gonna put AI to good use in arms races: more spam (automated applications, ads, ads, ads, abuse of services) and anti-spam. There’s gonna be so much economic activity. Disruptive.
The important part remains internalizing emission costs into the price of electricity. Fussing over individual users seems like a distraction to me. Rapid decarbonization of electricity is necessary regardless of who uses it. Demand will soar anyway as we electrify transportation, heating, and industry.
The projected electricity usage from all the planned data centers would cause an exponential increase in the amount of electricity they use. It's pretty fucking nuts and planning for it is a giant nightmare.
I agree but reducing consumption or increase of efficiency are still very important aspects of the energy transition. What is not consumed does not need to be generated.
If you internalize emissions costs into the price of electricity, reduced consumption will happen naturally. Precisely nobody likes higher energy bills, so there's a natural incentive to reduce consumption as long as you're paying for it.
I wonder how much households can really save here. Most "luxury" items using electricity don't really use much e.g. a modern laptop or modern smartphone. The stuff that does use a lot of electricity are things like your AC unit or your electric heater and electric stove. Seems there is little wiggle room there to me, people might end up just getting saddled with higher bills especially if slightly more efficient home appliances are out of reach (or not purchased by the renter at all). And for people who might get strongly affected out of their budget by these things for lack of income there are usually subsidies to help pay for their energy usage, which might further stymie market forces from changing behavior. Seems most high energy use consumers are high enough income where they won't be much affected by increased power costs like how we see them unaffected by water restrictions and higher fees for high water usage already.
Maybe that says the fees aren't yet high enough for high income people to change behavior, but I'm willing to bet they never truly will be due to the influence this subset of the population holds over politics.
Carbon taxes could be phased in over time, to give people a chance to make that decision over the course of natural appliances update lifecycles.
Even if rich people don’t consume much more energy than poor people (I have no idea, just engaging with your idea as stated), they must be buying something with their money… carbon taxes should raise the price of goods with lots of embodied carbon.
If they aren’t consuming much energy and am they aren’t buying stuff with much embodied carbon… I dunno, I guess that’s the goal, right?
It's not about households anyway, it's about transportation and industrial usage. Larger companies have enough scale that they can afford to invest in efficiency.
Some of these would benefit from changes (e.g. electric heating -> heat pump). Others would be better off with other changes. E.g. too much cooling? Consider better awnings, stronger blinds, or even IR rejecting films.
As for the stove, how much it uses is directly related to the kind of cooking you do, and for how long.
"internalizing emissions" is the kind of thing that's really easy to say, even conceptualize, but really difficult to implement.
You could do it better than we are doing now, but you'll always have people saying: "that's unfair, why are you picking on me"
Even pricing CO2 output from burning fossil gas, plus a % for upstream leaks, and the same for car combustion will go a long way.
Mind you people won't like that since we're so used to using the atmosphere as a free sewer. The idea of having to pay for our pollution isn't palatable since the gasses are mostly invisible.
Though it's sad that we're talking about market solutions rather than outright bans for the majority of applications like we did for leaded gas.
Outright bans are a non-starter because it requires an infrastructure transition. You couldn't possibly replace every car with an electric one overnight, we can't make them that fast. But if you price carbon then it would cause every new car to be electric, or at least a plug-in hybrid that runs in electric mode 95% of the time. And the people who drive a lot of miles would switch to electric first, which would make a big difference right away.
Meanwhile the people with a 10 year old car they drive 5000 miles a year will keep it until it's a 20 year old car, at which point they'll buy another 10 year old car, but by then that one will run on electricity.
Then you could theoretically ban it, but by then do you even need to?
Sometimes I see chatter about using solar or nuclear or whatever power for data centers, thereby making them "clean," and it's frustrating that there isn't always the acknowledgement that the clean energy could displace other dirty generation.
Even with things like orphaned natural gas that gets flared otherwise - rescuing the energy is great but we could use it for many things, not just LLMs or bitcoin mining!
> the clean energy could displace other dirty generation.
If you would have built 10GW of solar or nuclear to replace other generation and instead the data center operators provide funding to build 20GW so that 10GW can go to data centers, the alternative wasn't replacing any of the other dirty generation. And the economies of scale may give the non-carbon alternatives a better cost advantage so you can build even more.
I don't think it a given that reducing energy consumption is a required part of the transition.
Increasing demand can lead to stimulus of green energy production.
There’s no rule that increased demand will necessarily stimulate green energy production, only that it will stimulate energy production. And getting people to care about climate gets tougher, not easier, when energy demand goes up.
Yeah, but pricing signals are a good way of reaching those goals.
Businesses will only start doing that in significant amounts when carbon emissions are priced according to their environmental impact.
To do that we would need society to agree about what the emission cost is.
Making electricity so abundant and efficient is probably more solvable. You can’t solve stu… society
Indeed. However the problem with LLMs is that vast amounts of VC money are being thrown at them, in the [misplaced] hope of great returns. This results in a resource mis-allocation of biblical proportions, of which unnecessary carbon emissions are a part.
Best article I've ever read about the energy needs of AI.
Impressive how Big Tech refuses to share data with society for collective decisions.
I'd also recommend the Data Vampires podcast series:
https://techwontsave.us/episode/241_data_vampires_going_hype...
https://techwontsave.us/episode/243_data_vampires_opposing_d...
https://techwontsave.us/episode/245_data_vampires_sacrificin...
https://techwontsave.us/episode/247_data_vampires_fighting_f...
I found this bit interesting:
> In 2017, AI began to change everything. Data centers started getting built with energy-intensive hardware designed for AI, which led them to double their electricity consumption by 2023.
As we all know, the generative AI boom only really kicked into high gear in November 2022 with ChatGPT. That's five years of "AI" growth between 2017 and 2022 which presumably was mostly not generative AI.
People started using GPUs for ML at around that time.
When companies make ESG claims, sensible measurement and open traceability should always be the first proof they must provide. Without these, and validation from a credible independent entity such as a non-profit or government agency, all ESG claims from companies are merely PR puff pieces to keep the public at bay (especially in "AI").
esg?
Environmental/Social/Governance. From Wikipedia:
Environmental, social, and governance (ESG) is shorthand for an investing principle that prioritizes environmental issues, social issues, and corporate governance.
when has ESG not been FUD or a way to bypass sanctions from poorly thought out climate change targets?
Shameless plug . . . I run a startup who is working to help this https://neuralwatt.com We are starting with an os level (as in no model changes/no developer changes required) component which uses RL to run AI with a ~25% energy efficiency improvement w/out sacrificing UX. Feel free to dm me if you are interested in chatting either about problems you face with energy and ai or if you'd like to learn more.
I wonder what the carbon footprint of all those ads is.
Not just the ads, but also the overconsumption which they cause.
Not just overconsumption, but also waste due to supply chain fragility. If you can induce demand anywhere then supply has to do crazy things to keep up.
> unprecedented and comprehensive look at how much energy the AI industry uses
Not sure about comprehensive claim here if end-to-end query chains were not considered.
For example the mobile wireless node (that're being used by the majority of the users) contribution to the energy consumption are totally ignored. The wireless power amplifier or PA for both sides of users and base-stations are notorious for their inefficiency being only less than than 50% in practice although in theory can be around 80%. Almost all of the current AI applications are cloud based not local-first thus the end users energy consumption and contribution are necessary.
I ponder this a lot, but the interface of "MIT technology Review" is unbearably overdesigned, its got that annoying narrow smartphone format where you can't zoom out, and then all these fancy graphics. Can't we have crisp, easy-to-read HTML? The format annoyed me so much I didn't read the article because this kind of design makes me doubt the source. Alas
I work in DCO, thats Data Center Operations if you’re not aware. I’ve tried explaining the amount of power used to my elderly mom; it isn’t easy! But here’s my best take:
The racks I am personally responsible for consume 17.2kW. That’s consistent across the year; sure things dip a bit when applications are shut down, but in general 17.2kW is the number. Presuming a national average of 1.2kW per home, each rack of equipment I oversee could potentially power 14 houses. I am responsible for hundreds of these racks, while my larger organization has many thousands of these racks in many locations worldwide.
I’ve found no other way to let the scale of this sink in. When put this way she is very clear: the price isn’t worth it to humanity. Being able to get, say, Door Dash, is pretty neat! But not at the cost of all our hoarded treasure and certainly not at the cost of the environment on the only planet we have access to.
The work done by AI will only ever benefit the people at the top. Because to be frank: they won’t share. Because the very wealthy have hoarding disorder.
But it can’t really power “14 houses” because people in those houses are consuming external services such as those provided by your racks.
Unless your racks can only serve 14 customers.
It seems like you are having an emotional response to not understanding the general energy picture. For example, an A320 aloft uses the same energy as two thousand of your hypothetical racks (2.5 tons of kerosene per hour).
Each!
We are in no meaningful sense torching the biosphere to get AI.
With today’s AI systems, we still have very little visibility into their actual energy costs. As we push for larger models and faster responses, it’s worth asking whether we’re unintentionally accelerating energy use and emissions.
Finding the balance between innovation and long-term responsibility feels more important than ever.
We really need to improve the power grid. I don't think about "A. I." very much, but I am glad that something is making us upgrade the grid.
> I am glad that something is making us upgrade the grid
A few big consumers in centralized locations isn't changing the grid as much as the energy transition from fuels to electricity is
The transition to electric vehicles in the US has been disappointing, to say the least.
I thought everyone over there has an air conditioning system on electricity?
Yes, but we already do. We need new electricity demand.
40% of electricity consumption in Virgina will be data centers in 2030?
Table A1 , PDF page 29:
https://www.epri.com/research/products/000000003002028905
The brain uses 20% of the human body's energy.
I wouldn't be surprised if mankind will evolve similar to an organism and use 20% of all energy it produces on AI. Which is about 10x of what we use for software at the moment.
But then more AI also means more physical activity. When robots drive cars, we will have more cars driving around. When robots build houses, we will have more houses being built, etc. So energy usage will probably go up exponentially.
At the moment, the sun sends more energy to earth in an hour than humans use in a year. So the sun alone will be able to power this for the foreseeable future.
But the article says that energy use by AI is 48% more carbon intensive than the US average. So talk of solar power is a red herring -- that's not what it is running on now.
I am thinking about the future here.
I don't think there will be much carbon intensive energy creation in a few decades from now. It does not make sense economically.
You said "for the foreseeable future", which I interpret as being about now.
Anyway I hope you're right, but so far global CO2 output is still growing. All the other energy has only come on top of carbon intensive energy, it hasn't replaced any of it. Every time we build more, we find new ways of spending that much energy and more.
Seeing 20 years into the future is quite possible in some aspects.
I remember how me and my friends discovered email in 1999 and were like "Yay, in the future we'll all do this instead of sending letters!". And it took about 20 years until letters were largely replaced by email and the web. And when the first videos appeared on the web, it was quite clear to us that they would replace DVDs.
Similar with the advent of self driving cars and solar energy I think.
The energy use by AI probably is just as, if not more, carbon intensive, but the article never says that. It talks about the energy use of the general data center.
> The carbon intensity of electricity used by data centers was 48% higher than the US average.
In case anyone is wondering why that is, it's because they put data centers in the places with the cheapest electricity. Which, in the US, is in places like Virginia and Ohio, where they burn fossil fuels.
If the people always talking about how cheap solar is want to fix this, find a way to make that cheapness actually make it into the customer's electric bill.
> energy usage will probably go up exponentially
kindof sounds like Jevons paradox? https://wiki.froth.zone/wiki/Jevons_paradox
This assumes no technological adaptions towards efficiency. Consider yourself walking a mile and the energy expenditure. It isn't insignificant. Now imagine you have a bicycle. Some bicyclists will train and do century rides, a distance that were never possible merely walking for a day. But these are few bikers overall, most will not maximize capability to that extent but will still be taking advantage of the efficiency of the bike.
> When robots drive cars, we will have more cars driving around
This doesn't seem true. In SF, waymo with 300 cars does more rides than lyft with 45k drivers. If self driving cars interleave different tasks based on their routes I imagine they would be much more efficient per mile.
>This doesn't seem true.
Seems like we are way too early in the adoption curve to tell. Currently the average number of passengers per trip is >1.0 across the whole fleet. Some day, I'd expect that to dip below 1.0, as people send an empty car to pick up the dog from the vet, or circle the block to avoid having to pay for parking, etc.
Thank you for this data point. It massively lowers the embodied carbon footprint (carbon from manufacturing, supply chain, transportation, etc.). Operational carbon is a solved problem; it is easy to measure and can be supplied from renewable sources.
Is it really only 300 cars? They feel like they're everywhere!
I thought it was more than that too, but according to https://www.reuters.com/business/autos-transportation/alphab...
> With more than 700 vehicles in its fleet - 300 of which operate in San Francisco - Waymo is the only U.S. firm that runs uncrewed robotaxis that collect fares.
Those numbers are from April 2025.
>We’ve also incrementally grown our commercial fleet as we’ve welcomed more riders, with over 1,500 vehicles across San Francisco, Los Angeles, Phoenix, and Austin.
https://waymo.com/blog/2025/05/scaling-our-fleet-through-us-...
If waymo is doing more rides with 300 cars than 45k drivers on lyft, we can assume then that waymo cars are on the road serving customers at least 150x as long of time as a lyft driver. So yes it could really mean more cars are "around" even if the fleet is much smaller.
Existing rides will be done more efficiently but since rides are so much cheaper without a driver, much more rides will be done.
A car driving from A to B will cost less than 50% of the current price. Which will unlock a huge amount of new rides.
One problem: all this energy is eventually turned into heat ...
Most of the sunlight that hits a roof is already turned into heat. Whether you use that for calculations or not does not make a difference.
Not sure about the exact numbers, but I guess that at the moment normal roofs and solar panels absorb very roughly about the same percentage of sunlight.
So if in the future solar panels become more efficient, then yes, the amount of sunlight turned into heat could double.
Maybe that can be offset by covering other parts of earth with reflective materials or finding a way to send the heat back into the universe more effectively.
What if you put a solar farm in a desert, though?
And also, people should paint their roofs white.
Why not nuclear?
Building and running a nuclear reactor involves a lot of physical activity. And if the past is an indicator, we always move from physical activity to the flow of electrons.
The discussion about nuclear vs solar remind me of the discussions about spinning HDs versus solid state drives when they were new.
HDDs build the backbone of all large storage systems, they serve many purposes today. Magnetic tape is still in use too
> When you ask an AI model to write you a joke or generate a video of a puppy, that query comes with a small but measurable energy toll and an associated amount of emissions spewed into the atmosphere. Given that each individual request often uses less energy than running a kitchen appliance for a few moments, it may seem insignificant.
> But as more of us turn to AI tools, these impacts start to add up. And increasingly, you don’t need to go looking to use AI: It’s being integrated into every corner of our digital lives.
Forward looking, I imagine this will be the biggest factor in increasing energy demands for AI: companies shoving it into products that nobody wants or needs.
> increasing energy demands for AI: companies shoving it into products that nobody wants or needs
I think of this a little every time Google gives me another result with the AI summary and no option for anyone to turn it off. Apparently worldwide there are 8+ billion searches every day.
In the short term perhaps, but even without carbon pricing the raw electricity prices will probably tamp down the enthusiasm. At someone point it’ll become cool for activist investors to demand to see ROI for AI features on earnings calls, and then the fat will get trimmed just like any other trend that goes too far.
I think the bigger underrated concern is if LLMs fall into an unfortunate bucket where they are in fact generally useful, but not in ways that help us decarbonize our energy supply (or that do, but not enough to offset their own energy usage).
Try to buy something that isn’t wrapped in three layers of plastic. Or that isn’t made of plastic itself. Then go to the checkout and see their “PSA” about how asking for a plastic bag to carry your plastic merchandise kills the planet.
I’m sorry. I’m being blocked by some mysterious force from understanding what “actual human” means. And I don’t know how to get you in contact with your car manufacturer. Would you like me to repeat my 20 step suggestion on how to troubleshoot “why does my shitty car put the A/C on freezer mode whenever “Indian Summer” tops the charts in Bulgaria”, but with more festive emojis?
This gives me the silly idea to go try to measure the power consumption of the local data center by measuring the magnetic field coming off the utility lines.
Are these the same people who claimed that crypto was going to use more energy than the entire world by 2030?
With all the issues and inefficiencies listed, there is a lot of room for improvement. I'm hopeful that just as the stat they give for data center energy not rising from 2005-2017, so to will the AI energy needs flatten in a few years. GPUs are not very efficient. Switching to more task specific hardware will provide more efficiency eventually. This is already happening a little with stuff like TPUs.
I would like to see more data centers make use of large-scale Oil Immersion-Cooling. I feel like the fresh water use for cooling is a huge issue.
https://par.nsf.gov/servlets/purl/10101126
There are various data center cooling techniques, and not all of them use water. As a result, water usage, and water usage efficiency, vary wildly:
https://dgtlinfra.com/data-center-water-usage/
https://datacenters.microsoft.com/sustainability/efficiency/
Isn't water just the transfer medium between the server and the heat exchangers outside? How would changing that to oil help?
It wouldn't really help.
Oil might be able to carry more heat but it's more expensive to use.
Oil immersion is something nerds like to think is amazing but it's just a pain in the ass for negligible benefits. Imagine the annoyance of doing maintenance.
Wouldn't it be no different but your hands get a little oily? Say you take out a ram stick, oil goes into the empty dimm slot, but so what because its displaced again when you put in the new ram stick.
>you might think it’s like measuring a car’s fuel economy or a dishwasher’s energy rating: a knowable value with a shared methodology for calculating it. You’d be wrong.
But everyone knows fuel economy is everything but a knowable value. Everything from if it has rained in the past four hours to temperature to loading of the vehicle to the chemical composition of the fuel (HVO vs traditional), how worn are your tires? Are they installed the right way? Are your brakes lagging? The possibilities are endless. You could end up with twice the consumption.
By the way, copy-pasting from the website is terrible on desktop firefox, the site just lags every second, for a second.
fuel economy, like blood glucose levels, is impacted by many factors, but you can measure it over time. you might not be able to prescribe a course of action but you can make corrections to the course you're already on.
I wonder how the energy requirements are distributed between training and inference. Training should be extremely flexible, so one can only train when the sun shines and nobody uses the huge amount of solar power, or only when the wind turbines turn.
AFAICT the energy cost of training is still fairly low compared to cost of GPU's themselves so especially during a land grab it's important to drive as near as possible full utilization of the GPU's, energy be damned.
I doubt this is going to change.
That said, the flip side of energy cost being not a big factor is that you could probably eat the increase of energy cost by a factor of say 2 and this could possibly enable installation of short term (say 12h) battery storage to enable you to use only intermittent clean energy AND drive 100% utilization.
the numbers in the article are all over the place. I mean the article seems to try and some of the more general calculations on paper should work out but especially the image gen ones I can sorta disprove with my own experiences in local gen.
Even were it matches sorta (the 400 feet e-bike thing) that only works out for me because I use an AMD card. An NVIDIA card can have several times the generation speed at the same power draw so it all falls down again.
And the parameters they tried to standardize their figures with (the 1024x1024 thing) is also a bit meh because the SAME amount of pixels in a different aspect ratio can have huge variations in gen speed and thus power usage. for instance for most illustrious type checkpoints the speed is about 60% higher at aspect ratios other than 1024x1024. Its all a bit of a mess.
Well, this was disappointing:
> There is a significant caveat to this math. These numbers cannot serve as a proxy for how much energy is required to power something like ChatGPT 4o.
Otherwise this is an excellent article critiquing the very real problem that is opacity of these companies regarding model sizes and deployments. Not having an honest accounting of computing deployed worldwide is a problem, and while it's true that we didn't really do this in the past (early versions of Google searches were undoubtedly inefficient!), it's not an excuse today.
I also wish this article talked about the compute trends. That is, compute per token is going significantly down, but that also means use of that compute can spread more. Where does that lead us?
I'm glad that the needs of AI and the sustainable capabilities of nuclear fission go well together.
> This leaves even those whose job it is to predict energy demands forced to assemble a puzzle with countless missing pieces, making it nearly impossible to plan for AI’s future impact on energy grids and emissions. Worse, the deals that utility companies make with the data centers will likely transfer the costs of the AI revolution to the rest of us, in the form of higher electricity bills.
... So don't? Explicitly shift the cost to the customer.
If I want to hook up to the energy grid with 3-phase power, I pay the utility to do it.
If a business wants more power and it isn't available, then the business can pay for it.
Then only businesses that really need it will be willing to step up to the plate.
No amount of "accounting" or "energy needs prediction" will guard against regulatory capture.
Might have missed it but was disappointed to see no mention of externalized costs like the scraping burden imposed on every IP-connected server. From discussions on HN this sounds quite substantial. And again, why exactly should the few AI companies reap all the value when other companies and individuals are incurring costs for it?
This series of articles is driving me insane. The authors or editors are using inappropriate units to shock readers: billions of gallons, millions of square feet. But they are not putting the figures into context that the reader can directly comprehend. Because if they said the Nevada data centers would use 2% as much water as the hay/alfalfa industry in Nevada then the entire article loses its shock value.
What’s the net energy footprint of an employee working in an office whose job was made redundant by AI? Of course that human will likely have another job, but what’s the math of a person who was doing tedium solved by AI and now can do something more productive that AI can’t necessarily do. In other words, let’s calculate the “economic output per energy unit expended.”
On that note, what’s the energy footprint of the return to office initiatives that many companies have initiated?
> a person who was doing tedium solved by AI and now can do something more productive that AI can’t necessarily do.
Like driving a uber or delivering food on a bicycle ? Amazing!
> Of course that human will likely have another job, but what’s the math of a person who was doing tedium solved by AI and now can do something more productive that AI can’t necessarily do
That’s a lot of big assumptions - that the job getting replaced was tedious in the first place, that those other “more productive” job exists, that the thing AI can’t necessarily do will stay that way long enough for it not to be taken over by AI as well, that the tediousness was not part of the point (e.g. art)…
When human civilization crashes due to yearly climate change caused famines it won't matter how useful the work done by the AIs was.
Net energy change of people doing work on their desk versus browsing the internet versus playing games, you will likely not see difference at all. They're all at rest, more or less thinking something. People at home sofa always have metabolic processes running regardless of whether it produces additional value to some corporation
Weird I was assured that Bitcoin would be using all of the worlds electricity by now.
Which I already thought was odd, because London would need all that electricity to see through the giant mountain of poop piled up by all the horses the british use for transportation.
The book “AI Atlas” covers the energy and other costs of AI.
You mean this one? https://en.m.wikipedia.org/wiki/Atlas_of_AI
It's from 2021 so won't cover the 2022-onwards generative AI boom.
From the Wikipedia summary it sounds like it's about machine learning algorithms like classification, AlphaGo and concerns about ethics of training and bias.
Link?
Interesting, thanks for sharing! I share some concerns others have about this piece, but I’m most shocked about their finding that image generation is cheaper than text. As someone who’s gone down this rabbit hole multiple times, this runs against every single paper I’ve ever cited on the topic. Anyone know why? Maybe this is a recent change? It also doesn’t help that multimodal transformers are now blurring the lines between image and text, of course… this article doesn’t even handle that though, treating all image models as diffusion models.
The point that stood out to me as concerning was
"The carbon intensity of electricity used by data centers was 48% higher than the US average."
I'd be fine with as many data centers as they want if they stimulated production of clean energy to run them.
But that quote links to another article by the same author. Which says
"Notably, the sources for all this power are particularly “dirty.” Since so many data centers are located in coal-producing regions, like Virginia, the “carbon intensity” of the energy they use is 48% higher than the national average. The paper, which was published on arXiv and has not yet been peer-reviewed, found that 95% of data centers in the US are built in places with sources of electricity that are dirtier than the national average. "
Which in turn links to https://arxiv.org/abs/2411.09786
Which puts the bulk of that 48% higher claim on
"The average carbon intensity of the US data centers in our study (weighted by the energy they consumed) was 548 grams of CO2e per kilowatt hour (kWh), approximately 48% higher than the US national average of 369 gCO2e / kWh (26)."
Which points to https://ourworldindata.org/grapher/carbon-intensity-electric...
For the average of 369g/KWh. That's close enough to the figure in the table at https://www.epa.gov/system/files/documents/2024-01/egrid2022...
which shows 375g/KWh (after converting from lb/MWh)
But the table they compare against shows.
and the EPA table shows Which seem more likely to be true. The paper has California at only marginally better than the national average for renewables (Which I guess they needed to support their argument given the number of data centers there)I like arxiv, It's a great place to see new ideas, the fields I look at have things that I can test myself to see if the idea actually works. I would not recommend it as a source of truth. Peer review still has a place.
If they were gathering emissions data from states themselves, they should have caclulated the average from that data, not pulled the average from another potentially completely different measure. Then their conclusions would have been valid regardless what weird scaling factor they bought in to their state calculations. The numbers might have been wrong but the proportion would have been accurate, and it is the proportion that is being highlighted.
there are still negative externalities to high renewable-energy usage (heat and water usage, which itself requires energy to purify once returned to the sewer, plus the environmental impact of building an enormous heat island in places where there was little industry previously).
[dead]
[dead]
[flagged]
Today Google launched a model, Gemma 3n, that performs about as good as SOTA models from 1-2 years ago that runs locally on a cell phone.
Training SOTA models will, like steel mills or other large industrial projects, require a lot of environmental footprint to produce. But my prediction is that over time the vast majority of use cases in the hands of users will be essentially run on device and be basically zero impact, both in monetary cost and environment.