An LLM isn’t imagining anything, it’s sorting through the enormous collection of “imaginations” put out by humans to find the best match for “your” imagination. And the power used is in the training, not in each generation. Lastly, the training results in much more than just that one image you can’t stop thinking about, and you’d find the best ones if you could prompt better with your little brain.
I’m curious whose feelings I hurt. The anti-AI crowd? Certainly they’d agree with my point of LLMs not thinking. Users of LLMs? I hope most of you understand how the tool works. Maybe it’s the meme crowd who just wanted everyone to chuckle and not think about it too much.
The training is a huge power sink, but so is inference (I.e. generating the images). You are absolutely spinning up a bunch of silicon that’s sucking back hundreds of watts with each image that’s output, on top of the impacts of training the model.
It depends on the model but I’ve seen image generators range from 8.6 wH per image to over 100 wH per image. Parameter count and quantization make a huge difference there. Regardless, even at 10 wH per image that’s not nothing, especially given that most ML image generation workflows involve batch generation of 9 or 10 images. It’s several orders of magnitude less energy intensive than training and fine tuning, but it is not nothing by any means.
An LLM isn’t imagining anything, it’s sorting through the enormous collection of “imaginations” put out by humans to find the best match for “your” imagination. And the power used is in the training, not in each generation. Lastly, the training results in much more than just that one image you can’t stop thinking about, and you’d find the best ones if you could prompt better with your little brain.
I’m curious whose feelings I hurt. The anti-AI crowd? Certainly they’d agree with my point of LLMs not thinking. Users of LLMs? I hope most of you understand how the tool works. Maybe it’s the meme crowd who just wanted everyone to chuckle and not think about it too much.
What is it you think the brain is doing when imagining?
The training is a huge power sink, but so is inference (I.e. generating the images). You are absolutely spinning up a bunch of silicon that’s sucking back hundreds of watts with each image that’s output, on top of the impacts of training the model.
The inference takes <10 wH aka pretty much nothing.
It depends on the model but I’ve seen image generators range from 8.6 wH per image to over 100 wH per image. Parameter count and quantization make a huge difference there. Regardless, even at 10 wH per image that’s not nothing, especially given that most ML image generation workflows involve batch generation of 9 or 10 images. It’s several orders of magnitude less energy intensive than training and fine tuning, but it is not nothing by any means.