To add a bit more constructive feedback, think of the "AI cornucopia" and "Superintelligence destroys humanity" as opposite ends of all possible outcomes distribution (low probability event).
The most likely one is the "business as usual, but with AI" - some things will get better, some things will get worse, but overall state of affairs will remain mostly the same.
The people who've thought the most about this put it differently:
Think of a new, superintelligent model as if it was a new v1 Starship launching for the first time, with a full fuel tank. On the one hand, rockets have existed for some time, and some have gone to space successfully, including by this company.
On the other hand, this is a tube of metal full of highly explosive liquid going faster than most human objects ever go, for the first time ever in this novel and state of the art configuration.
If someone said, "really, the first Starship exploding is just at one end of the probability distribution, where the other is that everything goes fine and all its passengers have a nice trip in space," would you get on that rocket?
Or, more aptly, if you and every other living human was already on that rocket, would you push the launch button?
The analogy works because superintelligence is, like rocket fuel, an extremely powerful force that has a default tendency to break containment and go boom (consume lots of energy and heat and matter in a chain reaction, to pursue more intelligence to pursue whatever goal it is pursuing.)
We can learn from history: Albert Einstein famously tricked humanity into building nuclear weapons for him and was only prevented from wiping out all sentient life by the Princeton IAS Board of Alignment who published a very compelling blog post about realigning the A-bomb contra paperclips.
I would like to classify this as “tail risk fallacy” because it exploits humans natural tendency to be risk averse. No matter how small the tail risk is, the fact that it exists can be justified to stop whatever you want to stop. A thoughtful person might say there’s a threshold at which you might want to care about it but that’s not how ideas spread.
Take climate change: you can spread whatever dubious claims about tail risk being whatever probability with amateurish models and use that to justify degrowth. This scheme actually works - most people in the west are already brainrotted by the tail risk discourse in climate change.
Current AIs are closer to bottle rockets than to the Starship on the intelligence scale. Some property damage already happened, but you can't master the art of rocketry without trial and error.
Eh. An intelligence with perfect knowledge of the entire universe, unlimited memory, and infinite processing speed, would have a tendency to go boom. But the real world has limits, and intelligence, even superintelligence, does not equate to godhood. Some tasks are still hard no matter how smart you are.
Plus, we have no real reason to think LLMs are anywhere close to AGI or ASI. So arguments like these are just distracting from the very real, very present danger that LLMs pose: information breakdown, societal collapse and environmental destruction. In other words, this is criti-hype.
The kind of person who thinks AI contributes to information breakdown rather than information diffusal has no theory of how information spreads. The other claims like societal collapse and environmental destruction are still FUD but slightly more possible
This is the sort of pseudo-scientific reasoning by analogy that leads Elizier Yudkowsky to argue we should be terrified that an unfriendly ASI could rapidly develop "diamondide" nanobot viruses, because it will be super-intelligent and super-intelligence can do anything by first-principles:
> The concrete example I usually use here is nanotech, because there's been pretty detailed analysis of what definitely look like physically attainable lower bounds on what should be possible with nanotech, and those lower bounds are sufficient to carry the point.
...
> The nanomachinery builds diamondoid bacteria, that replicate with solar power and atmospheric CHON, maybe aggregate into some miniature rockets or jets so they can ride the jetstream to spread across the Earth's atmosphere, get into human bloodstreams and hide, strike on a timer.
Thought experiments ungrounded by any realistic technological constraints or scientific evidence are pretty much useless for actual forecasting except as an exercise in sci-fi worldbuilding.
But will “business as usual, but with AI” justify the current capital expenditures? I think the market is pricing things as being closer to “AI cornucopia”.
There's never been much of a "market" for systems like this. Even at SpaceX scales, nobody would ever be able to sell enough chips to make the effort worthwhile. The only reason to do it is to fulfill a need in a niche market.
I’m sure Thomas needs an upgraded RV, so that’s easily handled. And the rest of the conservative ‘justices’ seem happy to betray the constitution for free.
And with modern analytics it’s harder to make up the numbers like they did. Also cheaper to transmit and centralize the data, especially with some IOT type stuff. Doesn’t make it immune to human nature, but modern tech reduces the friction and increases the transparency.
There is a book "The People's Republic of Walmart: How the World's Biggest Corporations are Laying the Foundation for Socialism" describing how e.g. Walmart (there is also Amazon now) does its own large-scale planning.
It's kinda funny that the "socialist calculation debate" has largely settled on "okay, it's probably possible but requires way too much of computational resources and accurate data gathering" — and now we spend insane amount of resources on AI data centers and tracking/surveillance on literally everything. And, of course, all of those resources won't end up being used for solving (or even accurately figuring out) the actual economical problems we face today.
Of course you can. E.g. the steel production in the US has been government-planned (well, DoD-planned) since the WW2, and it's not even that big of a secret, just not something that's mentioned often.
I feel the same about the apple watch. All I use it for is literally tracking steps. I do not care about any other feature. If my watch fails I will just get a casio watch.
Bluetooth stays on only for three minutes and then turns itself off once you're done syncing stuff. I have had one for nearly two years now (ABL-100WE). Absolutely love it.
I work with accelerometer chips, a lot of them have features specifically for pedometers. Keeping the accelerometer on will draw a few uA, then you can have the accelerometer keep track of the step count internally and have the main processor query it as necessary to update the step count display. When there's no motion for a while, you can also put it into a lower power sleep mode that doesn't track steps, then when it detects motion it'll wake up the main processor which will then put the accelerometer back into pedometer mode. I think 2 years on a CR2016 (90 mAh) is reasonable if Casio knows what they're doing.
If it's like the G-Shock, it's not a real step counter, it's a small accelerometer. It will trigger on every single wave when you're planing if you're windsurfing, which makes it quite useless.
I mean, airtags are doing radio stuff at a regular cadence. I've noticed they update once every 15 minutes, so they are turning the radio on to do whatever BLE triangulation they need to do 96 times per day. That's milliamps over at least a few milliseconds to wake the radio, do whatever triangulation is needed. It's amazing that they last that long considering what they are doing. I think they are using Nordic NRF54 internally so it's testament to the power efficiency those chips are capable of.
Airtags don't do any triangulation. They just send out a BLE beacon periodically to any iPhone within earshot, and also spend some time listening for new instructions that almost never happen.
They have no idea where they are or what their airtag/BLE neighbors, if present, are doing. They don't grok GNSS or wifi. There's nothing for them to triangulate or trilaterate or tri-anything when they're just sitting there.
So yes: They are indeed pretty tidy about energy consumption. They accomplish this in large part by being very simple in normal operation and leaving the heavier lifting to the world of iPhones with relatively enormous batteries that get recharged ~every day.
Either the most profitable scam in human history or holding all the wealth and all the power over an enslaved remnant of humanity. And they are wondering why people don't love them.
That is no excuse whatsoever. It turns out that immoral actions don't become acceptable if you go "someone else would do it if I didn't". Perhaps they would, but you're still a bad person if you do the bad thing.
...and AI progress is actually a much more difficult thing to control than nuclear proliferation...
Unlike nuclear bombs there is no rare difficult-to-process ingredient which you can practically limit the supply of, unless you are willing to retroactively outlaw computing devices broadly (after we have spent decades weaving them into the fabric of most people's day to day lives).
And even if you were willing to try that, how do you get global consensus retroactively? It is an impossible situation.
None of what I've written here should be seen as a defense of the people running large AI labs, I don't believe any of them are forces for positive good in the world, but it is just a fact that if they don't "do it" someone else will.
Just because we can't stop it completely doesn't mean we can drastically slow it down: make it illegal to study to be an AI researcher, to train someone to be a researcher, to seek investment for AI development, to invest in AI development, to sell services based on AI models other than the ones that already exist, to publish new findings about AI.
I have been playing Cyberpunk 2077, and recently I did a side-quest where you can do something that violates an agreement, and when you reveal what you learned during that violation, you can extort somebody. The dialogue option when you extort that person is "Surprise - it's money." If you choose that option, Johnny chastises you for compromising your moral code.
For some reason that phrase "Surprise - it's money" as the obvious signal of extortion really stuck with me.
its ironic how no one needs to justify having abundance and freeing humans from the shackles of work.
its no longer the wholesome chungus luxury space communism that we wanted is it?
reply