Philip Tetlock's Forecasting Research Institute conducted an exercise last year on the topic of human extinction events. Why? Who knows. Their report was released yesterday and, long story short, a group of various experts predicted a roughly 10-20% chance of catastrophe by 2100 (10% of human race dies) and a 1-6% chance of total extinction (fewer than 5,000 humans left). This is the total sum of the risk from five possible scenarios:
- Artificial intelligence
- Nuclear
- Engineered virus
- Natural virus
- Meteor, sun going nova, etc.
Let me say from the start that I think this is kind of nuts. I'd personally put the overall risk of extinction from everything combined at about 0.1% or so. But that's just me. I'm an eternal optimist, I guess.
The proximate motivation for the study, of course, was the recent surge of interest in the possibility of some future AI going nuts and killing everyone. AI experts overall predicted a 13% chance of AIs causing an extinction level event, and I found this breakdown instructive:
Apparently there's a group of AI experts who are absolutely obsessed with extinction and have spent more than 1,000 hours thinking about it. These people seem to have thought themselves into a frenzy and they're obviously driving up the average a lot. The median extinction forecast, which eliminates the influence of the extreme outliers, is only 3%. Still too high, I think, but not completely crazy.
Unlike some things, this is not a case where thinking harder does you any good. There's really no concrete evidence one way or the other about the chances of AI-caused extinction, just speculation, and going down an extinction rabbit hole is little more than a fast path to the booby hatch. Just keep in mind that any possible super-threat—AI or otherwise—will almost certainly be met by a countervailing super-defense. COVID-19 was a dangerous pathogen, but advancing technology allowed us to create a vaccine in record time. Nuclear weapons are being met with missile defense systems. Climate change is being met by renewables and, maybe someday, geoengineering. AIs will be met by other AIs. So chill.