Brad DeLong comments this weekend on a couple of essays that lament the difficulty of living in an era where technological progress is improving exponentially:
I read these, and I think: this is how it has been since the 1870s. Perhaps this is how it has been since the late 1830s.
Well, yes. But there's a difference. Back in 2013, in Part 1 of my robot trilogy,¹ I illustrated exponential growth using the analogy of filling up Lake Michigan. In the first year, you add one drop of water. Then two. Then four.
In the second to last year, you add a quintillion gallons of water. Then two quintillion. Then four. For five or six decades you can barely see anything more than a bit of sogginess. Then, in the final decade, suddenly the job is done and the lake is filled.
Both ends of the chart are exponential growth. But to human beings, the right edge is very, very different from the left. Exponential growth is much more impressive when it produces an elephant than when it produces a flea, even if, technically speaking, both are equally miraculous.
So go ahead and feel more disoriented than your great-great grandfather! After all, here's what Brad has to say about the wonders of ChatGPT:
The “entity” we are conversing with is very sub-Turing indeed. It is page-level autocomplete.
....I do anticipate the use of this page-level autocomplete will spread around the world, considerably faster than use of personal computers or use of the Internet. We are primed to have conversations....A kind of animal that attributes Turing-class intelligence to the lightning—which we, at least some of us, personified as a huge guy with a red beard and severe anger management problems—is primed to make much use of page-level autocomplete.
....Will we use it for good or ill? That is in our hands.
Quite so.
¹Part 2 is here. Part 3 is still in the works. Maybe next year?
That depends on whether the field follows exponential growth, or the S-curve for the hype cycle. Like with self-driving cars, we could be riding up to the peak of expectations, only for it to run into problems and then disappointment sets in with another AI winter.
I read that somebody did a little study, laying out the doubling problem as Kevin does here with Lake Michigan, but saying the increment is such that it fills in one year. Then he asked participants to say when the lake would be half full. Almost everyone said something like 2/3 or 3/4 of a year. Almost no one said correctly on day 364.
One of the lessons from COVID is that almost everybody thinks exponential just means fast.
Google paid 99 dollars an hour on the internet. Everything I did was basic Οnline w0rk from comfort at hΟme for 5-7 hours per day that I g0t from this office I f0und over the web and they paid me 100 dollars each hour. For more details
visit this article... https://createmaxwealth.blogspot.com
Exponential growth cannot continue indefinitely. The question is whether or not it can continue "long enough."
And as to the assertion you made in your 2013 chart - "Computing Power Doubles Every 18 Months" - well, that is Moore's Law, tweaked ever so slightly (IIRC he was saying the number of transistors doubles every N months). And that has not been keeping pace - our ability to continue to shrink the process has ... slowed.
Another thing - for a long time, these process shrinks were giving us power improvements - or at least, we could put 2x the number of transistors into the same number of Watts. I do not believe that is the case any longer.
Except as Moore's Law began to slow down, Huang's Law began to take over, which is actually accelerating processing power faster than Moore's Law would have predicted.
https://en.wikipedia.org/wiki/Huang%27s_law
Yet GPUs need the same process improvements as CPUs.
I haven't followed the data relevant to Moore's Law in a while, so I'm not disputing your central point. But for context, I do remember the first time a computer scientist told me that the growth rates in computing power we had been seeing couldn't continue, because we were finally bumping up against essential physical limitations, like the speed of light.
That was in 1981.
What it seems like no one asks is: could a world created by AI be worse than our current world? Not worse for humans, but worse overall. Something I try to explore in parts of my latest. https://www.losingmyreligions.net/
Happy last day of winter, all!
Lots of things follow a kind of normal curve - they increase exponentially for a while but they reach a peak and decline. There is no reason to assume that anything will keep increasing exponentially to some desired end point, which is what Kevin seems to be assuming about AI. Kevin doesn't know what obstacles there may be to the development of computers that can replace all human thinking, let alone surpass it.
Lots of businesses show exponential growth at the beginning. The sale of pet rocks probably did. But unless you got in on the ground floor investing in pet rocks was probably a loser.
What's the difference between a bird with a human sized brain and a human with a bird sized brain? If scale is all that matters, then I think we might need to be a lot more careful about the lack of regulation of AI. If there are many other processes involved in sentient intelligence, exponential growth of AI won't ever matter; it'll never reach sentience.
An elephant is more impressive than a flea, but when the flea was made it was more impressive than the mite and we were all unable to comprehend the leap forward of the newly made flea.
In the future we will look back on the simple elephant and contrast its simplicity with the wonder of the newly created jungle ecosystem.
The sense of disorientation isnt really any different across time.
Contemplate a chart for your futuristic AI to determine when to leave a soldier behind.
Are you willing to hard-code this into your futuristic AI, or will you leave the AI to make its own, non-lizard brain determination based on risks, not on credo?
And what does all this super fast technology get you? An series of financial catastrophes and instant bank runs based on AI generated rumors. And now… this is starting, just a little, to sound like 2008 all over again! Yeah! Depression recession 2023. We’re doomed. Let’s hope the republicans fail to raise the debt ceiling and we can blame it on them.
March 19, 2023 Updated 6:23 p.m. ET
WASHINGTON — The Federal Reserve and other major global central banks on Sunday announced that they would work to make sure dollars remain readily available across the global financial system as bank blowups in America and banking issues in Europe create a strain.
Justin - have you read a book called "The End of the World is Just the Beginning" by Peter Zeihan? If not, you should.
No, but after your suggestion I read some reviews. Thanks.
Zeihan ignores advances in technogy just like the climate alarmists.
My baby ducks experienced exponential growth for the first two weeks of their lives. At the rate that they were growing, in about 18 months they were set to exceed the mass of the known Universe.
But luckily there was a decrease in the time constant 2 weeks in, and then more like linear growth after the first 4 weeks, and a leveling off about 8 weeks in.
But it was pretty impressive to monitor for the first month or so.
Please forgive my ignorance. I do not know what "page-level autocomplete" is. Could someone please explain? A search of the interwebs was not enlightening.
Right now e-mail, or text chat apps, will autocomplete a word, based upon context of what you've written and maybe a couple of letters. I'm imagining that here one is envisioning a whole page of text, or simple code, or ..., being filled in based upon a few sentences.
Is the pace of change so "extraordinary" and "accelerating" that humanity is having trouble keeping up?
Maybe you could make that claim for my dad's generation. He was born a decade after the birth of aviation, in a small town with no car (but a horse!) and the nearest telephone at a store in the village. Fast forward 30 or so years, he returned from a war halfway around the world to the same town, soon a bustling suburb filled with cars (too many cars!) and people (the population zoomed from a few hundred to 10,000+), and before the kids were out of school he got to watch men walk on the moon live on the color television set in our living room.
How much change have we seen since then? Cars look different, clothes too, but my old town looks more or less the same. Life for most is same as it ever was. People get up, go to work or school, come home for dinner and watch TV. Culture is different. For TV, movies, music, there's more available then ever before (technology playing a part there) -- the monoculture is over -- but it's hard to say it's better now. Society is different. We are more equal across races, we've reduced poverty, as income inequality has grown and we've become openly hostile across our divisions.
It's easy to find a few new tech marvels (the internet, e.g.), but has the pace of technological change been extraordinary and accelerating? I don't think so. I'm still waiting for my jet pack.
The world today is more like the one I was born into than what my father could have said during his lifetime, or what his father could have said during his. All this talk about "exponential change" makes me think people are living in the past. The rate of change today feels like it's slowing down.
I know, I know. AI is right around the corner. Look at Lake Michigan!
For the record, I'm not anti-AI. I am anti-AI hype. I'm old enough to know when I read once again, "this changes everything," the appropriate response is skepticism.
+10
Brad uses 1870 as the start point. My grand dad was born in that year and almost lived to see the atom bomb and jet planes. I have a photo of him and his horse and cart in front of the Cass Pier along with other farmers in the Cayucos area. Wish he had got into real estate instead of farming.
Brad ends his "long 20th century" history in 2010, with the leading "countries of the North Atlantic, still reeling from the Great Recession that had begun in 2008, and therefore unable to resume economic growth anything near the average pace that had been the rule since 1870." With that the case, and the decade that followed one of relative slow growth, it's hard to make the case that we are now living in an era of "exponential growth" that is "impressive," "extraordinary," and "accelerating." A better way to describe the present is that we're in the slowest era of growth since at least 1870. Maybe that will change. We'll see.
It's been 10 years since Kevin predicted "robot paradise" will arrive by 2040. However you define that, the pace of change needs to accelerate quickly if that's going to happen.
My granddad was born in 1869 and lived till 1946. His older brother saw Lincoln's casket in NYC and they all saw the Brooklyn Bridge go up a few years later. Electrification, mass transit, skyscrapers, cinema, telephones, airplanes, and automobiles all arrived within a few decades. My granduncles had the misfortune of opening a livery stable in Brooklyn at the end of the 1800s. It didn't last, though the family held onto to the property until after WWII.
The biggest change this generation will see is climate change and sea level rise.
Heath care has advanced considerably--small pox eradicated, polio almost gone and, of all things, the guinea worm may be gone (thanks Jimmy!). Effective HIV therapies, much better vaccine development (but none yet for HIV), and cancer treatments (Car-T). But people don't really recognize they are not getting sick--so the achievements don't register.
As for technology improvements--I don't see much happening until we each can get our own "Mr. Fusion".
Did you notice the sea level rise of 1 inch per decade? The IPCC predicts 1 to 2 inches per decade out to 2100 and you won't notice that either.
I asked Brad if AI would be used to plan the coming Ukrainian attack.
We've been seeing exponential growth in some areas of technology. But does not apply to every area.
Airplanes. They can get a little more energy efficient, cabins can be held at higher pressures, and they can get bigger--though that's not what airlines want now. But they won't fly faster. Yes, there is (always) talk of the super sonic transport plane--but the bulk of flying will be subsonic. Why? It takes a lot more energy per passenger to fly super sonic.
Chatbots/LGM's. Training sets have to grow exponentially for linear improvements. Exponential increases in memory, storage, processing power, etc., means larger training sets can be used--but that won't lead to exponential increases in LGM's abilities.
Note: LGM's are not the be all and end all of AI. They probably will be relegated to being the interface to AI's.
As for emergent properties, see article below, we still need to wait to see if they are real or if we're getting better at getting what was want from a ChatBot.
https://www.quantamagazine.org/the-unpredictable-abilities-emerging-from-large-ai-models-20230316/
Grain of rice on the chess board. One grain on one square on day one. Each day, put double the previous days number on the next square.
It's not just a matter of human perception that the right side of an exponential curve is different from the left side. The rate of change counts. One drop per year is DIFFERENT from a quintillion gallons a year, even though both are points on the same curve.
And on that note, a little aside on exponential growth: it answers the question of what came before the Big Bang, and what came before that and so on ad infitum. The most common scenario these days is an inflationary period where the universe expanded by a factor of 10^26 in 10^-33 to 10^-35 seconds followed by reheating (the beginning of the big bang, gosh, gee willikers wow! goes the media.) It doesn't get as much press, but the answer to what came before inflation is 'nothing', because exponential inflation pushes the origin infinetly far into to the past.
From that 2013 article:
Given we are approaching 18 months from that year, presumably then, we are half way there?
Maybe not that long. There are days before I have my coffee I feel like a half-empty Lake Michigan myself.
The assumption in the graph is that once computers reach 2.88 x 10^17 calcs per second, they "have the same power as a human brain." That's wrong on many levels. It's a fundamental misunderstanding of what intelligence is. The human brain is not a super-fast and accurate calculating machine, yet it clearly has capabilities of brilliance, insight, and creativity. Computers, if they only could understand, would be jealous. The best of AI researchers have a humility about their challenges ahead. But who we hear most from sound like corporate hucksters.
Get it to work on Lake Mead.
Software does not exhibit exponential growth. It may look like it when one is trying to get a handle on an old, poorly maintained code base, but that's just the Augean Stables Illusion. It may get bigger and do more in less time, but it only gets better slowly. It's more punctuated equilibrium with an occasional leap followed by long periods of stasis or even decline.
Let's ask a similar question about another form of software: are novels getting better exponentially? For modern readers, 19th century novels are a definite improvement over those of the 18th century, but were 20th century novels exponentially better. Are novels of the 21st century even better still and improving more quickly? Are we in the slow part of the exponential improvement in novels and our descendants, someday, will experience an era in which for novel after novel, the improvement will be remarkable?
If biology gives us a clue, things tend to improve step by step with long quiescent periods. Modern plants still use rubisco to extract energy from sunlight. The basic chemistry is common to bacteria, archaea and eukaryotes, so it has only changed slightly over the last few billion years. You might move from flea to elephant, but most of the biology and chemistry are the same. Worse, there's no evidence that rubisco can be made more efficient, and folks have been trying for decades now.
I have no idea why people think that AI systems are going to improve exponentially just because we've seen, at long last, some improvement in their abilities. If you have enough water and fat enough pipes, it is rather obvious that one could fill all five of the Great Lakes in any arbitrary amount of time and with any chosen pattern of fullness. We don't have a supply of artificial intelligence sitting around somewhere to dump into ChatGPT 27.3. This argument doesn't hold water.
I'm skeptical about the exponential growth of anything relative to human life or experience, including data and technology (and as a data engineer I've wrangled with billions upon billions of not only data points but database records, which has only really led me to chuckle about the number of data points that many social science researchers and conclusions are dealing with).
https://tinyurl.com/278yhutr (World’s Population Hasn’t Grown Exponentially for at Least Half a Century)
You also have to ask yourself "what" is growing exponentially?
If people are your measure, then you also have to account for space. Given an environment where there is still available space, one could project faster growth. But there is a limit to people growth on this planet. At some point, available land not needed for food production, leisure and wilds preservation will run out, and then the only choice will be to grow up or down, not horizontally, as people can only fit into so much space before violence and/or contagion break out. Growing down isn't really an option except in science fiction, so building more concentrated cities with ever more high-rises would be the only solution, and could this even be done to keep up with the exponential growth rate, aside from feasibility concerns of the limits of how far that can go, unless you inhabit space, which is even further beyond as far as our capabilities than would be the pace of exponential growth? Luckily human population growth has not been exponential for some time now however.
In technology, there is a certain amount of growth going on now, but no reason to believe it's unbounded in any way. Most things should be understood as phenomenon of their time, as there are always constraints that may not be applying pressure now but most certainly will at some point down the road. The unbounded/infinite is more a horizon for math and imagination than it is reality.