Lindy’s Law is an absolute gem, that I'm keeping.<p>If we don't understand the fundamental limits to any particular kind of trend, our default assumption should be that it will continue for about as long as it has gone on already.<p>We can, in fact, easily put a confidence interval on this. With 90% odds we're not in the first 5% of the trend, or the last 5% of the trend. Therefore it will probably go on between 1/19th longer, and 19 times longer. With a median of as long as it has gone on so far.<p>This is deeply counterintuitive. When we expect something to last a finite time, every year it goes on, brings us a year closer to when it stops. But every year that it goes on properly brings the expectation that it will go on for a year longer still.<p>We're looking at a trend. We believe that it will be finite. Our intuition for that is that every year spent, is a year closer to the end. But our expectation becomes that every year spent, means that it will last yet another year more!<p>How can we apply that? A simple way is stocks. How long should we expect a rapidly growing company, to continue growing rapidly?
It's an interesting idea, and it may be something that could be mathematically justified, but I do think this is an abuse of Lindy's Law in the absence of such a justification. Per Wikipedia [1]:<p>"The Lindy effect applies to non-perishable items, like books, those that do not have an "unavoidable expiration date"."<p>And later in the article you can see the mathematical formulation which says the law holds for things with a Pareto distribution [2]. I'd want to see some sort of good analysis that "the life span of exponential growth curves" is drawn from some Pareto distribution. I don't think it's completely out of the question. But I'm also nowhere near confident enough that it is a true statement to casually apply Lindy's Law to it.<p>[1]: <a href="https://en.wikipedia.org/wiki/Lindy_effect" rel="nofollow">https://en.wikipedia.org/wiki/Lindy_effect</a><p>[2]: <a href="https://en.wikipedia.org/wiki/Pareto_distribution" rel="nofollow">https://en.wikipedia.org/wiki/Pareto_distribution</a>
The analysis in the article explains why it applies to any phenomena that we might be able to notice.<p>The argument given is the same as the one that I first ran across, not by that name, in <a href="https://www.nature.com/articles/363315a0" rel="nofollow">https://www.nature.com/articles/363315a0</a>. <a href="https://en.wikipedia.org/wiki/Doomsday_argument" rel="nofollow">https://en.wikipedia.org/wiki/Doomsday_argument</a> claims that it was a rediscovery of something that was hypothesized a decade article.<p>I hadn't tried to give it a name, or thought to apply it outside of that context.<p>As for the mathematical qualms, I'm a big believer in not letting formal mathematical technicalities get in the way of adopting an effective heuristic. And the heuristic reasoning here is compelling enough that I would like to adopt it.
You can do that but you're laundering ignorance into precise-seeming mathematics. Better to just say "we're probably somewhere in the middle, not at the beginning or end" and leave it at that. Calling a peak is hard.
You speak about laundering ignorance into precise-seeming mathematics as if it was a bad thing.<p>But that's the entire idea of Bayesian reasoning. Which has proven to be surprisingly effective in a wide range of domains.<p>I'm all for quantifying my ignorance, and using it as an outside view to help guide my expectations. Read the book <i>Superforecasting</i> to understand how effective forecasters use an outside view to adjust their inside view, to allow them to forecast things more precisely.
This is the exact same heuristic used in CPU scheduling.<p>We expect fresh processes to terminate quickly and long running processes to last for a while longer.
I think an interesting thing about recent AI developments is that its all happening right as we hit the diminishing returns side of another "exponential that's actually a sigmoid" which is Moore's law.<p>The naive expectation is that AI will slow down b/c Moore's law is coming to an end, but if you really think about the models and how they are currently implemented in silicon, they are still inefficient as hell.<p>At some point someone will build a tensor processing chip that replaces all the digital matmuls with analogue logamp matmuls, or some breakthrough in memristors will start breaking down the barrier between memory and compute.<p>With the right level of research funding in hardware, the ceiling for AI can be very high.
Even at orders of magnitude greater speed, we've still hit diminishing returns for quality of output. We simply haven't found anything like superhuman reasoning ability, just superhuman (potentially) reasoning speed.
It's not that easy to assess diminishing returns with saturated benchmarks where asymptoting to 100% is mathematically baked in. I could point to the number of Erdos proofs being solved by AI going from 0 to many very recently as evidence for acceleration.
Possibly - but we've also seen that spending more tokens on a task can improve the quality of the output (reasoning, CoT, etc).<p>So it's not impossible to have things that seem orthogonal, like generation speed or context length, have an impact on quality of result.
they already did put a model into the silicon and it's crazy fast. <a href="https://chatjimmy.ai/" rel="nofollow">https://chatjimmy.ai/</a><p>I'm pretty sure there's a 3 year design goal starting this year that'll do that to any of the qwen, deepseek, etc models. There's a lot you could do with sped up models of these quality.<p>It might even be bad enough that the real bubble is how much we don't need giant data centers when 80-90% of use cases could just be a silicon chip with a model rather than as you say, bloated SOTA
I don't know what the Y-axis is supposed to be on that Wharton AI capabilities graph, but I am not really convinced that Opus 4.6 has more than double the intelligence/capability/whatever of GPT 5.1 Max.
IIRC that graph tracks capabilities as time_to_solve a task for humans (i.e. the model can now handle tasks that usually take a human ~8h). Which, depending on what tasks you look at, could be a reasonable finding. I could see Opus 4.6 handling tasks that take ~8h for humans, and that 5.1 couldn't previously handle (with 5.1 being "limited" at 4h tasks let's say). It is a bit arbitrary, but I think this is what they're tracking.
"It is a bit arbitrary, but I think this is what they're tracking."<p>I don't know if they can get their numbers right this way, but this seems a way more useful metric, than theoretic capabilities.
Without knowing more about their methodology, it seems like a lot of the recent improvements have involved the AI itself taking time to complete the task.<p>At first the models turned a 5 minute task into a 5 second task (by 5 seconds I mean a very short amount of time, not precisely 5 seconds). Then they turned a 15 minute task into a 5 second task.<p>Opus 4.6 completes 8 hour tasks all the time but (at least in my experience) it isn't spitting the answer out in 5 seconds anymore. It's using chain of thought and tools and the time to completion is measured in minutes or maybe hours.<p>In my experiments with local LLMs, a substantial part of the gap between frontier and local (for everyday use) is in tooling and infrastructure.<p>That is why I am sympathetic to the idea we are leveling off. But to bring in the air speed example from the article, I don't think we've reached the equivalent of the ramjet yet. I suspect in the coming years there will be new architectures, new hardware, and new ways to get even more capable models.
I don't know why people are so impressed by 8h.<p>I trained an LLM to write the whole Harry Potter series, and that took JK Rowling like 17 years.<p>For my next point on the graph, I'll train the LLM to write the Bible, something that took humans >1500 years.
Check out Re-Bench and HCAST.<p>The tasks are obviously all of the form "Go do this, and if you get the following output you passed". Setting up a web server apparently takes 15 minutes for a human, which is news to me since I'm able to search for <a href="https://gist.github.com/willurd/5720255" rel="nofollow">https://gist.github.com/willurd/5720255</a>, find the python one-liner, and copy it within about ten seconds.<p>Anyway, this is cool but it does not mean Claude can perform any human tasks that take less than 8 hours and are within its physical capabilities.
<a href="https://podcasts.apple.com/us/podcast/machine-learning-street-talk-mlst/id1510472996" rel="nofollow">https://podcasts.apple.com/us/podcast/machine-learning-stree...</a> is a pretty good primer on METR, what it measures, and its limitations.
> more than double the intelligence/capability/whatever<p>I'm curious what people really mean when they say this. Intelligence is famously hard to define, let alone measure; it certainly doesn't scale linearly; it only loosely correlates to real-world qualities that are easy to measure; etc. Are you referring to coding ability or...?
According to this article: whenever someone games a benchmark to make an upward chart on some y-axis, it's YOUR responsibility to prove how and why that trend can't continue indefinitely.<p>emoji face with eyes rolling upward
Seems to me that the default is "I don't know what's going to happen" and if you're making a confident prediction, bring evidence.<p>Scott makes a Lindy effect argument which is plausible, but don't let that fool you, we still don't know what's going to happen.
I'm pretty sure that gaming benchmarks <i>can</i> continue indefinitely.
<a href="https://metr.org/time-horizons/" rel="nofollow">https://metr.org/time-horizons/</a> on linear scale. Clickbait garbage article as most of his in the last year.
> all exponentials eventually become sigmoids<p>Except innovation. When one sigmoid tapers off we keep finding new ones to keep the climb going.
> It’s true that birth rates must eventually flatten out and become sigmoid<p>All positive growth eventually flattens out and becomes sigmoid, but a lot of phenomena experience negative growth and nose dive. No gentle curve, but a hard kink and perfect flat line at zero. Forever. I think it would be a stretch to categorize that pattern as sigmoid. Predicting a sigmoid pattern for negative growth implies some sort of a soft landing (depending on your definition of soft).<p>We can think of many populations that are no longer with us. So just a caution about over applying this reasoning in the negative case.
> What if you don’t fully understand the process? AI forecasters know some things (like how data centers work and how much it costs to build them). But they’re unsure about other things (researchers keep inventing new paradigms of data generation that get over data walls, but for how long?), and other things are entirely opaque (What is intelligence really? Why do scaling laws work? Might they just stop working at some point?) Is there anything you can do here?<p>This is the crux of the article. To a large extent continued progress depends on a stable increase in compute, an increase in training data, and an increase in good ideas to squeeze more out of both of them.<p>One calculation you could do is a survival function: for each of the above, how long before it is disrupted? For example, China could crack down on AI or invade Taiwan. Or data centers become politically unpopular in the US. Or, we could run out of great ideas. Very hard to predict.
We did hit the sigmoid's plateau on airplane speed, but the applications of airplane speed are still coming (how fast can a Chinese company airship the PCB you ordered three minutes ago?). I expect the the same will happen with LLMs, though I also happen to believe things are just getting started on end capabilities.
But they do explain the improvement of AI driving 2017-2021 vs 2022-2026.
> then what is their model?<p>My mental model has been 3D computer graphics: doubling the polygon count had huge returns early on but delivered diminishing returns over time.<p>Ultimately, you can't make something look more realistic than real.<p>I don't know what the future holds, but the answer to the question "can LLMs be more realistic than real" will determine much about whether or not you think the curve will level off soon.
John D Cook gives more technical details here: "Trying to fit a logistic curve" <a href="https://www.johndcook.com/blog/2025/12/20/fit-logistic-curve/" rel="nofollow">https://www.johndcook.com/blog/2025/12/20/fit-logistic-curve...</a>
The curve is a smoothed step curve (y=1 if x>1 otherwise 0). Nature doesn't allow any change to happen instantly at any degree of rate of change. The curveis just a manifestation a change with exponential smoothening of the sharp corners.<p>For example, When a car starts, it's speed and acceleration become more than zero. But what about rate of change in higher degrees? It suddenly doesn't change from zero acceleration to non-zero. That means the car has a non-zero derivative at all degrees. In other words, the movement is exponential. The same thing happens in reverse when the car reaches a constant speed.
If the scary AI is so inevitable, why do you feel such an overwhelming need to convince people about that? Surely you can just wait a bit, and they'll see for themselves.
By that reasoning, why even warn people about anything? Why do road construction crews put up signs saying "ROAD CLOSED AHEAD" when you can just drive on and see for yourself?
1. It's not inevitable.
2. Those that see AI as an existential risk don't generally think it's a guarantee, but if it's say a 5% chance then that's worth addressing/mitigating.
3. That's not what this article was even about.
Sounds like the burden is on you to explain either<p><pre><code> 1. If you're not treating my claim as a black box, explain explicitly what is your model of what the article was about? Are you aware, for example of the last paragraph of the article? I think that WAS what the article was about. Do you have specific opinions on e.g. how I went wrong and where my model differs?
2. If you are treating it as a black box, what's your default expectation based on the law of Nothing Ever Happens?
</code></pre>
Just kidding, you don't need to explain anything. A"I" fearmongers should though.
The point of the article is that people are historically bad at predicting when exponential curves plateau, even if they're correct that there will be a plateau.<p>This does *not* imply the inevitability of AGI. It does not imply AGI is necessarily bad.<p>It does mean that "the capabilities of AI will eventually plateau" offers no meaningful predictive power or relevance to the overall AI discussion.
Well, curve shape aside, the high watermark might be lower than where it tapers off.<p><a href="https://news.ycombinator.com/item?id=46199723">https://news.ycombinator.com/item?id=46199723</a>
Stein's Law: "If something cannot go on forever, it will stop."
News flash: predicting the future is hard
The other thing people don’t understand is exponential curves are self similar. The start of an exponential looks like an exponential. People always look at and think ‘well that’s it it’s exponential now, have missed it, can’t sustain’. Nope.<p>Good example of this is number of submissions to neurips/icml/iclr. In 2017 that curve was exponential.
Hmmm, this is quite an interesting take by Scott.<p>Lindy's Law is not actually a law and many exact minds will be provoked by the very name; it also fails spectacularly in certain contexts (e.g. lifetime of a single organism, though not necessarily existence of entire species).<p>But at the same time, I am willing to take its invocation in the context of AI somewhat seriously. There is an international arms race with China, which has less compute, but more engineers and scientists. This sort of intellectual arms race does not exhaust itself easily.<p>A similar space race in the 1950s and 1960s progressed from first unmanned spaceflight to a moonwalk in mere 12 years, which is probably less than what it takes to approve a bicycle lane in Chicago now.
It's not a law per se, but there are rules for reasoning under uncertainty to get the most out of what limited knowledge you have, and Lindy's law arises from that. To do better than Lindy's law requires having additional information about the problem beyond just the one data point.
"There is an international arms race with China"<p>I keep seeing this. Where did it come from? Has China said that they intend to attack other countries using AI? Have other countries declared that they intend to attack China with AI?<p>Also, why does anyone believe that AI could actually be that dangerous, given it's inherent unpredictable and unreliable performance? I would be terrified to rely on AI in a life or death situation.
AI in war is like Palintirs whole business model. You have a system that can effectively deal with ambiguity and has superhuman performance on reasoning plus superhuman physical abilities via embodiment…<p>Inherent unpredictable and unreliable performance is also quite the feature of human beings as well.
<a href="https://www.forbes.com/sites/greatspeculations/2025/11/25/why-the-us-china-ai-arms-race-is-entering-a-critical-new-phase/" rel="nofollow">https://www.forbes.com/sites/greatspeculations/2025/11/25/wh...</a>
It was a metaphor. I meant, and later clarified, an intellectual arms race.<p>BTW your handle is an actual Czech word, minus a diacritic sign ("křupan"), and a bit amusing one. It basically means hillbilly. Not that it matters, just FYI.<p>Anyway: AI <i>will</i> be used in military context, and it probably already is. Both for target acquisition and maybe even driving the weapon itself. As of now, the Ukrainians are almost certainly operating some AI-enabled killer drones.
"Exponentials all tend to become sigmoids but you can't predict exactly when" is a true statement, but I'm not sure it needed an article.<p>This doesn't say much, and the author fights their own points a couple times, suggesting that they maybe didn't think through what they wanted to write until they were in the middle of writing it and started realizing their assumptions didn't match what they expected the data to say.<p>I really don't get the point of what I just read.
The point is the tiring arguments from AI skeptics saying “things are flattening, they have to” which while technically correct says nothing because no one knows when that will happen and we see no mechanism for this yet. Lindy’s law as a reasonable prediction under total uncertainty is interesting and insightful and a lot of people don’t know about it or why it holds. I did enjoy the reference to this!
A lot of words to say "The initial part of a sigmoidal curve is not very informative about the parameters of the sigmoid function in question."
That is true, but I generally enjoy reading a lot of words from Scott, who has a talent for writing.<p>The entire plot of the Lord of the Rings could probably be compressed into less than 10 kB of text too.<p>Edit: this seems to be a controversial comment, but IMHO a blog of Scott Alexander's type is an art form, not just a communication channel.
I find him more interesting when he talks about non-AI topics. Lots of other interesting people are like this too. I'd rather get my knowledge on AI from people who have unique insights into it. Scott has a lot of unique perspectives of his own, but his views on AI are bog-standard for his social group.
<a href="https://xkcd.com/605/" rel="nofollow">https://xkcd.com/605/</a>
If you use the log scale you'll see that the time horizon of opus 4.6 was as expected...
[dead]
Why is this author tolerated on Hacker News? He's not actually knowledgable about 99% of subjects he posts about.
I think there are many ways someone with his lack of expertise can still be valuable, including:<p>- Making connections to other subjects that an expert would miss. The hall of fame of sigmoid predictions is just excellent, I already know I'm going to be reminded of it some time in the future. Very entertaining way to get the point across.<p>- Writing about tricky concepts in a very accessible and elegant way, which experts are notoriously bad at doing themselves - they are often optimizing for other specialists.<p>- Being able to write with an air of speculation and experimentation with ideas that experts and institutions often can't afford. Experts have to maintain their track record; Scott Alexander can say "lol just double the timeline"
you do you, I don't come here for superficially informed-looking articles written by people who are in fact not experts, informed or educated, I come here for the real deal<p>it doesn't help that sCotT aLexAndEr is also as close as you can come to the modern dressed up version of a eugenicist (again, not based on any actual expertise)<p>but I rest my case
Because HN is YCombinator which has invested in probably hundreds of «AI» firms by now. Including OpenAI.<p>Allowing slop articles like this literally prints them evaluation money.