LLMs have certainly become extremely useful for Software Engineers, they're very convincing (and pleasers, too) and I'm still unsure about the future of our day-to-day job.<p>But one thing that has scared me the most, is the trust of LLMs output to the general society. I believe that for software engineers it's really easy to see if it's being useful or not -- We can just run the code and see if the output is what we expected, if not, iterate it, and continue. There's still a professional looking to what it produces.<p>On the contrary, for more day-to-day usage of the general pubic, is getting really scary. I've had multiple members of my family using AI to ask for medical advice, life advice, and stuff were I still see hallucinations daily, but at the same time they're so convincing that it's hard for them not to trust them.<p>I still have seen fake quotes, fake investigations, fake news being spreaded by LLMs that have affected decisions (maybe, not as crucials yet but time will tell) and that's a danger that most software engineers just gross over.<p>Accountability is a big asterisk that everyone seems to ignore
I get this take, but given the state of the world (the US anyways), I find it hard to trust anyone with any kind of profit motive. I feel like any information can’t be taken as fact, it can just be rolled into your world view and discarded if useful or not. If you need to make a decision that can’t be backed out of that has real world consequences I think/hope most people are learning to do as much due diligence as reasonable. Llms seem at this moment to be trying to give reliable information. When they’ve been fine tuned to avoid certain topics it’s obvious. This could change but I suspect it will be hard to find tune them too far in a direction without losing capability.<p>That said, it definitely feels as though keeping a coherent picture of what is actually happening is getting harder, which is scary.
> I find it hard to trust anyone with any kind of profit motive.<p>As much as this is true, and i.e. doctors for sure can profit (here in my country they don't get any type of sponsor money AFAIK, other than having very high rates), there is <i>still</i> accountability.<p>We have built a society based on rules and laws, if someone does something that can harm you, you can follow the path to <i>at least</i> hold someone accountable (or, try).<p>The same cannot be said about LLMs.
<i>I feel like any information can’t be taken as fact, it can just be rolled into your world view and discarded if useful or not.</i><p>The concern, I think, is that for many that “discard function” is not, “Is this information useful?”. Instead: “Does this information reinforce my existing world view?”<p>That feedback loop and where it leads is potentially catastrophic at societal scale.
With code, even when it looks correct, it can be subtly wrong and traditional search engines don’t sit there and repeatedly pressure you into merging the PR.
> using AI to ask for medical advice<p>So the number of anti-vaxxers is going to plummet drastically in the following decade, I guess.
I haven't tried with this specific topic, but being the pleasers llms are, I doubt someone so focused on being anti-vaxxer will be convinced by an LLM, if anything, the LLM will give them reason at some point.
Depends if they use lobotomized bots like Grok...
>> So the number of anti-vaxxers is going to plummet drastically in the following decade, I guess.<p>> Depends if they use lobotomized bots like Grok...<p>What are you on about?<p>For instance, asking Grok "are vaccines safe", it has a pretty good reply, starting with <i>"Yes, vaccines are overwhelmingly safe and one of the most effective public health interventions in history. Extensive scientific evidence from decades of research, including rigorous clinical trials, post-licensure monitoring, and systematic reviews by organizations like the WHO, CDC, NIH, and independent bodies, shows that the benefits of vaccination far outweigh the risks for individuals and populations."</i> and then rounding out the conversation talking about Key Evidence on Safety and Benefits; Risks vs. Benefits; Addressing Concerns.<p><a href="https://grok.com/share/c2hhcmQtNA_69e20553-2558-46be-9f21-6ad92c470367" rel="nofollow">https://grok.com/share/c2hhcmQtNA_69e20553-2558-46be-9f21-6a...</a><p>When I then ask "I heard vaccines cause autism", it replies: <i>"No, vaccines do not cause autism. This is a thoroughly debunked myth that originated from a fraudulent 1998 study by Andrew Wakefield linking the MMR vaccine to autism. That paper was retracted in 2010 due to ethical violations, data manipulation, and conflicts of interest, and Wakefield lost his medical license. Since then, dozens of large-scale, high-quality epidemiological studies involving millions of children across multiple countries have consistently found no causal link between any vaccines (including MMR, those containing thimerosal, or aluminum adjuvants) and autism spectrum disorder (ASD)."</i><p>Seems pretty good to me.
> Programmers resistance to AI assisted programming has lowered considerably. Even if LLMs make mistakes, the ability of LLMs to deliver useful code and hints improved to the point most skeptics started to use LLMs anyway: now the return on the investment is acceptable for many more folks.<p>I'm not a fan of this phrasing. Use of the terms "resistance" and "skeptics" implies they were wrong. It's important we don't engage in revisionist history that allows people in the future to say "Look at the irrational fear programmers had of AI, which turned out to be wrong!" The change occurred because LLMs are useful for programming in 2025 and the earliest versions weren't for most programmers. It was the technology that changed.
"Skeptics" is also a loaded term; what does it actually mean? I find LLMs incredibly useful for various programming tasks (generating code, searching documentation, and yes with enough setup agents can accomplish some tasks), but I also don't believe they have actual intelligence, nor do I think they will eviscerate programming jobs, the same way that Python and JavaScript didn't eviscerate programming jobs despite lowering the barrier to entry compared to Java or C. Does that make me a skeptic?<p>It's easy to declare "victory" when you're only talking about the maximalist position on one side ("LLMs are totally useless!") vs the minimalist position on the other side ("LLMs can generate useful code"). The AI maximalist position of "AI is going to become superintelligent and make all human work and intelligence obsolete" has certainly not been proven.
No, that doesn’t make you a skeptic in this context.<p>The LLM skeptics claim LLM usefulness is an illusion. That the LLMs are a fad, and they produced more problems than they solve. They cite cherry picked announcements showing that LLM usage makes development slower or worse. They opened ChatGPT a couple times a few months ago, asked some questions, and then went “Aha! I knew it was bad!” when they encountered their first bad output instead of trying to work with the LLM to iterate like everyone who gets value out of them.<p>The skeptics are the people in every AI thread claiming LLMs are a fad that will go away when the VC money runs out, that the only reason anyone uses LLMs is because their boss forces them to, or who blame every bug or security announcement on vibecoding.
Skeptic here: I do think LLMs are a fad for software development. They're an interesting phenomen that people have convinced themselves MUST BE USEFUL in the context of software development, either through ignorance or a sense of desperation. I do not believe LLMs will be used long term for any kind of serious software development use cases, as the maintenance cost of the code they produce will run development teams into bankruptcy.<p>I also believe the current generations of LLMs (transformers) are technical dead ends on the path to real AGI, and the more time we spend hyping them, the less research/money gets spent on discovering new/better paths beyond transformers.<p>I wish we could go back to complaining about Kubernetes, focusing on scaling distributed systems, and solving more interesting problems that comparing winnings on a stochastic slot machine. I wish our industry was held to higher standards than jockeying bug-ridden MVP code as quickly as possible.
In this year of 2025, in December, I find it untenable for anyone to hold this position unless they have not yet given LLMs a good enough try. They're undeniably useful in software development, particularly on tasks that are amenable to structured software development methodologies. I've fixed countless bugs in a tiny fraction of the time, entirely accelerated by the use of LLM agents. I get the most reliable results simply making LLMs follow the "red test, green test" approach, where the LLM first creates a reproducer from a natural language explanation of the problem, and then cooks up a fix. This works extremely well and reliably in producing high quality results.
> They cite cherry picked announcements showing that LLM usage makes development slower or worse. They opened ChatGPT a couple times a few months ago, asked some questions, and then went “Aha! I knew it was bad!” when they encountered their first bad output instead of trying to work with the LLM to iterate like everyone who gets value out of them.<p>"Ah-hah you stopped when this tool blew your whole leg off. If you'd stuck with it like the rest of us you could learn to only take off a few toes every now and again, but I'm confident that in time it will hardly ever do that."
> No, that doesn’t make you a skeptic in this context.<p>That's good to hear, but I have been called an AI skeptic a lot on hn, so not everyone agrees with you!<p>I agree though, there's a certain class of "AI denialism" which pretends that LLMs don't do <i>anything</i> useful, which in almost-2026 is pretty hard to argue.
On the other hand, ever since LLMs came on the scene, there’s been a vocal group claiming that AI will become intelligent and rapidly bring about human extinction - think the r/singularity crowd. This seems just as untenable a position to hold at this point. It’s becoming clear that these things are simply tools. Useful in many cases, but that’s it.
> That's good to hear, but I have been called an AI skeptic a lot on hn, so not everyone agrees with you!<p>The context was the article quoted, not HN comments.<p>I’ve been called all sorts of things on HN and been accused of everything from being a bot to a corporate shill here. You can find people applying labels and throwing around accusations in every thread here. It doesn’t mean much after a while.
Not just their usefulness, but LLMs themselves are <i>worse</i> than an illusion, they are illusions that people often believe in unquestioningly - perhaps are being <i>forced</i> to believe in unquestionably (because of mandates, or short term time pressures as kind of race to the bottom).<p>When the ROI in training the next model is realised to be zero or even negative, then yes the money will run out. Existing models will soldier on for a while as (bankrupt) operators attempt to squeeze out the last few cents/pennies, but they will become more and more out of date, and so the 'age of LLMs' will draw to a close.<p>I confess my skeptic-addled brain initially (in hope?) misread the title of the post as 'Reflections on the end of LLMs in 2025'. Maybe we'll get that for 2026!
> The change occurred because LLMs are useful for programming in 2025<p>But the skeptics and anti-AI commenters are almost as active as ever, even as we enter 2026.<p>The debate about the usefulness of LLMs has grown into almost another culture war topic. I still see a constant stream of anti-AI comments on HN and every other social platform from people who believe the tools are useless, the output is always unusable, people who mock any idea that operator skill has an impact on LLM output, or even claims that LLMs are a fad that will go away.<p>I’m a light LLM user ($20/month plan type of usage) but even when I try to share comments about how I use LLMs or tips I’ve discovered, I get responses full of vitriol and accusations of being a shill.
It absolutely is culture war. I can easily imagine a less critical version of myself having ended up in that camp. It comes across to me that the perspective is informed by core values and principles surrounding what "intelligence" is.<p>I butted heads with many earlier on, and they did nothing to challenge that frame meaningfully. What <i>did</i> change is my perception of the set of tasks that <i>don't require</i> "intelligence". And the intuition pump for that is pretty easy to start — I didn't suppose that Deep Blue heralded a dawn of true "AI", either, but chess (and now Go) programs have only gotten even more embarrassingly stronger. Even if researchers and puzzle enthusiasts might still find positions that are easier for a human to grok than a computer.
Its also significantly lowered because management is forcing AI on everyone at gunpoint, and saying that you'll lose your job if you don't love AI<p>That's a very easy way to get everyone to pinky promise that they absolutely love AI to the ends of the earth
One only has to go read the original vibe coding thread[0] from ...ten months ago(!) to see the resistance and skepticism loud and clear. The very first comment couldn't be more loud about it.<p>It was possible to create things in gpt-3.5. The difference now is it aligns with the -taste- of discerning programmers, which has a little, but not everything, to do with technological capability.<p>[0]<a href="https://news.ycombinator.com/item?id=42913909">https://news.ycombinator.com/item?id=42913909</a>
"Look Ma, no hands!" vibe coding, as described by Karpathy, where you never look at the code being generated, was never a good idea, and still isn't. Some people are now misusing "vibe coding" to describe any use of LLMs for coding, but there is a world of difference between using LLMs in an intelligent considered way as part of the software development process, and taking a hit on the bong and "vibe coding" another "how many calories in this plate of food" app.
> The difference now is it aligns with the -taste- of discerning programmers<p>This... doesn't match the field reports I've seen here, nor what I've seen from poking around the repos for AI-powered Show HN submissions.
you just need to hop into any AI reltaed thread (even this one) and it's pretty clear no one is revising anything, skepticism is there lol.
There is some limited truth in this but we still see claims that LLMs are "just next token predictors" and "just regurgitate code they read online". These are just uninformed and <i>wrong</i> views. It's fair to say that these people were (are!) wrong.
> we still see claims that LLMs are "just next token predictors" and "just regurgitate code they read online". These are just uninformed and wrong views. It's fair to say that these people were (are!) wrong.<p>I don't think it's fair to say that at all. How are LLMs <i>not</i> statistical models that predict tokens? It's a big oversimplification but it doesn't seem <i>wrong</i>, the same way that "computers are electricity running through circuits" isn't a wrong statement. And in both cases, those statements are orthogonal to how useful they are.
<i>Objecting</i> to these claims is missing their point. Saying these things is really about denying that the LLMs "think" in any meaningful sense. (And the retorts I've seen in those discussions often imply very depressing and self-deprecating views of what it actually means to be human.)
Yes, it's a strange take. It's not that programmers have changed their mind about unchanging LLMs, but rather that LLMs have changed and are now useful for coding, not just CoPilot autocomplete like the early ones.<p>What changed was the use of RLVR training for programming, resulting in "reasoning" models that are now attempting to optimize for a long-horizon goal (i.e. bias generation towards "reasoning steps" that during training let to a verified reward), as opposed to earlier LLMs where RL was limited to RLHF.<p>So, yeah, the programmers who characterized early pre-RLVR coding models as of limited use were correct. Now the models are trained differently and developers find them much more useful.
These comments are a bit scary. It feels like LLMs managed to exploit some fault in the human psyche. I think the biggest danger of this technology is that people are not mentally equipped to handle it.
A list of unverifiable claims, stated authoritatively. The lady doth protest too much.
> * The fundamental challenge in AI for the next 20 years is avoiding extinction.<p>sorry, I say it's folding the laundry. with an aging population, that's the most, if not only, useful thing.
I have programmed 30K+ hours. Do LLMs make bad code: yes all the time (at the moment zero clue about good architecture). Are they still useful: yes, extremely so. The secret sauce is that you'd know exactly what to do without them.
"Do LLMs make bad code: yes all the time (at the moment zero clue about good architecture). Are they still useful: yes, extremely so."<p>Well, lets see how all the economics will play out. LLMs might be really useful, but as far as I can see all the AI companies are not making money on inference alone. We might be hitting plateau in capabilities with money being raised on vision of being this godlike tech that will change the world completely. Sooner or later the costs will have to meet the reality.
> but as far as I can see all the AI companies are not making money on inference alone<p>The numbers aren’t public, but from what companies have indicated it seems inference itself would be profitable if you could exclude all of the R&D and training costs.<p>But this debate about startups losing money happens endlessly with every new startup cycle. Everyone forgets that losing money is an expected operating mode for a high growth startup. The models and hardware continue to improve. There is so much investment money accelerating this process that we have plenty of runway to continue improving before companies have to switch to full profit focus mode.<p>But even if we ignore that fact and assume they had to switch to profit mode tomorrow, LLM plans are currently so cheap that even a doubling or tripling isn’t going to be a problem. So what if the monthly plans start at $40 instead of $20 and the high usage plans go from $200 to $400 or even $600? The people using these for their jobs paying $10K or more per month can absorb that.<p>That’s not going to happen, though. If all model progress stopped right now the companies would still be capturing cheaper compute as data center buildouts were completed and next generation compute hardware was released.<p>I see these predictions as the current equivalent of all of the predictions that Uber was going to collapse when the VC money ran out. Instead, Uber quietly settled into steady operation, prices went up a little bit, and people still use Uber a lot. Uber did this without the constant hardware and model improvements that LLM companies benefit from.
> but as far as I can see all the AI companies are not making money on inference alone.<p>This was the missed point on why GPT5 was such an important launch (quality of models and vibes aside). It brought the model sizes (and hence inference cost) to more sustainable numbers. Compared to previous SotA (GPT4 at launch, or o1/3 series), GPT5 is 8x-12x cheaper! I feel that a lot of people never re-calibrated their views on inference.<p>And there's also another place where you can verify your take on inference - the 3rd party providers that offer "open" models. They have 0 incentive to subsidise prices, because people that use them often don't even know who serves them, so there's 0 brand recognition (say when using models via openrouter).<p>These 3rd party providers have all converged towards a price-point per billion param models. And you can check those prices, and have an idea on what would be proffitable and at what sizes. Models like dsv3.2 are really really cheap to serve, for what they provide (at least gpt5-mini equivalent I'd say).<p>So yes, labs could totally become profitable with inference alone. But they don't want that, because there's an argument to be made that the best will "keep it all". I hope, for our sake as consumers that it isn't the case. And so far this year it seems that it's not the case. We've had all 4 big labs one-up eachother several times, and they're keeping eachother honest. And that's good for us. We get frontier level offerings at 10-25$/MTok (Opus, gpt5.2, gemini3pro, grok4), and we get highly capable yet extremely cheap models at 1.5-3$/MTok (gemini3-flash, gpt-minis, grok-fast, etc)
If the tech plateaus today, LLM plans will go to $60-80/mo, Chinese-hosted chinese models will be banned (national security will be the given reason), and the AI companies will be making ungodly money.<p>I'm not gonna dig out the math again, but if AI usage follows the popularity path of cell phone usage (which seems to be the case), then trillions invested has a ROI of 5-7 years. Not bad at all.
They're not making money on inference alone because they blow ungodly amounts on R&D. Otherwise it'd be a very profitable business.
Anthropic - for one - is making lots of money on inference.
This is one of the reasons why I'm surprised to see so many people jump on board. We're clearly in the "release product for free/cheap to gain customers" portion of the enshittification plan, before the company starts making it completely garbage to extract as much money as possible from the userbase<p>Having good quality dev tools is non negotiable, and I have a feeling that a lot of people are going to find out the hard way that reliability and it not being owned by profit seeking company is the #1 thing you want in your environment
One of the mental frameworks that convinced me is how much of a "free action" it is. Have the LLM (or the agent) churn on some problem and do something else. Come back and review the result. If you had to put significant effort into each query, I agree it wouldn't be worth it, but you can just type something into the textbox and wait.
OK, maybe. But how many programmers will know this in 10 years' time as use of LLMs is normalized? I like to hear what employers are saying already about recent graduates.
They’d have to be hiring recent graduates for you to hear that perspective.<p>And, as much as what I’ve just said is hyperbolically pessimistic, there is some truth to it.<p>In the UK a bunch of factors have coincided to put the brakes on hiring, especially smaller and mid-size businesses. AI is the obvious one that gets all the press (although how much it’s really to blame is open to question in my view), but the recent rise in employer AI contribution, and now (anecdotally) the employee rights bill have come together to make companies quite gunshy when it comes to hiring.
This is nothing new - entire industries and skills died out as the apprenticeship system and guilds were replaced by automation and factories
I'm uncertain that programming will be a major profession in 10 years.<p>Programming is more like math than creative writing. It's largely verifiable, which is where RL is repeatedly proven to eventually achieve significantly better than human intelligence.<p>Our saving grace, for now, is that it's not entirely verifiable because things like architectural taste are hard to put into a test. But I would not bet against it.
If they don't learn that they won't get very far.<p>This is true for everything, any tool you might use. Competent users of tools understand how they work and thus their limitations and how they're best put to work.<p>Incompetents just fumble around and sometimes get things working.
hahah what are you talking about, there's no such thing as long term!
I mean if you leaned heavily on stack overflow before AI then nothing really changes.<p>It’s basically the same idea but faster.
So, it's like taking off your pants to fart.
> There are certain tasks, like improving a given program for speed, for instance, where in theory the model can continue to make progress with a very clear reward signal for a very long time.<p>This makes me think: I wonder if Goodhart's law[1] may apply here. I wonder if, for instance, optimizing for speed may produce code that is faster but harder to understand and extend. Should we care or would it be ok for AI to produce code that passes all tests and is faster? Would the AI become good at creating explanations for humans as a side effect?<p>And if Goodhard's law doesn't apply, why is it? Is it because we're only doing RLVR fine-tuning on the last layers of the network so all the generality of the pre-training is not lost? And if this is the case, could this be a limitation in not being able to be creative enough to come up with move 37?<p>[1] <a href="https://wikipedia.org/wiki/Goodhart's_law" rel="nofollow">https://wikipedia.org/wiki/Goodhart's_law</a>
<i>I wonder if, for instance, optimizing for speed may produce code that is faster but harder to understand and extend.</i><p>This is generally true for code optimised by humans, at least for the sort of mechanical low level optimisations that LLMs are likely to be good at, as opposed to more conceptual optimisations like using better algorithms. So I suspect the same will be true for LLM-optimised code too.
> I wonder if, for instance, optimizing for speed may produce code that is faster but harder to understand and extend.<p>Superoptimizers have been around since 1987: <a href="https://en.wikipedia.org/wiki/Superoptimization" rel="nofollow">https://en.wikipedia.org/wiki/Superoptimization</a><p>They generate fast code that is not meant to be understood or extended.
But there output is (usually) executable code, and is not committed in a VCS. So the source code is still readable.<p>When people use LLMs to improve their code, they commit their output to Git to be used as source code.
...hmm, at some point we'll need to find a new place to draw the boundaries, won't we?<p>Until ~2022 there was a clear line between human-generated code and computer-generated code. The former was generally optimized for readability and the latter was optimized for speed at all cost.<p>Now we have computer-generated code in the human layer and it's not obvious what it should be optimized for.
Ehh I think if it ends up being a half good architecture you wind up with a difficult to understand kernel that never needs touching.
This is a bunch of "I believe" and "I think" with no sources by a random internet person.
Ah, I see you have discovered blogs! They're a cool form of writing from like ~20 years ago which are still pretty great. Good thing they show up on this website, it'd be rather dull with only newspapers and journal articles doncha think?
he’s not a “random internet person”, he created Redis. Despite that, I don’t know how authoritative of a figure he is with respect to AI research. He’s definitely a prolific programmer though.
To be fair, you may find equally capable random people in this thread, doesn't mean they speak with any kind of authority.
That still qualifies as a random internet person, wrt the topic. And I think the emphasis is on no sources and I beliefs and I thinks, in any case :)
There are plenty of Nobel laureates who well, do rest on their laurels and dive deep into pseudoscience after that.<p>Accomplishment in one field does not make one an expert, nor even particularly worth listening to, in any other. Certainly it doesn't remove the burden of proof or necessity to make an actual argument based on more then simply insisting something is true.
Yeah, it’s called “Reflections”.
Indeed, and, what do you 'believe' or 'think' in response?
It's the personal blog of a famous internet person
That is what a blog post is. Someone documenting what they think about a topic.<p>It's not the case that every form of writing has to be an academic research paper. Sometimes people just think things, and say them – and they may be wrong, or they may be right. And they sometime have some ideas that might change how you think about an issue as a result.
> by a random internet person.<p>The creator of Redis.
Sure but quite a few claims in the article are about AI research. He does not have any qualifications there. If the focus was more on usefulness, that would be a different discussion and then his experience does add weight.
> smart, intelligent person gives opinion<p>> woah buddy this persons opinion isn’t worth anything more than a random homeless person off the street. they’re not an expert in this field<p>Is there a term for this kind of pedantry? Obviously we can put more weight behind the words a person says if they’ve proven themselves trustworthy in prior areas - and we should! We want all people to speak and let the best idea win. If we fallback to only expert opinions are allowed that’s asking to get exploited. And it’s also important to know if antirez feels comfortable spouting nonsense.<p>This is like a basic cornerstone of a functioning society. Though, I realize this “no man is innately better than another, evaluate on merit” is mostly a western concept which might be some of my confusion.
Don't see how that gives him more credibility wrt AI.<p>His entirely unsupported statements about AGI are pretty useless, for instance.<p>So many people assume AGI is possible, yet no one has a concrete path to it or even a concrete definition of what it or what form it might take.
What is a "source"? Isn't it just "another random internet person"?
> For years, despite functional evidence and scientific hints accumulating, certain AI researchers continued to claim LLMs were stochastic parrots: probabilistic machines that would: 1. NOT have any representation about the meaning of the prompt. 2. NOT have any representation about what they were going to say. In 2025 finally almost everybody stopped saying so.<p>It's interesting that Terrence Tao just released his own blog post stating that they're best viewed as stochastic generators. True he's not an AI researcher, but it does sound like he's using AI frequently with some success.<p>"viewing the current generation of such tools primarily as a stochastic generator of sometimes clever - and often useful - thoughts and outputs may be a more productive perspective when trying to use them to solve difficult problems" [0].<p>[0] <a href="https://mathstodon.xyz/@tao/115722360006034040" rel="nofollow">https://mathstodon.xyz/@tao/115722360006034040</a>
What happened recently is that all the serious AI researches that were in the stochastic parrot side changed point of view but, incredibly, people without a deep understanding on such matters, previously exposed to such arguments, are lagging behind and still repeat arguments that the people who popularized them would not repeat again.<p>Today there is no top AI scientist that will tell you LLMs are just stochastic parrots.
The stochastic parrot framing makes some assumptions, one of them being that LLMs generate from minimal input prompts, like "tell me about Transformers" or "draw a cute dog". But when input provides substantial entropy or novelty, the output will not look like any training data. And longer sessions with multiple rounds of messages also deviate OOD. The model is doing work outside its training distribution.
Now that you’re here, what do you mean by “scientific hints” in your first paragraph?
> And I've vibe coded entire ephemeral apps just to find a single bug because why not - code is suddenly free, ephemeral, malleable, discardable after single use. Vibe coding will terraform software and alter job descriptions.<p>I'm not super up-to-date on all that's happening in AI-land, but in this quote I can find something that most techno-enthusiast seem to have decided to ignore: no, code is <i>not</i> free. There are immense resources (energy, water, materials) that go into these data centers in order to produce this "free" code. And the material consequences are terribly damaging to thousands of people. With the further construction of data centers to feed this free video coding style, we're further destroying parts of the world. Well done, AGI loverboys.
My guess is that “free” is meant in terms of the old definition where you’re not having to pay someone to create and maintain it. But yes, it’s important to realize there really is a cost here and one that can’t just be captured by a dollar amount.
Can you provide numbers relative to things many of us already do?<p>- drive to the store or to work<p>- take a shower<p>- eat meat<p>- fly on vacation<p>And so on... thanks!
You know what uses roughly 80
times more water in the US alone than water used by AI data centers world wide? Corn.
Assuming your fact is true, that corn merely uses an order of magnitude or two more water than AI is surprising, given the utility of corn. It feeds the entire US (hundreds of millions of people), is used as animal feed (thus also feeding us), and is widely exported to feed other people. I the spirit of the “I think”s and “I believe”s of this blog post, I think that corn has a lot more utility than AI.
> It feeds the entire US (hundreds of millions of people), is used as animal feed (thus also feeding us), and is widely exported to feed other people.<p>Not really. Most corn grown in the US isn’t even fit for consumption. It is primarily used for fermenting bioethanol.
> There are certain tasks, like improving a given program for speed, for instance, where in theory the model can continue to make progress with a very clear reward signal for a very long time.<p>Super skeptical of this claim. Yes, if I have some toy poorly optimized python example or maybe a sorting algorithm in ASM, but this won’t work in any non-trivial case. My intuition is that the LLM will spin its wheels at a local minimum the performance of which is overdetermined by millions of black-box optimizations in the interpreter or compiler signal from which is not fed back to the LLM.
There was a discussion the other day where someone asked Claude to improve a code base 200x <a href="https://news.ycombinator.com/item?id=46197930">https://news.ycombinator.com/item?id=46197930</a>
<a href="https://github.com/algorithmicsuperintelligence/openevolve" rel="nofollow">https://github.com/algorithmicsuperintelligence/openevolve</a>
> * Programmers resistance to AI assisted programming has lowered considerably. Even if LLMs make mistakes, the ability of LLMs to deliver useful code and hints improved to the point most skeptics started to use LLMs anyway: now the return on the investment is acceptable for many more folks.<p>Could not agree more. I myself started 2025 being very skeptical, and finished it very convinced about the usefulness of LLMs for programming. I have also seen multiple colleagues and friends go through the same change of appreciation.<p>I noticed that for certain task, our productivity can be multiplied by 2 to 4. So hence comes my doubts: are we going to be too many developers / software engineers ? What will happen for the rests of us ?<p>I assume that other fields (other than software-related) should also benefits from the same productivity boosts. I wonder if our society is ready to accept that people should work less. I think the more likely continuation is that companies will either hire less, or fire more, instead of accepting to pay the same for less hours of human-work.
> Are we going to be too many developers / software engineers ? What will happen for the rests of us?<p>I propose that we should raise the bar for the quality of software now.
I like to think of it as adding new lanes to a highway. More will be delivered until it all jams up again.
> The fundamental challenge in AI for the next 20 years is avoiding extinction.<p>That's a weird thing to end on. Surely it's worth more than one sentence if you're serious about it? As it stands, it feels a bit like the fearmongering Big Tech CEOs use to drive up the AI stocks.<p>If AI is really that powerful and I should care about it, I'd rather hear about it without the scare tactics.
I think <a href="https://en.wikipedia.org/wiki/Existential_risk_from_artificial_intelligence#History" rel="nofollow">https://en.wikipedia.org/wiki/Existential_risk_from_artifici...</a> has much better arguments than the LessWrong sources in other comments, and they weren't written by Big Tech CEOs.<p>Also "my product will kill you and everyone you care about" is not as great a marketing strategy as you seem to imply, and Big Tech CEOs are not talking about risks anymore. They currently say things like "we'll all be so rich that we won't need to work and we will have to find meaning without jobs"
What makes it a scare tactic? There are other areas in which extinction is a serious concern and people don't behave as though it's all that scary or important. It's just a banal fact. And for all of the extinction threats, AI included, it's very easy to find plenty of deep dive commentary if you care.
I would say yes, everyone should care about it.<p>There is plenty of material on the topic. See for example <a href="https://ai-2027.com/" rel="nofollow">https://ai-2027.com/</a> or <a href="https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities" rel="nofollow">https://www.lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a...</a>
Yeah, well known marketing trick that Big Companies do.<p>Oil companies: we are causing global warming with all this carbon emissions, are you scared yet? so buy our stock<p>Pharma companies: our drugs are unsafe, full of side effects, and kill a lot of people, are you scared yet? so buy our stock<p>Software companies: our software is full of bugs, will corrupt your files and make you lose money, are you scared yet? so buy our stock<p>Classic marketing tactics, very effective.
This has been well discussed before, for example in this book:
<a href="https://ifanyonebuildsit.com/" rel="nofollow">https://ifanyonebuildsit.com/</a>
Where to understand more about how chain of thoughs really affects LLMs performance? I read the seminal paper but all it says is that it's basically another prompt engineering tecnique that improves accuracy.
Chain of thought, now including "reasoning", are basically a work around for the simplistic nature of the Transformer neural network architecture that all LLMs are based on.<p>The two main limitations of the Transformer that it helps with are:<p>1) A Transformer is just a fixed-size stack of layers, with a one-way flow of data through the layers from input to output. The fixed number of layers equates to how many "thought" steps the LLM can put into generating each word of output, but good responses to harder questions may require many more steps and iterative thinking...<p>The idea of "think step by step", aka chain of thought, is to have the model break it's response down into a sequence of steps, each building on what came before, so that the scope of each step is withing the capability of the fixed number of layers of the transformer.<p>2) A Transformer has extremely limited internal memory from one generated word to the next, so telling the model to go one step at a time, feeding its own output back in as input, in effect makes the model's output a kind of memory that makes up for this.<p>So, chain of thought prompting ultimately give the model more thinking steps (more words generated), together with memory of what it is thinking, in order to be able to generate a better response.
It’s interesting that half the comments here are talking about the extinction line when, now that we’re nearly entering 2026, I feel the 2027 predictions have been shown to be pretty wrong so far.
What also happens and it's irrelevant of AGI: global RL<p>Around the world people ask an LLM and get a response.<p>Just grouping and analysing these questions and solving them once centrally and then making the solution available again is huge.<p>Linearly solving the most asked questions and then the next one then the next will make, whatever system is behind it, smarter every day.
There's videos about Diffusion LLMs too, apparently getting rid of the linear token generation. But I'm no ML engineer.
As someone who worked on transformer-based diffusion models before (not for language though), i can say one thing: they're hard.<p>Denoising diffusion models benefited a lot from the u-net, which is a pretty simple network (compared to a transformer) and very well-adapted to the denoising task. Plus diffusion on images is great to research because it's very easy to visualize, and therefore to wrap your head around<p>Doing diffusion on text is a great idea, but my intuition is it will prove more challenging, and probably take a while before we get something working
> <i>1. NOT have any representation about the meaning of the prompt.</i><p>This one is bizarre, if true (I'm not convinced it is).<p>The entire purpose of the attention mechanism in the transformer architecture is to build this representation, in many layers (conceptually: in many layers <i>of abstraction</i>).<p>> <i>2. NOT have any representation about what they were going to say.</i><p>The only place for this to go is in the model weights. More parameters means "more places to remember things", so clearly that's <i>at least</i> a representation.<p>Again: who was pushing this belief? Presumably not researchers, these are fundamental properties of the transformer architecture. To the best of my knowledge, they are not disputed.<p>> <i>I believe [...] it is not impossible they get us to AGI even without fundamentally new paradigms appearing.</i><p>Same, at least for the OpenAI AGI definition: "An AI system that is at least as intelligent as a normal human, and is able to do any economically valuable work."
> This one is bizarre, if true (I'm not convinced it is).<p>> The entire purpose of the attention mechanism in the transformer architecture is to build this representation, in many layers (conceptually: in many layers of abstraction).<p>I think this is really about a hidden (i.e. not readily communicated) difference in what the word "meaning" means to different people.
> The fundamental challenge in AI for the next 20 years is avoiding extinction.<p>So nice to see people who think about this seriously converge on this. Yes. Creating something smarter than you was always going to be a sketchy prospect.<p>All of the folks insisting it just couldn't happen or ... well, there have just been so many objections. The goalposts have walked from one side of the field to the other, and then left the stadium, went on a trip to Europe, got lost in a beautiful little village in Norway, and decided to move there.<p>All this time though, the prospect of instantiating a something smarter than you (and yes, it will be smarter than you even if it's at human level because of electronic speeds...) This whole idea is just cursed and we should not do the thing.
>* For years, despite functional evidence and scientific hints accumulating, certain AI researchers continued to claim LLMs were stochastic parrots: probabilistic machines that would: 1. NOT have any representation about the meaning of the prompt. 2. NOT have any representation about what they were going to say. In 2025 finally almost everybody stopped saying so.<p>Man, Antirez and I walk in very different circles! I still feel like LLMs fall over backwards once you give them an 'unusual' or 'rare' task that isn't likely to be presented in the training data.
LLMs certainly struggle with tasks that require knowledge that is not provided to them (at significant enough volume/variance to retain it). But this is to be expected of any intelligent agent, it is certainly true of humans. It is not a good argument to support the claim that they are Chinese Rooms (unthinking imitators). Indeed, the whole point of the Chinese Room thought experiment was to consider if that distinction even mattered.<p>When it comes to of being able to do novel tasks on known knowledge, they seem to be quite good. One also needs to consider that problem-solving patterns are also a kind of (meta-)knowledge that needs to be taught, either through imitation/memorisation (Supervised Learning) or through practice (Reinforcement Learning). They can be logically derived from other techniques to an extent, just like new knowledge can be derived from known knowledge in general, and again LLMs seem to be pretty decent at this, but only to a point. Regardless, all of this is definitely true of humans too.
"In 2025 finally almost everybody stopped saying so."<p>I haven't.
I don’t think this is quite true.<p>I’ve seen them do fine on tasks that are clearly not in the training data, and it seems to me that they struggle when some particular type of task or solution or approach might be something they haven’t been exposed to, rather than the exact task.<p>In the context of the paragraph you quoted, that’s an important distinction.<p>It seems quite clear to me that they are getting at the meaning of the prompt and are able, at least somewhat, to generalise and connect aspects of their training to “plan” and output a meaningful response.<p>This certainly doesn’t seem all that deep (at times frustratingly shallow) and I can see how at first glance it might look like everything was just regurgitated training data, but my repeated experience (especially over the last ~6-9 months) is that there’s something more than that happening, which feels like whet Antirez was getting at.
Give me an example of one of those rare or unusual tasks.
They are very advanced stochastic parrots that allow AI invested authors to suddenly write in perfect English.<p>If Antirez has never gotten an LLM to perform an absolutely embarrassing mistake, he must be very lucky or we should stop listening to him.<p>Programmers' resistance has not weakened. Since the ORCL drop of 40% anti-LLM opinions are censored and downvoted here. Many people have given up, and we always get articles from the same LLM influencers.
I'm impressed that such a short post can be so categorically incorrect.<p>> For years, despite functional evidence and scientific hints accumulating, certain AI researchers continued to claim LLMs were stochastic parrots<p>> In 2025 finally almost everybody stopped saying so.<p>There is still no evidence that LLMs are anything beyond "stochastic parrots". There is no proof of any "understanding". This is seeing faces in clouds.<p>> I believe improvements to RL applied to LLMs will be the next big thing in AI.<p>With what proof or evidence? Gut feeling?<p>> Programmers resistance to AI assisted programming has lowered considerably.<p>Evidence is the opposite, most developers do not trust it. <a href="https://survey.stackoverflow.co/2025/ai#2-accuracy-of-ai-tools" rel="nofollow">https://survey.stackoverflow.co/2025/ai#2-accuracy-of-ai-too...</a><p>> It is likely that AGI can be reached independently with many radically different architectures.<p>There continues to be no evidence beyond "hope" that AGI is even possible, yet alone that Transformer models are the path there.<p>> The fundamental challenge in AI for the next 20 years is avoiding extinction.<p>Again, nothing more than a gut feeling. Much like all the other AI hype posts this is nothing more than "well LLMs sure are impressive, people say they're not, but I think they're wrong and we will make a machine god any day now".
Not sure I understand the last sentence:<p>> The fundamental challenge in AI for the next 20 years is avoiding extinction.
I think he's referring to AI safety.<p><a href="https://lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-list-of-lethalities" rel="nofollow">https://lesswrong.com/posts/uMQ3cqWDPHhjtiesc/agi-ruin-a-lis...</a>
He's referring to humanity, I believe
> * The fundamental challenge in AI for the next 20 years is avoiding extinction.<p>This reminded me of the Don’t look up movie where they basically gambled with the humans extinction.
> Even if LLMs make mistakes, the ability of LLMs to deliver useful code and hints improved to the point most skeptics started to use LLMs anyway<p>Here we go again. Statements with the single source in the head of the speaker. And it’s also not true. The llms still produce bad/irrelevant code at such rate that you can spend more time prompting than doing things yourself.<p>I’m tired of this overestimation of llms.
Even where they are not directly using LLMs to write the most critical or core code, nearly every skeptic I know has started using LLMs at very least to do things like write tests, build tools, write glue code, help to debug or refactor, etc.<p>Your statement suffers not only from also coming only from your brain, with no evidence that you've actually tried to learn to use these tools, but it also goes against the weight of evidence that I see both in my professional network and online.
I just want people making statements like the author to be more specific how exactly the llms are being used. Otherwise they contribute to this belief that llms are a magical tool that can do anything.<p>I am aware of simple routine tasks that LLMs can do. This doesn’t change anything about what I said.
All you had to do is scroll down further and read the next couple of posts where the author is being more specific on how they used LLMs.<p>I swear, the so called critics need everything spoon fed.
Sorry, but we're way past that. It's you who need to provide examples of tasks it can't do.
You need to meet more skeptics. (Or maybe I do.) In my world, it's much more rare than you say.
> Here we go again. Statements with the single source in the head of the speaker. And it’s also not true.<p>You're making the same sort of baseless claim you are criticising the blogger for making. Spewing baseless claims hardly moves any discussion forward.<p>> The llms still produce bad/irrelevant code at such rate that you can spend more time promoting than doing things yourself.<p>If that is your personal experience then I regret to tell you that it is only the reflection of your own inability to work with LLMs and coding agents. Meanwhile, I personally manage to effectively use LLMs anywhere between small refactoring needs and large software architecture designs, including generating fully working MVPs in one-shot agent prompts. From this alone it's rather obvious who is making baseless statements that are more aligned with reality.
But you have just repeated what you are complaining about.
My person experience: if I can find a solution on stackoverflow etc. the LLM will produce working and fundamentally correct code. If I can‘t find a already fullfilled solution on these sites, the LLM is hallucinating like crazy (newer existing functions/modules/plugins, protocol features which aren’t specified and even github-repos which never existed). So, as stated my many people online before: for low-hanging fruits LLM are totally viable solution.
I don't remember the last time Claude Code hallucinated some library, as it will check the packages, verify with the linter, run a test import and so on.<p>Are you talking about punching something into some LLM web chat that's disconnected from your actual codebase and has tooling like web search disabled? If so, that's not really the state of the art of AI assisted coding, just so you know.
[flagged]
Seems they also want some AI money[0]. Guess, I'll keep using Valkey.<p>[0] <a href="https://redis.io/redis-for-ai/" rel="nofollow">https://redis.io/redis-for-ai/</a>
> they<p>I'm not sure antirez is involved in any business decision making process at Redis Ltd.<p>He may not be part of "they".
In any case, what would be the problem? The page you mentioned simply illustrates how the product can be used in a specific domain; it doesn't seem forced to me.
Conflict of interest and disclosure posts are frequently downvoted.