For a fair comparison you need to look at the total cost, because 4.7 produces significantly fewer output tokens than 4.6, and seems to cost significantly less on the reasoning side as well.<p>Here is a comparison for 4.5, 4.6 and 4.7 (Output Tokens section):<p><a href="https://artificialanalysis.ai/?models=claude-opus-4-7%2Cclaude-opus-4-6-adaptive%2Cclaude-opus-4-5-thinking" rel="nofollow">https://artificialanalysis.ai/?models=claude-opus-4-7%2Cclau...</a><p>4.7 comes out slightly <i>cheaper</i> than 4.6. But 4.5 is about half the cost:<p><a href="https://artificialanalysis.ai/?models=claude-opus-4-7%2Cclaude-opus-4-6-adaptive%2Cclaude-opus-4-5-thinking#cost" rel="nofollow">https://artificialanalysis.ai/?models=claude-opus-4-7%2Cclau...</a><p>Notably the cost of <i>reasoning</i> has been cut almost in half from 4.6 to 4.7.<p>I'm not sure what that looks like for most people's workloads, i.e. what the cost breakdown looks like for Claude Code. I expect it's heavy on both input and reasoning, so I don't know how that balances out, now that input is more expensive and reasoning is cheaper.<p>On reasoning-heavy tasks, it might be cheaper. On tasks which don't require much reasoning, it's probably more expensive. (But for those, I would use Codex anyway ;)
It thinks less and produces less output tokens because it has forced adaptive thinking that even API users can't disable. Same adaptive thinking that was causing quality issues in Opus 4.6 not even two weeks ago. The one bcherny recommended that people disable because it'd sometimes allocate zero thinking tokens to the model.<p><a href="https://news.ycombinator.com/item?id=47668520">https://news.ycombinator.com/item?id=47668520</a><p>People are already complaining about low quality results with Opus 4.7. I'm also spotting it making really basic mistakes.<p>I literally just caught it lazily "hand-waving" away things instead of properly thinking them through, even though it spent like 10 minutes churning tokens and ate only god knows how many percentage points off my limits.<p>> What's the difference between this and option 1.(a) presented before?<p>> Honestly? Barely any. Option M is option 1.(a) with the lifecycle actually worked out instead of hand-waved.<p>> Why are you handwaving things away though? I've got you on max effort. I even patched the system prompts to reduce this.<p>> Fair call. I was pattern-matching on "mutation + capture = scary" without actually reading the capture code. Let me do the work properly.<p>Can provide session feedback IDs if needed.
Some have defined "fair" as tests of the same model at different times, as the behavior and token usage of a model changes despite the version number remaining the same. So testing model numbers at different times matters, unfortunately, and that means recent tests might not be what you would want to compare to future tests.
The bump from 4.6 to 4.7 is not very noticeable to me in improved capabilities so far, but the faster consumption of limits is very noticeable.<p>I hit my 5 hour limit within 2 hours yesterday, initially I was trying the batched mode for a refactor but cancelled after seeing it take 30% of the limit within 5 minutes. Had to cancel and try a serial approach, consumed less (took ~50 minutes, xhigh effort, ~60% of the remaining allocation IIRC), but still very clearly consumed much faster than with 4.6.<p>It feels like every exchange takes ~5% of the 5 hour limit now, when it used to be maybe ~1-2%. For reference I'm on the Max 5x plan.<p>For now I can tolerate it since I still have plenty of headroom in my limits (used ~5% of my weekly, I don't use claude heavily every day so this is OK), but I hope they either offer more clarity on this or improve the situation. The effort setting is still a bit too opaque to really help.
I'd be ok with paying more if results were good, but it seems like Anthropic is going for the Tinder/casino intermittent reinforcement strategy: optimized to keep you spending tokens instead of achieving results.<p>And yes, Claude models are generally more fun to use than GPT/Codex. They have a personality. They have an intuition for design/aesthetics. Vibe-coding with them feels like playing a video game. But the result is almost always some version of cutting corners: tests removed to make the suite pass, duplicate code everywhere, wrong abstraction, type safety disabled, hard requirements ignored, etc.<p>These issues are not resolved in 4.7, no matter what the benchmarks say, and I don't think there is any interest in resolving them.
Mirrors my sentiment. Those tools seem mostly useful for a Google alternative, scaffolding tedious things, code reviewing, and acting as a fancy search.<p>It seems that they got a grip on the "coding LLM" market and now they're starting to seek actual profit. I predict we'll keep seeing 40%+ more expensive models for a marginal performance gain from now on.
You have to guide an ai. Not let roam freely. If you got skills to guide you can make it output high quality
> ... but it seems like Anthropic is going for the Tinder/casino intermittent reinforcement strategy: optimized to keep you spending tokens instead of achieving results.<p>This part of the above comment strikes me as uncharitable and overconfident. And, to be blunt, presumptuous. To claim to know a company's strategy as an outsider is messy stuff.<p>My prior: it is 10X to 20X more likely Anthropic has done something other than shift to a <i>short-term squeeze their customers</i> strategy (which I think is only around ~5%)<p>What do I mean by "something other"? (1) One possibility is they are having capacity and/or infrastructure problems so the model performance is degraded. (2) Another possibility is that they are not as tuned to to what customers want relative to what their engineers want. (3) It is also possible they have slowed down their models down due to safety concerns. To be more specific, they are erring on the side of caution (which would be consistent with their press releases about safety concerns of Mythos). Also, the above three possibilities are not mutually exclusive.<p>I don't expect us (readers here) to agree on the probabilities down to the ±5% level, but I would think a large chunk of informed and reasonable people can probably converge to something close to ±20%. At the very least, can we agree all of these factors are strong contenders: each covers maybe at least 10% to 30% of the probability space?<p>How short-sighted, dumb, or back-against-the-wall would Anthropic have to be to shift to a "let's make our new models intentionally _worse_ than our previous ones?" strategy? Think on this. I'm not necessarily "pro" Anthropic. They could lose standing with me over time, for sure. I'm willing to think it through. What would the world have to look like for this to be the case.<p>There are other factors that push back against claims of a "short-term greedy strategy" argument. Most importantly, they aren't stupid; they know customers care about quality. They are playing a longer game than that.<p>Yes, I understand that Opus 4.7 is not impressing people or worse. I feel similarly based on my "feels", but I also know I haven't run benchmarks nor have I used it very long.<p>I think most people viewed Opus 4.6 as a big step forward. People are somewhat conditioned to expect a newer model to be better, and Opus 4.7 doesn't match that expectation. I also know that I've been asking Claude to help me with Bayesian probabilistic modeling techniques that are well outside what I was doing a few weeks ago (detailed research and systems / software development), so it is just as likely that I'm pushing it outside its expertise.
> To claim to know a company's strategy as an outsider is messy stuff.<p>I said "it seems like". Obviously, I have no idea whether this is an intentional strategy or not and it could as well be a side effect of those things that you mentioned.<p>Models being "worse" is the perceived effect for the end user (subjectively, <i>it seems like</i> the price to achieve the same results on similar tasks with Opus has been steadily increasing). I am claiming that there is no incentive for Anthropic to address this issue because of their business model (maximize the amount of tokens spent and price per token).
AFAICT this uses a token-counting API so that it counts how many tokens are in the prompt, in two ways, so it's measuring the tokenizer change in isolation. Smarter models also sometimes produce shorter outputs and therefore fewer output tokens. That doesn't mean Opus 4.7 necessarily nets out cheaper, it might still be more expensive, but this comparison isn't really very useful.
For some real data, Artificial Analysis reported that 4.6 (max) and 4.7 (max) used 160M tokens and 100M tokens to complete their benchmark suite, respectively:<p><a href="https://artificialanalysis.ai/?intelligence-efficiency=intelligence-efficiency-output-token-breakdown#output-tokens-used-to-run-artificial-analysis-intelligence-index" rel="nofollow">https://artificialanalysis.ai/?intelligence-efficiency=intel...</a><p>Looking at their cost breakdown, while input cost rose by $800, output cost dropped by $1400. Granted whether output offsets input will be very use-case dependent, and I imagine the delta is a lot closer at lower effort levels.
Why is it not useful? Input token pricing is the same for 4.7. The same prompt costs roughly 30% more now, for input.
The idea is that smarter models might use fewer turns to accomplish the same task - reducing the overall token usage<p>Though, from my limited testing, the new model is far more token hungry overall
That's valid, but it's also worth knowing it's only one part of the puzzle. The submission title doesn't say "input".
Yes. I actually noticed my token usage go down on 4.6 when I started switching every session to max effort. I got work done faster with fewer steps because thinking corrected itself before it cycled.<p>I’ve noticed 4.7 cycling a lot more on basic tasks. Though, it also seems a bit better at holding long running context.
With AIs, it seems like there never is a comparison that is useful.
For now, I'm planning to stick with Opus 4.5 as a driver in VSCode Copilot.<p>My workflow is to give the agent pretty fine-grained instructions, and I'm always fighting agents that insist on doing too much. Opus 4.5 is the best out of all agents I've tried at following the guidance to do only-what-is-needed-and-no-more.<p>Opus 4.6 takes longer, overthinks things and changes too much; the high-powered GPTs are similarly flawed. Other models such as Sonnet aren't nearly as good at discerning my intentions from less-than-perfectly-crafted prompts as Opus.<p>Eventually, I quit experimenting and just started using Opus 4.5 exclusively knowing this would all be different in a few months anyway. Opus cost more, but the value was there.<p>But now I see that 4.7 is going to replace both 4.5 and 4.6 in VSCode Copilot, and with a 7.5x modifier. Based on the description, this is going to be a price hike for slower performance — and if the 4.5 to 4.6 change is any guide, more overthinking targeted at long-running tasks, rather than fine-grained. For me, that seems like a step backwards.
It's increasingly looking naive to assume scaling LLMs is all you need to get to full white-collar worker replacement. The attention mechanism / hopfield network is fundamentally modeling only a small subset of the full human brain, and all the increasing sustained hype around bolted-on solutions for "agentic memory" is, in my opinion, glaring evidence that these SOTA transformers alone aren't sufficient even when you just limit the space to text. Maybe I'm just parroting Yann LeCun.
> you just limit the space to text<p>And even then... why can't they write a novel? Or lowering the bar, let's say a novella like <i>Death in Venice</i>, <i>Candide</i>, <i>The Metamorphosis</i>, <i>Breakfast at Tiffany's</i>...?<p>Every book's in the training corpus...<p>Is it just a matter of someone not having spent a hundred grand in tokens to do it?
I know someone spending basically every day writing personal fan fiction stories using every model you can find. She doesn't want to share it, and does complain about it a lot, seems like maintaining consistency for something say 100 pages long is difficult
Never mind novels, it can't even write a good Reddit-style or HN-style comment. agentalcove.ai has an archive of AI models chatting to one another in "forum" style and even though it's a good show of the models' overall knowledge the AIisms are quite glaring.
They definitely can, and do.<p>It's just that the ones that manage to suppress all the AI writing "tells" go unnoticed as AI. This is a type of survivorship bias, though I feel there must be a better term for it that eludes me.
I don’t understand - there are hundreds/thousands of AI written books available now.
Who says they can't? What's your bar that needs to be passed in order for "written a novella" to be achieved?<p>There's a lot of bad writing out there, I can't imagine nobody has used an LLM to write a bad novella.
You probably are.<p>The "small subset" argument is profoundly unconvincing, and inconsistent with both neurobiology of the human brain and the actual performance of LLMs.<p>The transformer architecture is incredibly universal and highly expressive. Transformers power LLMs, video generator models, audio generator models, SLAM models, entire VLAs and more. It not a 1:1 copy of human brain, but that doesn't mean that it's incapable of reaching functional equivalence. Human brain isn't the only way to implement general intelligence - just the one that was the easiest for evolution to put together out of what it had.<p>LeCun's arguments about "LLMs can't do X" keep being proven wrong empirically. Even on ARC-AGI-3, which is a benchmark specifically designed to be adversarial to LLMs and target the weakest capabilities of off the shelf LLMs, there is no AI class that beats LLMs.
Should the title here be 4.6 to 4.7 instead of the other way around?
I was using Opus 4.7 just yesterday to help implement best practices on a single page website.<p>After just ~4 prompts I blew past my daily limit. Another ~7 more prompts & I blew past my weekly limit.<p>The entire HTMl/CSS/JS was less than 300 lines of code.<p>I was shocked how fast it exhausted my usage limits.
I haven't used Claude. Because I suspect this sort of things to come.<p>With enterprise subscription, the bill gets bigger but it's not like VP can easily send a memo to all its staff that a migration is coming.<p>Individuals may end their subscription, that would appease the DC usage, and turn profits up.
What's your reasoning effort set to? Max now uses way more tokens and isn't suggested for most usecases. Even the new default (xhigh) uses more than the old default (medium).
Which plan are you on? I could see that happening with Pro (which I think defaults to Sonnet?), would be surprised with Max…
It eats even the Max plan like crazy.
Pro. It even gave me $20 free credits, and exhausted free credits nearly instantly.
Are you using Claude subscription? Because that's not how it works there.
My impression is that the quality of the conversation is unexpectedly better: more self-critical, the suggestions are always critical, the default choices constantly best. I might not have as many harnesses as most people here, so I suspect it’s less obvious but I would expect this to make it far more valuable for people who haven’t invested as much.<p>After a few basic operations (retrospective look at the flow of recent reviews, product discussions) I would expect this to act like a senior member of the team, while 4.6 was good, but far more likely to be a foot-gun.
> Opus 4.7 (Adaptive Reasoning, Max Effort) cost ~$4,406 to run the Artificial Analysis Intelligence Index, ~11% less than Opus 4.6 (Adaptive Reasoning, Max Effort, ~$4,970) despite scoring 4 points higher. This is driven by lower output token usage, even after accounting for Opus 4.7's new tokenizer. This metric does not account for cached input token discounts, which we will be incorporating into our cost calculations in the near future.
We dropped Claude. It's pretty clear this is a race to the bottom, and we don't want a hard dependency on another multi-billion dollar company just to write software<p>We'll be keeping an eye on open models (of which we already make good use of). I think that's the way forward. Actually it would be great if everybody would put more focus on open models, perhaps we can come up with something like the "linux/postgres/git/http/etc" of the LLMs: something we all can benefit from while it not being monopolized by a single billionarie company. Wouldn't it be nice if we don't need to pay for tokens? Paying for infra (servers, electricity) is already expensive enough
>we don't want a hard dependency on another multi-billion dollar company just to write software<p>One of two main reasons why I'm wary of LLMs. The other is fear of skill atrophy. These two problems compound. Skill atrophy is less bad if the replacement for the previous skill does not depend on a potentially less-than-friendly party.
I was worried about skill atrophy. I recently started a new job, and from day 1 I've been using Claude. 90+% of the code I've written has been with Claude. One of the earlier tickets I was given was to update the documentation for one of our pipelines. I used Claude entirely, starting with having it generate a very long and thorough document, then opening up new contexts and getting it to fact check until it stopped finding issues, and then having it cut out anything that was granular/one query away. And <i>then</i> I read what it had produced.<p>It was an experiment to see if I could enter a mature codebase I had zero knowledge of, look at it entirely through an AI, and come to understand it.<p>And it worked! Even though I've only worked on the codebase through Claude, whenever I pick up a ticket nowadays I know what file I'll be editing and how it relates to the rest of the code. If anything, I have a significantly better understanding of the codebase than I would without AI at this point in my onboarding.
Yeah, +1. I will never be working on unsolved problems anyhow. Skill atrophy is not happening if you stay curious and responsible.
I have never learned so quickly in my entire life than to post a forum thread in its entirety into a extended think LLM and then be allowed to ask free form questions for 2 hours straight if I want to. Having my questions answered NOW is so important for me to learn. Back in the day by the time I found the answer online I forgot the question
Same. I work in the film industry, but I’ve always been interested in computers and have enjoyed tinkering with them since I was about 5. However, coding has always been this insurmountably complicated thing- every time I make an effort to learn, I’m confronted with concepts that are difficult for me to understand and process.<p>I’ve been 90% vibe coding for a year or so now, and I’ve learned <i>so much</i> about networking just from spinning up a bunch of docker containers and helping GPT or Claude fix niggling issues.<p>I essentially have an expert (well, maybe not an expert but an entity far more capable than I am on my own) who’s shoulder I can look over and ask as many questions I want to, and who will explain every step of the process to me if I want.<p>I’m finally able to create things on my computer that I’ve been dreaming about for years.
Some people talk like skill atrophy is inevitable when you use LLMs, which strikes me as pretty absurd given that you are talking about a tool that will answer an infinite number of questions with infinite patience.<p>I usually learn way more by having Claude do a task and then quizzing it about what it did than by figuring out how to do it myself. When I have to figure out how to do the thing, it takes much more time, so when I'm done I have to move on immediately. When Claude does the task in ten minutes I now have several hours I can dedicate entirely to understanding.
You lose some, you win some. The win could be short-term much higher, however imagine that the new tool suddenly gets ragged pulled from under your feet. What do you do then? Do you still know how to handle it the old way or do you run into skill atrophy issues? I’m using Claude/Codex as well, but I’m a little worried that the environment we work in will become a lot more bumpy and shifty.
The "infinite patience" thing I find particularly interesting.<p>Every now and then I pause before I ask an LLM to undo something it just did or answer something I know it answered already, somewhere. And then I remember oh yeah, it's an LLM, it's not going to get upset.
Asking infinite questions about something does not make you good at “doing” that thing, you get pretty good at asking questions
Understanding is not learning. Zero effort gives zero rewards, I ask Claude plenty of things, I get answers but not learnings.
I used to speak Russian like I was born in Russia. I stopped talking Russian … every day I am curious ans responsible but I can hardly say 10 words in Russian today. if you don’t use it (not just be curious and responsible) you will lose it - period.
Programming language is not just syntax, keywords and standard libraries, but also: processes, best practices and design principles. The latter group I guess is more difficult to learn and harder to forget.
I respectfully completely disagree. not only will you just as easily lose thr processed, best practices and design principles but they will be changing over time (what was best practice when I got my first gig in 1997 is <i>not</i> a best practice today (even just 4-5 years ago not to go all the back to the 90’s)). all that is super easy to both forget and lose unless you live it daily
I have also found LLMs are a great tool for understanding a new code base, but it's not clear to me what your comment has to do with skill atrophy.
What do you mean “cut out anything that was granular/one query away”? This was a very cool workflow to hear about—I will be applying it myself
Are you sure you would know if it didn't work? I use Claude extensively myself, so I'm not saying this from a "hater" angle, but I had 2 people last week who believe themselves to be in your shoes send me pull requests which made absolutely no sense in the context of the codebase.
It's good that it's working for you but I'm not sure what this has to do with skill atrophy. It sounds like you never had this skill (in this case, working with that particular system) to begin with.<p>>I have a significantly better understanding of the codebase than I would without AI at this point in my onboarding<p>One of the pitfalls of using AI to learn is the same as I'd see students doing pre-AI with tutoring services. They'd have tutors explain the homework to do them and even work through the problems with them. Thing is, any time you see a problem or concept solved, your brain is tricked into thinking you understand the topic enough to do it yourself. It's why people think their job interview questions are much easier than they really are; things just seem obvious when you've thought about the solution. Anyone who's read a tutorial, felt like they understood it well, and then struggled for a while to actually start using the tool to make something new knows the feeling very well. That Todo List app in the tutorial seemed so simple, but the author was making a bunch of decisions constantly that you didn't have to think about as you read it.<p>So I guess my question would be: If you were on a plane flight with no wifi, and you wanted to do some dev work locally on your laptop, how comfortable would you be vs if you had done all that work yourself rather than via Claude?
[dead]
Not so much atrophy as apathy.<p>I've worked with people who will look at code they don't understand, say "llm says this", and express zero intention of learning something. Might even push back. Be proud of their ignorance.<p>It's like, why even review that PR in the first place if you don't even know what you're working with?
I cringed when I saw a dev literally copy and paste an AI's response to a concern. The concern was one that had layers and implications to it, but instead of getting an answer as to why it was done a certain way and to allay any potential issues, that dev got a two paragraph lecture on how something worked on the surface of it, wrapped in em dashes and joviality.<p>A good dev would've read deeper into the concern and maybe noticed potential flaws, and if he had his own doubts about what the concern was about, would have asked for more clarification. Not just feed a concern into AI and fling it back. Like please, in this day and age of AI, have the benefit of the doubt that someone with a concern would have checked with AI himself if he had any doubts of his own concern...
Is this the same subset of people who copy/paste code directly from stack overflow without understanding ? I’m not sure this is a new problem.
It's a new problem in the sense that now executive management at many (if not most) software companies is pushing for all employees to work this way as much as possible. Those same people probably don't know what stack overflow even is.
In my experience, no - I think the ability to build more complete features with less/little/no effort, rather than isolated functions, is (more) appealing to (more) developers.
I don't think so. I'll spend a ton of time and effort thinking through, revising, and planning out the approach, but I let the agent take the wheel when it comes to transpiling that to code. I don't actually care about the code so long as it's secure and works.<p>I spent years cultivating expertise in C++ and .NET. And I found that time both valuable and enjoyable. But that's because it was a path to solve problems for my team, give guidance, and do so with both breadth and depth.<p>Now I focus on problems at a higher level of abstraction. I am certain there's still value in understanding ownership semantics and using reflection effectively, but they're broadly less relevant concerns.
It's difficult to copy & paste an entire app from Stack Overflow
Copied and pasted without noting the license that stack overflow has on code published there, no doubt
Hey. I resemble that remark sometimes!! quit being a hater (sarcasm) :P
We've had such developers around, long before LLMs.
It’s a lot like someone bragging that they’re bad at math tossing around equations.
If I wanted to know what the LLM says, I would have asked it myself, thanks…
What is it in the broader culture that's causing this?
People who got into the job who don’t really like programming
These people have always existed. Hell, they are here, too. Now they have a new thing to delegate responsibility to.<p>And no, I don't understand them at all. Taking responsibility for something, improving it, and stewarding it into production is a fantastic feeling, and much better than reading the comment section. :)
You can argu that you will have skill atrophy by not using LLMs.<p>We have gone multi cloud disaster recovery on our infrastructure. Something I would not have done yet, had we not had LLMs.<p>I am learning at an incredible rate with LLMs.
I kind feel the same. I’m learning things and doing things in areas that would just skip due to lack of time or fear.<p>But I’m so much more detached of the code, I don’t feel that ‘deep neural connection’ from actual spending days in locked in a refactor or debugging a really complex issue.<p>I don’t know how a feel about it.
I strongly agree on the refactor, but for debugging I have another perspective: I think debugging is changing for the better, so it looks different.<p>Sure, you don't know the code by heart, but people debugging code translated to assembly already do that.<p>The big difference is being able to unleash scripts that invalidate enormous amount of hypothesis very fast and that can analyze the data.<p>Used to do that by hand it took hours, so it would be a last resort approach. Now that's very cheap, so validating many hypothesis is way cheaper!<p>I feel like my "debugging ability" in terms of value delivered has gone way up. For skill, it's changing. I cannot tell, but the value i am delivering for debugging sessions has gone way up
As someone who's switched from mobile to web dev professionally for the last 6 months now. If you care about code quality, you'll develop that neural connection after some time.<p>But if you don't and there's no PR process (side projects), the motivation to form that connection is quite low.
> If you care about code quality, you'll develop that neural connection after some time.<p>No, because you can get LLMs to produce high quality code that has gone through an infinite number of refinement/polish cycles and is far more exhaustive than the code you would have written yourself.<p>Once you hit that point, you find yourself in a directional/steering position divorced from the code since no matter what direction you take, you'll get high quality code.
Yes, you certainly can argue that, but you'd be wrong. The primary selling point of LLMs is that they solve the problem of needing skill to get things done.
That is not the entire selling point - so you are very wrong.<p>You very much decide how you employ LLMs.<p>Nobody are keeping a gun to your head to use them. In a certain way.<p>Sonif you use them in a way that increase you inherent risk, then you are incredibly wrong.
I see it completely the opposite way, you use an LLM and correct all its mistakes and it allows you to deliver a rough solution very quickly and then refine it in combination with the AI but it still gets completely lost and stuck on basic things. It’s a very useful companion that you can’t trust, but it’s made me 4-5x more productive and certainly less frustrated by the legacy codebase I work on.
Yeah I whole hardheartedly disagree with this. Because I understand the basics of coding I can understand where the model gets stuck and prompt it in other directions.<p>If you don't know whats going on through the whole process, good luck with the end product.
They <i>purportedly</i> solve the problem of needing skill to get things done. IME, this is usually repeated by VC backed LLM companies or people who haven’t knowingly had to deal with other people’s bad results.<p>This all bumps up against the fact that most people default to “you use the tool wrong” and/or “you should only use it to do things where you already have firm grasp or at least foundational knowledge.”<p>It also bumps against the fact that the average person is using LLM’s as a replacement for standard google search.
You're learning at your standard rate of learning, you're just feeding yourself over-confidence on how much you're absorbing vs what the LLM is facilitating you rolling out.
This is such a weird statement in so many levels.<p>The latent assumption here is that learning is zero sum.<p>That you can take a 30 year old from 1856 bring them into present day and they will learn whatever subject as fast as a present day 20 year old.<p>That teachers doesn't matter.<p>That engagement doesn't matter.<p>Learning is not zero sum. Some cultural background makes learning easier, some mentoring makes is easier, and some techniques increases engagement in ways that increase learning speed.
> We have gone multi cloud disaster recovery on our infrastructure. Something I would not have done yet, had we not had LLMs.<p>That’s product atrophy, not skill atrophy.
> I am learning at an incredible rate with LLMs<p>Could you do it again without the help of an LLM?<p>If no, then can you really claim to have learned anything?
So, you havent really learned anything from any teacher if you could not do it again without them?
> So, you havent really learned anything from any teacher if you could not do it again without them?<p>Well, yes?<p>What do you think "learning" means? If you cannot do something without the teacher, you haven't learned that thing.
Yes.
I mean...yeah?<p>If your child says they've learned their multiplication tables but they can't actually multiply any numbers you give them do they actually know how to do multiplication? I would say no.
For some reason people are perfectly able to understand this in the context of, say, cursive, calculator use, etc., but when it comes to their own skillset somehow it's going to be really different.
Yes that's exactly right.
That would be the definition of learning something, yes.
I could definitely maintain the infrastructure without an llm. Albeit much slower.<p>And yes. If LLMs disappear, then we need to hire a lot of people to maintain the infrastructure.<p>Which naturally is a part of the risk modeling.
I think this is a bit dismissive.<p>It’s quite possible to be deep into solving a problem with an LLM guiding you where you’re reading and learning from what it says. This is not really that different from googling random blogs and learning from Stack Overflow.<p>Assuming everyone just sits there dribbling whilst Claude is in YOLO mode isn’t always correct.
>> I am learning a new skill with instructor at an incredible rate<p>> Could you do it again on your own?<p>Can you you see how nonsensical your stance is? You're straight up accusing GP of lying they are learning something at the increased rate OR suggesting if they couldn't learn that, presumably at the same rate, on they own, they're not learning anything.<p>That's not very wise to project your own experiences on others.
Actually, it’s much like taking a physics or engineering course, and after the class being fully able to explain the class that day, and yet realize later when you are doing the homework that you did not actually fully understand like you thought you did.
The challenge is not if you could do <i>all</i> of it without AI but <i>any</i> of it that you couldn't before.<p>Not everyone learns at the same pace and not everyone has the same fault tolerance threshold. In my experiencd some people are what I call "Japanese learners" perfecting by watching. They will learn with AI but would never do it themselves out of fear of getting <i>something</i> wrong while they understand most of it, others that I call "western learners" will start right away and "get their hands dirty" without much knowledge and also get it wrong right away. Both are valid learning strategies fitting different personalities.
Also AI could help you pick those skills up again faster, although you wouldn’t need to ever pick those skills up again unless AI ceased to exist.<p>What an interesting paradox-like situation.
I believe some professor warned us about being over reliant on Google/reddit etc: “how would you be productive if internet went down” dilemma.<p>Well, if internet is down, so is our revenue buddy. Engineering throughput would be the last of our concerns.
>I am learning at an incredible rate with LLMs.<p>I don't believe it. Having something else do the work for you is not learning, no matter how much you tell yourself it is.
If you've seen further it's only because you've stood on the shoulders of giants.<p>Having other people do work for you is how people get to focus on things they actually care about.<p>Do you use a compiler you didn't write yourself? If so can you really say you've ever learned anything about computers?
It is easy to not believe if you only apply an incredibly narrow world view.<p>Open your eyes, and you might become a believer.
Using LLMs as a learning tool isn’t what causes skill atrophy. It’s using them to solve entire problems without understanding what they’ve done.<p>And not even just understanding, but verifying that they’ve implemented the optimal solution.
It's partly that, but also reading and surface level understanding something vs generating yourself are different skills with different depths. If you're learning a language, you can get good at listening without getting good at speaking for example.
<a href="https://hex.ooo/library/power.html" rel="nofollow">https://hex.ooo/library/power.html</a><p>When future humans rediscover mathematics.
Yeah I am worried about skill atrophy too. Everyone uses a compiler these days instead of writing assembly. Like who the heck is going to do all the work when people forget how to use the low level tools and a compiler has a bug or something?<p>And don’t get me started on memory management. Nobody even knows how to use malloc(), let alone brk()/mmap(). Everything is relying on automatic memory management.<p>I mean when was the last time you actually used your magnetized needle? I know I am pretty rusty with mine.
> an LLM is exactly like a compiler if a compiler was a black box hosted in a proprietary cloud and metered per symbol<p>Yeah, exactly.
Snark aside, this is an actual problem for a lot of developers in varying degrees, not understand anything about the layers below make for terrible layers above in very many situations.
[dead]
Another aspect I haven’t seen discussed too much is that if your competitor is 10x more productive with AI, and to stay relevant you also use AI and become 10x more productive. Does the business actually grow enough to justify the extra expense? Or are you pretty much in the same state as you were without AI, but you are both paying an AI tax to stay relevant?
This is the “ad tax” reasoning, but ultimately I think the answer is greater efficiency. So there is a real value, even if all competitors use the tools.<p>It’s like saying clothing manufacturers are paying the “loom tax” tax when they could have been weaving by hand…
Software development is not a production line, the relationship between code output and revenue is extremely non-linear.<p>Where producing 2x the t-shirts will get you ~2x the revenue, it's quite unlikely that <i>10x</i> the code will get you even close to 2x revenue.<p>With how much of this industry operates on 'Vendor Lock-in' there's a very real chance the multiplier ends up 0x. AI doesn't add anything when you can already 10x the prices on the grounds of "Fuck you. What are you gonna do about it?"
Yep and in a vendor lock in scenario, fixing deep bugs or making additions in surgical ways is where the value is. And Claude helps you do that, by giving you more information, analyzing options, but it doesn’t let you make that decision 10x faster.
We already know how to multiply the efficiency of human intelligence to produce better quality than LLMs and nearly match their productivity - <i>open source</i> - in fact coding LLMs wouldn't even exist without it.<p>Open source libraries and projects together with <i>open source AI</i> is the only way to avoid the existential risks of closed source AI.
Where's the evidence of competitors being 10x more productive? So far, everyone is simply bragging about how much code they have shipped last week, but that has zero relevance when it comes to productivity
I work at a 20-year-old mid-sized SaaS company. As long as the company has been around, product managers have longed for more engineers and strategies for engineers to ship features faster. As of around February, those same product managers across the org are complaining that they can't keep up with the pace at which engineers are shipping their features. This isn't just lines of code. This is the entire company trying to figure out how to help the PMs because engineers suddenly stopped being the bottleneck.<p>I don't know about 10x, but this could only happen if PMs suddenly got really lazy or the engineers actually got at least 1.5x faster. My gut says it's way more because we're now also consistently up to date on our dependencies and completing massive refactors we were putting off for years.<p>There are lots of reasons this could be the case. Quality suddenly changed, the nature of the work changed, engineers leveled up... But for this to have happened consistently across a bunch of engineering teams is quite the coincidence if not this one thing we are all talking about.
Read it as just a given rate. The number doesn’t matter too much here, if company B does believe claims from company A they are N times more productive that’s enough to force B to adopt the same tooling.
I feel like a lot of the AI advocacy today is like the Cloud advocacy of a few years ago or the Agile advocacy before that. It's this season's silver bullet to make us all 10x more effective according to metrics that somehow never translate into adding actually useful functionality and quality 10x as fast.<p>The evangelists told us 20 years ago that if we weren't doing TDD then we weren't really professional programmers at all. The evangelists told us 10 years ago that if we were still running stuff locally then we must be paying a fortune for IT admin or not spending our time on the work that mattered. The evangelists this week tell us that we need to be using agents to write all our code or we'll get left in the dust by our competitors who are.<p>I'm still waiting for my flying car. Would settle for some graphics software on Linux that matches the state of the art on Windows or even reliable high-quality video calls and online chat rooms that don't make continental drift look fast.
The alternative is probably also true. If your F500 competitor is also handicapped by AI somehow, then you're all stagnant, maybe at different levels. Meanwhile Anthropic is scooping up software engineers it supposedly made irrelevant with Mythos and moving into literally 2+ new categories per quarter
it's worse than a tie. 10x everyone just floods the market and tanks per-unit price. you pay the AI tax and your output is worth less.
If the business doesn’t grow then you shed costs like employees
Either the business grows, or the market participants shed human headcount to find the optimal profit margin. Isn’t that the great unknown: what professions are going to see headcount reduction because demand can’t grow that fast (like we’ve seen in agriculture), and which will actually see headcount stay the same or even expand, because the market has enough demand to keep up with the productivity gains of AI? Increasingly I think software writ large is the latter, but individual segments in software probably are the former.
> your competitor is 10x more productive with AI<p>This doesn't happen. Literally zero evidence of this.
Open models keep closing the eval gap for many tasks, and local inference continues to be increasingly viable. What's missing isn't technical capability, but productized convenience that makes the API path feel like the only realistic option.<p>Frontier labs are incentivized to keep it that way, and they're investing billions to make AI = API the default. But that's a business model, not a technical inevitability.
im hoping and praying that local inference finds it's way to some sort of baseline that we're all depending on claude for here. that would help shape hardware designs on personal devices probably something in the direction of what apple has been doing.<p>ive had to like tune out of the LLM scene because it's just a huge mess. It feels impossible to actually get benchmarks, it's insanely hard to get a grasp on what everyone is talking about, bots galore championing whatever model, it's just way too much craze and hype and misinformation. what I do know is we can't keep draining lakes with datacenters here and letting companies that are willing to heel turn on a whim basically control the output of all companies. that's not going to work, we collectively have to find a way to make local inference the path forward.<p>everyone's foot is on the gas. all orgs, all execs, all peoples working jobs. there's no putting this stuff down, and it's exhausting but we have to be using claude like _right now_. pretty much every company is already completely locked in to openai/gemini/claude and for some unfortunate ones copilot. this was a utility vendor lock in capture that happened faster than anything ive ever seen in my life & I already am desperate for a way to get my org out of this.
I'm frustrated that there's not "solid" instructional tooling. I either see people just saying "keep trying different prompts and switching models until you get lucky" or building huge cantilevered toolchains that seems incredibly brittle, and even then, how well do they really work?<p>I get choice paralysis when you show me a prompt box-- I don't know what I can reasonably ask for and how to best phrase it, so I just panic. It doesn't help when we see articles saying people are getting better outcomes by adding things like "and no bugs plz owo"<p>I'm sure this is by design-- anything with clear boundaries and best practices would discourage gacha style experimentation. Can you trust anyone who sells you a metered service to give you good guidance on how to use it efficiently?
yea that is probably the worst part of these techs becoming mainstream services and local-LLM'ing taking off in general: working with them at many points in any architecture no longer feels... deterministic i guess. way too fucking much "heres what i use" but no real best practices yet, just a lot of vague gray area and everyones still in discovery-mode on how to best find some level of determinism or workflow and ways we are benchmarking is seriously a moving target. everyone has their own branded take on what the technology is and their own branded approach on how to use it, and it's probably the murkiest and foggiest time to be in technology fields that i've ever seen :\ seems like weekly/monthly something is outdated, not just the models but the tooling people are parroting as the current best tooling to use. incredibly frustrating. there's simply too much ground to cover for any one person to have any absolute takes on any of it, and because a handful of entities are currently leading the charge draining lakes and trying to compete for every person and every businesses money, there's zero organized frameworks at the top to make some sense of this. they all are banking on their secret sauce, and i _really_ want us all to get away from this. local inference has to succeed imo but goddamn there needs to be some collective working together to rally behind some common strats/frameworks here. im sure there's already countless committees that have been established to try and get in front of this but even that's messy.<p>i don't know how else to phrase it: this feels like such an unstable landscape, "beta" software/services are running rampant in every industry/company/org/etc and there's absolutely no single resource we can turn to to help stay ahead of & plan for the rapidly-evolving landscape. every, and i mean every company, is incredibly irresponsible for using this stuff. including my own. once again though, cat's already out of the bag. now we fight for our lives trying to contain it and ensure things are well understood and implemented properly...which seems to be the steepest uphill battle of my life
I'm hopeful that new efficiencies in training (Deepseek et al.), the impressive performance of smaller models enhanced through distillation, and a glut of past-their-prime-but-functioning GPUs all converge make good-enough open/libre models cheap, ubiquitous, and less resource-intensive to train and run.
The lock in is so incredibly poor. I could switch to whatever provider in minuets.<p>But it requires that one does not do something stupid.<p>Eg. For recurring tasks: keep the task specification in the source code and just ask Claude to execute it.<p>The same with all documentation, etc.
>perhaps we can come up with something like the "linux/postgres/git/http/etc" of the LLMs<p>I fear that this may not be feasible in the long term. The open-model free ride is not guaranteed to continue forever; some labs offer them for free for publicity after receiving millions in VC grants now, but that's not a sustainable business model. Models cost millions/billions in infrastructure to train. It's not like open-source software where people can just volunteer their time for free; here we are talking about spending real money upfront, for something that will get obsolete in months.<p>Current AI model "production" is more akin to an industrial endeavor than open-source arrangements we saw in the past. Until we see some breakthrough, I'm bearish on "open models will eventually save us from reliance on big companies".
"get obsolete in months"<p>If you mean obsolete in the sense of "no longer fit for purpose" I don't think that's true. They may become obsolete in terms of "can't do hottest new thing" but that's true of pretty much any technology. A capable local model that can do X will always be able to do X, it just may not be able to do Y. But if X is good enough to solve your problem, why is a newer better model needed?<p>I think if we were able to achieve ~Opus 4.6 level quality in a local model that would probably be "good enough" for a vast number of tasks. I think it's debatable whether newer models are always better - 4.7 seems to be somewhat of a regression for example.
What open models are truly competing with both Claude Code and Opus 4.7 (xhigh) at this stage?
I've had a good experience with GLM-5.1. Sure it doesn't match xhigh but comes close to 4.6 at 1/3rd the cost
That's a lame attitude. There are local models that are last year's SOTA, but that's not good enough because this year's SOTA is even better yet still...<p>I've said it before and I'll say it again, local models are "there" in terms of true productive usage for complex coding tasks. Like, for real, there.<p>The issue right now is that buying the compute to run the top end local models is absurdly unaffordable. Both in general but also because you're outbidding LLM companies for limited hardware resources.<p>You have a $10K budget, you can legit run last year's SOTA agentic models locally and do hard things well. But most people don't or won't, nor does it make cost effective sense Vs. currently subsidized API costs.
I completely see your point, but when my / developer time is worth what it is compared to the cost of a frontier model subscription, I'm wary of choosing anything but the best model I can. I would love to be able to say I have X technique for compensating for the model shortfall, but my experience so far has been that bigger, later models out perform older, smaller ones. I genuinely hope this changes through. I understand the investment that it has taken to get us to this point, but intelligence doesn't seem like it's something that should be gated.
Right; but every major generation has had diminishing returns on the last. Two years ago the difference was HUGE between major releases, and now we're discussing Opus 4.6 Vs. 4.7 and people cannot seem to agree if it is an improvement or regression (and even their data in the card shows regressions).<p>So my point is: If you have the attitude that unless it is the bleeding edge, it may have well not exist, then local models are never going to be good enough. But truth is they're now well exceeding what they need to be to be huge productivity tools, and would have been bleeding edge fairly recently.
I feel like I'm going to have to try the next model. For a few cycles yet. My opinion is that Opus 4.7 is performing worse for my current work flow, but 4.6 was a significant step up, and I'd be getting worse results and shipping slower if I'd stuck with 4.5. The providers are always going to swear that the latest is the greatest. Demis Hassabis recently said in an interview that he thinks the better funded projects will continue to find significant gains through advanced techniques, but that open source models figure out what was changed after about 6 months or so. We'll see I guess. Don't get me wrong, I'd love to settle down with one model and I'd love it to be something I could self host for free.
> I completely see your point, but when my / developer time is worth what it is compared to the cost of a frontier model subscription, I'm wary of choosing anything but the best model I can.<p>Don't you understand that by choosing the best model we can, we are, collectively, step by step devaluating what our time is worth? Do you really think we all can keep our fancy paychecks while keep using AI?
Do you think if you or me stopped using AI that everyone else will too? We're still what we always were - problem solvers who have gained the ability to learn and understand systems better that the general population, communicate clearly (to humans and now AIs). Unfortunately our knowledge of language APIs and syntax has diminished in value, but we have so many more skills that will be just as valuable as ever. As the amount of software grows, so will the need for people who know how to manage the complexity that comes with it.
First, making sure to offer an upvote here. I happen to be VERY enthusiastic about local models, but I've found them to be incredibly hard to host, incredibly hard to harness, and, despite everything, remarkably powerful if you are willing to suffer really poor token/second performance...
$10k is a lot of tokens.
> that are last year's SOTA<p>Early last year or late last year?<p>opus 4.5 was quite a leap
GLM 5.1 competes with Sonnet. I'm not confident about Opus, though they claim it matches that too.
I have it as failover to Opus 4.6 in a Claude proxy internally. People don't notice a thing when it triggers, maybe a failed tool call here and there (harness remains CC not OC) or a context window that has gone over 200k tokens or an image attachment that GLM does not handle, otherwise hunky-dory all the way. I would also use it as permanent replacement for haiku at this proxy to lower Claude costs but have not tried it yet. Opus 4.7 has shaken our setup badly and we might look into moving to Codex 100% (GLM could remain useful there too).
> open models<p>Google just released Gemma 4, perhaps that'd be worth a try?
yep!! had similar thoughts on the the "linux/postgres/git/http/etc" of the LLMs<p>made a HN post of my X article on the lock-in factor and how we should embrace the modular unix philosophy as a way out: <a href="https://news.ycombinator.com/item?id=47774312">https://news.ycombinator.com/item?id=47774312</a>
Any recommendations on good open ones? What are you using primarily?
LMArena actually has a nice Pareto distribution of ELO vs price for this<p><pre><code> model elo $/M
---------------------------------------
glm-5.1 1538 2.60
glm-4.7 1440 1.41
minimax-m2.7 1422 0.97
minimax-m2.1-preview 1392 0.78
minimax-m2.5 1386 0.77
deepseek-v3.2-thinking 1369 0.38
mimo-v2-flash (non-thinking) 1337 0.24
</code></pre>
<a href="https://arena.ai/leaderboard/code?viewBy=plot&license=open-source" rel="nofollow">https://arena.ai/leaderboard/code?viewBy=plot&license=open-s...</a>
LMArena isn't very useful as a benchmark, however I can vouch for the fact that GLM 5.1 is astonishingly good. Several people I know who have a $100/mo Claude Code subscription are considering cancelling it and going all in on GLM, because it's finally gotten (for them) comparable to Opus 4.5/6. I don't use Opus myself, but I can definitely say that the jump from the (imvho) previous best open weight model Kimi K2.5 to this is otherworldly — and K2.5 was already a huge jump itself!
qwen3.5/3.6 (30B) works well,locally, with opencode
Mind you, a 30B model (3B active) is not going to be comparable to Opus. There are open models that are near-SOTA but they are ~750B-1T total params. That's going to require substantial infrastructure if you want to use them agentically, scaled up even further if you expect quick real-time response for at least some fraction of that work. (Your only hope of getting reasonable utilization out of local hardware in single-user or few-users scenarios is to <i>always</i> have something useful cranking in the background during downtime.)
For a business with ten or more engineers/people-using-ai, it might still make sense to set this up. For an individual though, I can’t imagine you’d make it through to positive ROI before the hardware ages out.
It's hard to tell for sure because the local inference engines/frameworks we have today are not really that capable. We have barely started exploring the implications of SSD offload, saving KV-caches to storage for reuse, setting up distributed inference in multi-GPU setups or over the network, making use of specialty hardware such as NPUs etc. All of these can reuse fairly ordinary, run-of-the-mill hardware.
Since you need at least a few of H100 class hardware, I guess you need at least few tens of coders to justify the costs.
What near SOTA open models are you referring to?
I'm backing up a big dataset onto tapes, so I wanted to automate it. I have an idle 64Gb VRAM setup in my basement, so I decided to experiment and tasked it with writing an LTFS implementation. LTFS is an open standard for filesystems for tapes, and there's an implementation in C that can be used as the baseline.<p>So far, Qwen 3.6 created a functionally equivalent Golang implementation that works against the flat file backend within the last 2 days. I'm extremely impressed.
The thing I dislike about OpenCode is the lack of capabilities of their editor, also, resource intensive, for some reason on a VM it chuckles each 30 mins, that I need to discard all sessions, commits, etc.<p>I don't know if it is bun related, but in task manager, is the thing that is almost at the top always on CPU usage, turns out for me, bun is not production ready at all.<p>Wish Zed editor had something like BigPickle which is free to use without limits.
I want to bump this more than just a +1 by recommending everyone try out OpenCode. It can still run on a Codex subscription so you aren’t in fully unfamiliar territory but unlocks a lot of options.
The Codex TUI harness is also open source and you can use open models with it, so you can stay in even more familiar territory.
pi-coding-agent (pi.dev) is also great. I've been using it with Gemma 4 and Qwen 3.6.
Is this sort of setup tenable on a consumer MBP or similar?
Qwen’s 30B models run great on my MBP (M4, 48GB) but the issue I have is cooling - the fan exhaust is straight onto the screen, which I can’t help thinking will eventually degrade it, given the thermal cycling it would go through. A Mac Studio makes far more sense for local inference just for this reason alone.
For a 30B model, you want at least 20GB of VRAM and a 24GB MBP can’t quite allocate that much of it to VRAM. So you’d want at least a 32GB MBP.
I have 24GB VRAM available and haven't yet found a decent model or combination.
Last one I tried is Qwen with continue, I guess I need to spend more time on this.
It's a MoE model so I'd assume a cheaper MBP would simply result in some experts staying on CPU? And those would still have a sizeable fraction of the unified memory bandwidth available.
Is there any model that practically compares to Sonnet 4.6 in code and vision and runs on home-grade (12G-24G) cards?
im currently running a custom Gemma4 26b MoE model on my 24gb m2... super fast and it beat deepseek, chatgpt, and gemini in 3 different puzzles/code challenges I tested it on. the issue now is the low context... I can only do 2048 tokens with my vram... the gap is slowly closing on the frontier models
The Mac Minis (probably 64GB RAM) are the most cost effective.
How are you running it with opencode, any tips/pointers on the setup?
GLM 5.1 via an infra provider. Running a competent coding capable model yourself isn't viable unless your standards are quite low.
I am using GLM 5.1 and MiniMax 2.7.
Who’s your “we,” if you don’t mind sharing? I’m curious to learn more about companies/organizations with this perspective.
I’m imagining a (private/restricted) tracker style system where contributors “seed” compute and users “leech”.
Open models are only near SOTA because of distillation from closed models.
> I think that's the way forward. Actually it would be great if everybody would put more focus on open models,<p>I'm still surprised top CS schools are not investing in having their students build models, I know some are, but like, when's the last time we talked about a model not made by some company, versus a model made by some college or university, which is maintained by the university and useful for all.<p>It's disgusting that OpenAI still calls itself "Open AI" when they aren't truly open.
> we don't want a hard dependency on another multi-billion dollar company just to write software<p>My manager doesn't even want us to use copilot locally. Now we are supposed to only use the GitHub copilot cloud agent. One shot from prompt to PR. With people like that selling vendor lock in for them these companies like GitHub, OpenAI, Anthropic etc don't even need sales and marketing departments!
Opencode go with open models is pretty good
I can recommend this stack. It works well with the existing Claude skills I had in my code repos:<p>1. Opencode<p>2. Fireworks AI: GLM 5.1<p>And it is SIGNIFICANTLY cheaper than Claude. I'm waiting eagerly for something new from Deepseek. They are going to really show us magic.
or just use codex
I'm increasingly thinking the same as our spend on tokens goes up.<p>If you have HPC or Supercompute already, you have much of the expertise on staff already to expand models locally, and between Apple Silicon and Exo there are some amazingly solutions out there.<p>Now, if only the rumors about Exo expanding to Nvidia are true..
My understanding is that the major part of the cost of a given model is the training - so open models depend on the training that was done for frontier models? I'm finding hard to imagine (e.g.) RLHF being fundable through a free software type arrangement.
No, the training between proprietary and open models is completely different. The speculation that open models might be "distilled" from proprietary ones is just that, speculation, and a large portion of it is outright nonsense. It's physically possible to train on chat logs from another model but that's not "distilling" anything, and it's not even eliciting any real fraction of the other model's overall knowledge.
>perhaps we can come up with something like the "linux/postgres/git/http/etc" of the LLMs: something we all can benefit from while it not being monopolized by a single billionarie company<p>Training and inference costs so we would have to pay for them.
Is that why they are racing to release so many products? It feels to me like they want to suck up the profits from every software vertical.
[dead]
[dead]
[dead]
This is part of the reason why I'm really worried that this is all going to result in a greater economic collapse than I think people are realizing.<p>I think companies that are shelling out the money for these enterprise accounts could honestly just buy some H100 GPUs and host the models themselves on premises. Github CoPilot enterprise charges $40 per user per month (this can vary depending on your plan of course), but at this price for 1000 users that comes out to $480,000 a year. Maybe I'm missing something, but that's roughly what you're going to be spending to get a full fledged hosting setup for LLMs.
Out of curiosity, how many concurrent users could you get with a hosting setup at that price? If let's say 10% of those 1000 users were using it at the same time would it handle it? What about 30% or 100%?
Most companies don't want to host it themselves. They want someone to do it for them, and they are happy to pay for it. If it makes their lives easier and does not add complexity, then it has a lot of value.
Comments here overall do not reflect my experience -- i'm puzzled how the vast majority are using this technology day to day. 4.7 is absolute fire and an upgrade on 4.6.
I suspect the distinction is API vs subscription. The app has some kind of very restrictive system prompt which appears to heavily restrict compute without some creative coaxing. API remains solid. So if you're using OpenCode or some other harness with an API key, that's why you're still having a good time.
My initial experience with Opus 4.7 has been pretty bad and I'm sticking to Codex. But these results are meaningless without comparing outcome. Wether the extra token burn is bad or not depends on whether it improves some quality / task completion metric. Am I missing something?
Brutal. I've been noticing that 4.7 eats my Max Subscription like crazy even when I do my best to juggle tasks (or tell 4.7 to use subagents with) Sonnet 4.6 Medium and Haiku. Would love to know if anybody's found ideal token-saving approaches.
The whole version naming for models is very misleading. 4 and 4.1 seem to come from a different "line" than 4.5 and 4.6, and likewise 4.7 seems like a new shape of model altogether. They aren't linear stepwise improvements, but I think overall 4.7 is generally "smarter" just based on conversational ability.
You can configure the status line to get a feel for token usage:<p>[Opus 4.6] 3% context | last: 5.2k in / 1.1k out<p>add this to .claude/settings.json<p>"statusLine": {
"type": "command",
"command": "jq -r '\"[\\(.model.display_name)]
\\(.context_window.used_percentage // 0)% context | last:
\\(((.context_window.current_usage.input_tokens // 0) / 1000 * 10 | floor / 10))k in /
\\(((.context_window.current_usage.output_tokens // 0) / 1000 * 10 | floor / 10))k out\"'"
}
Subsidies don't last forever.
Running an open like Kimi constantly for an entire month will cost around 100-200$, being roughly equal to a pro-tier subscription. This is not my estimate so I’m more than open to hearing refutations. Kimi isn’t at all Opus-level intelligent but the models are roughly evenly sized from the guesses I’ve seen. So I don’t think it’s the infra being subsidized as much as it’s the training.
Kimi costs 0.3/$1.72 on OpenRouter, $200 for that gives you way more than you would get out of a $200 Claude subscription. There are also various subscription plans you can use to spend even less.
How do you get anything sensible out of Kimi?
I’m using Composer 2, Cursor’s model they built on top of Kimi, and it’s great. Not Opus level, but I’m finding many things don’t need Opus level.
Tell that to oil and defense companies.<p>If tech companies convince Congress that AI is an existential issue (in defense or even just productivity), then these companies will get subsidies forever.
Yeah, USA winning on AI is a national security issue. The bubble is unpoppable.<p>And shafting your customers <i>too</i> hard is bad for business, so I expect only moderate shafting. (Kind of surprised at what I've been seeing lately.)
I've been assuming this for a while. If I have a complex feature, I use Opus 4.6 in copilot to plan (3 units of my monthly limit). Then have Grok or Gemini (.25-.33) of my monthly units to implement and verify the work. 80% of the time it works every time. Leave me plenty of usage over the month.
I have a very newcomer-type question. What is the output format of your plan such that you can break context and get the other LLM to produce satisfactory results? What level of details is in the plan, bullet points, pseudo-code, or somewhere in the middle?
Yeah I've been arriving at the same thing. The other models give me way more usage but they don't seem to have enough common sense to be worth using as the main driver.<p>If I can have Claude write up the plan, and the other models actually execute it, I'd get the best of both worlds.<p>(Amusingly, I think Codex tolerates being invoked by Claude (de facto tolerated ToS violation), but not the other way around.)
Yesterday, I killed my weekly limit with just three prompts and went into extra usage for ~18USD on top
I wanted to better understand the potential impact for the tokenizer change from 4.6 and 4.7.<p>I'm surprised that it's 45%. Might go down (?) with longer context answers but still surprising. It can be more than 2x for small prompts.
Anthropic is playing a strange game. It's almost like they want you to cancel the subscription if you're an active user and only subscribe if you only use it once per month to ask what the weather in Berlin is.<p>First they introduce a policy to ban third party clients, but the way it's written, it affects claude -p too, and 3 months later, it's still confusing with no clarification.<p>Then they hide model's thinking, introduce a new flag which will still show summaries of thinking, which they break again in the next release, with a new flag.<p>Then they silently cut the usage limits to the point where the exact same usage that you're used to consumes 40% of your weekly quota in 5 hours, but not only they stay silent for entire 2 weeks - they actively gaslight users saying they didn't change anything, only to announce later that they did, indeed change the limits.<p>Then they serve a lobotomized model for an entire week before they drop 4.7, again, gaslighting users that they didn't do that.<p>And then this.<p>Anthropic has lost all credibility at this point and I will not be renewing my subscription. If they can't provide services under a price point, just increase the price or don't provide them.<p>EDIT: forgot "adaptive thinking", so add that too. Which essentially means "we decide when we can allocate resources for thinking tokens based on our capacity, or in other words - never".
Token consumption is huge compared to 4.6 even for smaller tasks. Just by "reasoning" after my first prompt this morning I went over 50% over the 5 hours quota.
I've spent the past 4+ months building an internal multi-agent orchestrator for coding teams. Agents communicate through a coordination protocol we built, and all inter-agent messages plus runtime metrics are logged to a database.<p>Our default topology is a two-agent pair: one implementer and one reviewer. In practice, that usually means Opus writing code and Codex reviewing it.<p>I just finished a 10-hour run with 5 of these teams in parallel, plus a Codex run manager. Total swarm: 5 Opus 4.7 agents and 6 Codex/GPT-5.4 agents.<p>Opus was launched with:<p>`export CLAUDE_AUTOCOMPACT_PCT_OVERRIDE=35
claude --dangerously-skip-permissions --model 'claude-opus-4-7[1M]' --effort high --thinking-display summarized`<p>Codex was launched with:<p>`codex --dangerously-bypass-approvals-and-sandbox --profile gpt-5-4-high`<p>What surprised me was usage: after 10 hours, both my Claude Code account and my Codex account had consumed 28% of their weekly capacity from that single run.<p>I expected Claude Code usage to be much higher. Instead, on these settings and for this workload, both platforms burned the same share of weekly budget.<p>So from this datapoint alone, I do not see an obvious usage-efficiency advantage in switching from Opus 4.7 to Codex/GPT-5.4.
Opus 4.6 is the main model on <a href="https://playcode.io" rel="nofollow">https://playcode.io</a>.<p>Not a secret, the model is the best on the world. Yet it is crazy expensive and this 35% is huge for us. $10,000 becomes $13,500. Don’t forget, anthropic tokenizer also shows way more than other providers.<p>We have experimented a lot with GLM 5.1. It is kinda close, but with downsides: no images, max 100K adequate context size and poor text writing. However, a great designer. So there is no replacement. We pray.
is it really unthinkable that another oss/local model will be released by deepseek, alibaba, or even meta that once again give these companies a run for their money
> is it really unthinkable that another oss/local model will be released by deepseek, alibaba, or even meta that once again give these companies a run for their money<p>Plenty of OSS models being released as of late, with GLM and Kimi arguably being the most interesting for the near-SOTA case ("give these companies a run for their money"). Of course, actually running them locally for anything other than very slow Q&A is hard.
For my working style (fine-grained instructions to the agent), Opus 4.5 is basically ideal. Opus 4.6 and 4.7 seem optimized for more long-running tasks with less back and forth between human and agent; but for me Opus 4.6 was a regression, and it seems like Opus 4.7 will be another.<p>This gives me hope that even if future versions of Opus continue to target long-running tasks and get more and more expensive while being less-and-less appropriate for my style, that a competitor can build a model akin to Opus 4.5 which is suitable for my workflow, optimizing for other factors like cost.
I'm betting on a company like Taalas making a model that is perhaps less capable but 100x as fast, where you could have dozens of agents looking at your problem from all different angles simultaneously, and so still have better results and faster.
I'm excited for Taalas, but the worry with that suggestion is that it would blow out energy per net unit of work, which kills a lot of Taalas' buzz. Still, it's inevitable if you make something an order of magnitude faster, folk will just come along and feed it an order of magnitude more work. I hope the middleground with Taalas is a cottage industry of LLM hosts with a small-mid sized budget hosting last gen models for quite cheap. Although if they're packed to max utilisation with all the new workloads they enable, latency might not be much better than what we already have today
Yeah, it's a search problem. When verification is cheap, reducing success rate in exchange for massively reducing cost and runtime is the right approach.
Nothing is unthinkable, I could think of Transformers.V2 that might look completely different, maybe iterations on Mamba turns out fruitful or countless of other scenarios.
Now that Anthropic have started hiding the chain of thought tokens, it will be a lot harder for them
Qwen released a new model the same day (3.6). The headline was kind of buried by Anthropic's release, though.<p><a href="https://news.ycombinator.com/item?id=47792764">https://news.ycombinator.com/item?id=47792764</a>
This regression put Anthropic behind Chinese models actually.
Price is now getting to be more in line with the actual cost. Th models are dumber, slower and more expensive than what we’ve been paying up until now. OpenAI will do it too, maybe a bit less to avoid pissing people off after seeing backlash to Anthropic’s move here. Or maybe they won’t make it dumber but they’ll increase the price while making a dumber mode the baseline so you’re encouraged to pay more. Free ride is over. Hope you have 30k burning a hole in your pocket to buy a beefy machine to run your own model. I hear Mac Studios are good for local inference.
We should clarify 'Scaling up' here. Does higher token consumption actually correlate with better accuracy, or are we just increasing overhead?
If anyone's had 4.7 update any documents so far - notice how concise it is at getting straight to the point. It rewrote some of my existing documentation (using Windsurf as the harness), not sure I liked the decrease in verbosity (removed columns and combined / compressed concepts) but it makes sense in respect to the model outputting less to save cost.<p>To me this seems more that it's trained to be concise by default which I guess can be countered with preference instructions if required.<p>What's interesting to me is that they're using a new tokeniser. Does it mean they trained a new model from scratch? Used an existing model and further trained it with a swapped out tokeniser?<p>The looped model research / speculation is also quite interesting - if done right there's significant speed up / resource savings.
I have been seeing this messaging everywhere and I have not noticed this. I have had the inverse with 4.7 over 4.6.<p>I think people aren’t reading the system cards when they come out. They explicitly explain your workflow needs to change. They added more levels of effort and I see no mention of that in this post.<p>Did y’all forget Opus 4? That was not that long ago that Claude was essentially unusable then. We are peak wizardry right now and no one is talking positively. It’s all doom and gloom around here these days.
This, the push towards per-token API charging, and the rest are just a sign of things to come when they finally establish a moat and full monoply/duopoly, which is also what all the specialized tools like Designer and integrations are about.<p>It's going to be a very expensive game, and the masses will be left with subpar local versions. It would be like if we reversed the democratization of compilers and coding tooling, done in the 90s and 00s, and the polished more capable tools are again all proprietary.
I doubt that’s the case. My guess is we’ll hit asymptomatic returns from transformers, but price-to-train will fall at moore’s law.<p>So over time older models will be less valuable, but new models will only be slightly better. Frontier players, therefore, are in a losing business. They need to charge high margins to recoup their high training costs. But latecomers can simply train for a fraction of the cost.<p>Since performance is asymptomatic, eventually the first-mover advantage is entirely negligible and LLMs become simple commodity.<p>The only moat I can see is data, but distillation proves that this is easy to subvert.<p>There will probably be a window though where insiders get very wealthy by offloading onto retail investors, who will be left with the bag.
If only there were an Open AI company who's mandate, built into the structure of the company, were to make frontier models available to everyone for the good of humanity.<p>Oh well
Things used to be better... really.<p>OpenAI was built as you say. Google had a corporate motto of "Don't be evil" which they removed so they could, um, do evil stuff without cognitive dissonance, I guess.<p>This is the other kind of enshitification where the businesses turn into power accumulators.
Yep, between this and the pricing for the code review tool that was released a couple weeks ago (15-25 a review), and the usage pricing and very expensive cost of Claude Design, I do wonder if Anthropic is making a conscious, incremental effort to raise the baseline for AI engineering tasks, especially for enterprise customers.<p>You could call it a rug pull, but they may just be doing the math and realize this is where pricing needs to shift to before going public.
Does this have anything to do with the default xhigh effort?
I wonder if this is like when a restaurant introduces a new menu to increase prices.<p>Is Opus 4.7 that significantly different in quality that it should use that much more in tokens?<p>I like Claude and Anthropic a lot, and hope it's just some weird quirk in their tokenizer or whatnot, just seems like something changed in the last few weeks and may be going in a less-value-for-money direction, with not much being said about it. But again, could just be some technical glitch.
One thing I don't see often mentioned - OpenAI API's auto token caching approach results in MASSIVE cost savings on agent stuff. Anthropic's deliberate caching is a pain in comparison. Wish they'd just keep the KV cache hot for 60 seconds or so, so we don't have to pay the input costs over and over again, for every growing conversation turn.
Came to a similar conclusion after running a bunch of tests on the new tokenizer<p>It was on the higher end of Anthropics range - closer to 30-40% more tokens<p><a href="https://www.claudecodecamp.com/p/i-measured-claude-4-7-s-new-tokenizer-here-s-what-it-costs-you" rel="nofollow">https://www.claudecodecamp.com/p/i-measured-claude-4-7-s-new...</a>
I’m trying to understand how this is useful information on its own?<p>Maybe I missed it, but it doesn’t tell you if it’s more successful for less overall cost?<p>I can easily make Sonnet 4.6 cost way more than any Opus model because while it’s cheaper per prompt it might take 10x more rounds (or never) solve a problem.
Everything in AI moves super quickly, including the hivemind. Anthropic was the darling a few weeks ago after the confrontation with the DoD, but now we hate them because they raised their prices a little. Join us!
Is this a weird way of saying Opus got "cheaper" somehow from 4.6 to 4.7?
Probably due to the new tokenizer: <a href="https://www.claudecodecamp.com/p/i-measured-claude-4-7-s-new-tokenizer-here-s-what-it-costs-you" rel="nofollow">https://www.claudecodecamp.com/p/i-measured-claude-4-7-s-new...</a>
Makes me think the model could actually not even be smarter necessarily, just more token dependent.
My impression the reverse is true when upgrading to GPT-5.4 from GPT-5; it uses fewer tokens(?).
But with the same tokenizer, right?<p>The difference here is Opus 4.7 has a new tokenizer which converts the same input text to a higher number of tokens. (But it costs the same per token?)<p>> Claude Opus 4.7 uses a new tokenizer, contributing to its improved performance on a wide range of tasks. This new tokenizer may use roughly 1x to 1.35x as many tokens when processing text compared to previous models (up to ~35% more, varying by content), and /v1/messages/count_tokens will return a different number of tokens for Claude Opus 4.7 than it did for Claude Opus 4.6.<p>> Pricing remains the same as Opus 4.6: $5 per million input tokens and $25 per million output tokens.<p>ArtificialAnalysis reports 4.7 significantly reduced <i>output</i> tokens though, and overall ~10% cheaper to run the evals.<p>I don't know how well that translates to Claude Code usage though, which I think is extremely input heavy.
was shocked to see phone verification roll out like last month as well... yikes
Still worth it imho for important code, but it shows that they are hitting a ceiling while trying to improve the model which they try to solve by making it more token-inefficient.
Not sure if this equates to more spend. Smarter models make fewer mistakes and thus fewer round trips.
Anthropic may have its biases, but its product is undeniably excellent.
the better the tokenizer maps text to its internal representation, the better the understanding of the model what you are saying - or coding! But 4.7 is much more verbose in my experience, and this probably drives cost/limits a lot.
45% is brutal if you're building on top of these models
as a bootstrapped founder. The unit economics just don't
work anymore at that price point for most indie products.<p>What I've been doing is running a dual-model setup —
use the cheaper/faster model for the heavy lifting where
quality variance doesn't matter much, and only route to
the expensive one when the output is customer-facing and
quality is non-negotiable. Cuts costs significantly
without the user noticing any difference.<p>The real risk is that pricing like this pushes smaller
builders toward open models or Chinese labs like Qwen,
which I suspect isn't what Anthropic wants long term.
That's the risk you take on.<p>There are 2 things to consider:<p><pre><code> * Time to market.
* Building a house on someone else's land.
</code></pre>
You're balancing the 2, hoping that you win the time to market, making the second point obsolete from a cost perspective, or you have money to pivot to DIY.
One could reconsider whether building your business on top of a model without owning the core skills to make your product is viable regardless.<p>A smaller builder might reconsider (re)acquiring relevant skills and applying them. We don't suddenly lose the ability to program (or hire someone to do it) just because an inference provider is available.
> if you're building on top of these models as a bootstrapped founder<p>This is going to be blunt, but this business model is fundamentally unsustainable and "founders" don't get to complain their prospecting costs went up. These businesses are setting themselves up to get Sherlocked.<p>The only realistic exit for these kinds of businesses is to score a couple gold nuggets, sell them to the highest bidder, and leave.
releases 4.8 and deletes everything else. and now 4.8 costs 500% more than 4.7. i wonder what it would take for people to start using kimi or qwen or other such.
Conspiracy time: they released a new version just so hey could increase the price so that people wouldn't complain so much along the lines of "see this is a new version model, so we NEED to increase the price") similar to how SaaS companies tack on some shit to the product so that they can increase prices
i think it is quite clear that staying with opus 4.6 is the way to go, on top of the inflation, 4.7 is quite... dumb. i think they have lobotomized this model while they were prioritizing cybersecurity and blocking people from performing potentially harmful security related tasks.
Hey, Boris from the Claude Code team here. People were getting extra cyber warnings when using old versions of Claude Code with Opus 4.7. To fix it, just run claude update to make sure you're on the latest.<p>Under the hood, what was happening is that older models needed reminders, while 4.7 no longer needs it. When we showed these reminders to 4.7 it tended to over-fixate on them. The fix was to stop adding cyber reminders.<p>More here: <a href="https://x.com/ClaudeDevs/status/2045238786339299431" rel="nofollow">https://x.com/ClaudeDevs/status/2045238786339299431</a>
How do you justify the API and web UI versions of 4.7 refusing to solve NYT Connections puzzles due to "safety"?<p><a href="https://x.com/LechMazur/status/2044945702682309086" rel="nofollow">https://x.com/LechMazur/status/2044945702682309086</a>
What is your response to:<p>> 4.7 is quite... dumb. i think they have lobotomized this model<p>Is adaptive thinking still broken? Why was the option to disable it taken away?
4.7 is super variable in my one day experience - it occasionally just nails a task. Then I'm back to arguing with it like it's 2023.
My experience as well, unfortunately. I am really looking forward to reading, in a few years, a proper history of the wild west years of AI scaling. What is happening in those companies at the moment must be truly fascinating. How is it possible, for instance, that I never, ever, had an instance of not being able to use Claude despite the runaway success it had, and - i'd guess - expotential increase in infra needs. When I run production workloads on vertex or bedrock i am routinely confronted with quotas, here - it always works.
That has been my Friday experience as well… very frustrating to go back to the arguing, I forgot how tense that makes me feel
Does anyone know what changed in the tokenizer? Does it output multiple tokens for things that were previously one token?
Yeah I'm seriously considering dropping my Max subscription, unless they do something in the next few days - something like dropping Sonnet 4.7 cheap and powerful.
I spent one day with Opus 4.7 to fix a bug. It just ran in circles despite having the problem "in front of its eyes" with all supporting data, thorough description of the system, test harness that reproduces the bug etc. While I still believe 4.7 is much "smarter" than GPT-5.4 I decided to give it ago. It was giving me dumb answers and going off the rails. After accusing it many times of being a fraud and doing it on purpose so that I spend more money, it fixed the bug in one shot.<p>Having a taste of unnerfed Opus 4.6 I think that they have a conflict of interest - if they let models give the right answer first time, person will spend less time with it, spend less money, but if they make model artificially dumber (progressive reasoning if you will), people get frustrated but will spend more money.<p>It is likely happening because economics doesn't work. Running comparable model at comparable speed for an individual is prohibitively expensive. Now scale that to millions of users - something gotta give.
I enjoy using Claude but I find the vibing stuff starts to cause source-code amnesia. Even if I design something and put forth a thoughtful plan, the more I increase my output the less I feel the “vibes”.<p>It’s funny everyone says “the cost will just go down” with AI but I don’t know.<p>We need to keep the open source models alive and thriving. Oh, but wait the AI companies are buying all the hardware.
We (my wallet and I) are pretty happy with GLM 5.1 and MiniMax 2.7.
Definitely seems like AI money got tight the last month or two - that the free beer is running out and enshittification has begun.
The latest qwen actually performs a little better for some tasks, in my experience<p>latest claude still fails the car wash test
Not just _wrong_. It is confused! It is actually right in the second sentence.
This was Friday, Opus 4.6.<p>>I want to wash my car. The car wash is 50 meters away. Should I walk or drive?<p>Walk. It's 50 meters — you're going there to clean the car anyway, so drive it over if it needs washing, but if you're just dropping it off or it's a self-service place, walking is fine for that distance.
This is actually a good diagnostic of whether the model is skimping on the thinking loop. Try raising thinking effort and it should get it right. Of course, if you're running this in a coding harness with a whole lot of extraneous context, the model will be awfully confused as to what it should be thinking about.
I'm going to suggest what's going on here is Hanlon's Razor for models: "Never attribute to malice that which is adequately explained by <i>a model's</i> stupidity."<p>In my opinion, we've reached some ceiling where more tokens lead only to incremental improvements. A conspiracy seems unlikely given all providers are still competing for customers and a 50% token drives infra costs up dramatically too.
The cope is real with this model. Needing an instruction manual to learn how to prompt it "properly" is a glaring regression.<p>The whole magic of (pre-nerfed) 4.6 was how it magically seemed to understand what I wanted, regardless of how perfectly I articulated it.<p>Now, Anth says that needing to explicitly define instructions are as a "feature"?!
Had a pretty heavy workload yesterday, and never hid the limit on claude code. Perhaps they allowed for more tokens for the launch?<p>Claude design on the other hand seemed to eat through (its own separate usage limit) very fast. Hit the limit this morning in about 45 mins on a max plan. I assume they are going to end up spinning that product off as a separate service.
Wow this is pretty spectacular. And with the losses anthro and OAI are running, don’t expect this trend to change. You will get incremental output improvements for a dramatically more expensive subscription plan.
Indeed, and if we accept the argument of this tech approaching AGI, we should expect that within x years, the subscription cost may exceed the salary cost of a junior dev.<p>To be clear, I'm not saying that it's a good thing, but it does seem to be going in this direction.
If LLMs do reach AGI (assuming we have an actual agreed upon definition), it would make sense to pay way more than a junior salary.
But also, LLMs won’t give us AGI (again, assuming we have an actual, meaningful definition)
I absolutely do not accept that argument. It’s clear models hit a plateau roughly a year ago and all incremental improvements come at an increasingly higher cost.<p>And junior devs have never added much value. The first two years of any engineer’s career is essentially an apprenticeship. There’s no value add from have a perpetually junior “employee”.
Is it just me? I don't feel difference between 4.6 and 4.7
[dead]
[dead]
[dead]
[flagged]
The long-term pitch of these AI companies is that the AI will essentially replace workers for low cost.<p>If the models don't get to a higher level of 'intelligence' and still struggle with certain basic tasks at the SOTA while also getting more expensive, then the pitch is misleading and unlikely to happen.<p>So yes, I expect the price to go down.
I thought it would be to get better, to stay competitive with the competitors and free models.
[flagged]