It could be interesting to do the metric of intelligence per second.<p>ie intelligence per token, and then tokens per second<p>My current feel is that if Sonnet 4.6 was 5x faster than Opus 4.6, I'd be primarily using Sonnet 4.6. But that wasn't true for me with prior model generations, in those generations the Sonnet class models didn't feel good enough compared to the Opus class models. And it might shift again when I'm doing things that feel more intelligence bottlenecked.<p>But fast responses have an advantage of their own, they give you faster iteration. Kind of like how I used to like OpenAI Deep Research, but then switched to o3-thinking with web search enabled after that came out because it was 80% of the thoroughness with 20% of the time, which tended to be better overall.
Yeah I agree with this. We might be able to benchmark it soon (if we can’t already) but asking different agentic code models to produce some relatively simple pieces of software. Fast models can iterate faster. Big models will write better code on the first attempt, and need less loop debugging. Who will win?<p>At the moment I’m loving opus 4.6 but I have no idea if its extra intelligence makes it worth using over sonnet. Some data would be great!
Interesting perspective. Perhaps also the user would adopt his queries knowing he can only to small (but very fast) steps. I wonder who would win!
I'm not sold on diffusion models.<p>Other labs like Google have them but they have simply trailed the Pareto frontier for the vast majority of use cases<p>Here's more detail on how price/performance stacks up<p><a href="https://artificialanalysis.ai/models/mercury-2" rel="nofollow">https://artificialanalysis.ai/models/mercury-2</a>
I’d push back a bit on the Pareto point.<p>On speed/quality, diffusion has actually moved the frontier. At comparable quality levels, Mercury is >5× faster than similar AR models (including the ones referenced on the AA page). So for a fixed quality target, you can get meaningfully higher throughput.<p>That said, I agree diffusion models today don’t yet match the very largest AR systems (Opus, Gemini Pro, etc.) on absolute intelligence. That’s not surprising: we’re starting from smaller models and gradually scaling up. The roadmap is to scale intelligence while preserving the large inference-time advantage.
Co-founder / Chief Scientist at Inception here. If helpful, I’m happy to answer technical questions about Mercury 2 or diffusion LMs more broadly.
How big is Mercury 2? How many tokens is it trained on?<p>Is it's agentic accuracy good enough to operate, say, coding agents without needing a larger model to do more difficult tasks?
Seems to work pretty well, and it's especially interesting to see answers pop up so quickly! It is easily fooled by the usual trick questions about car washes and such, but seems on par with the better open models when I ask it math/engineering questions, and is obviously much faster.
What excites me most about these new 4figure/second token models is that you can essentially do multi-shot prompting (+ nudging) and the user doesn't even feel it, potentially fixing some of the weird hallucinatory/non-deterministic behavior we sometimes end up with.
It seems like the chat demo is really suffering from the effect of everything going into a queue. You can't actually tell that it is fast at all. The latency is not good.<p>Assuming that's what is causing this. They might show some kind of feedback when it actually makes it out of the queue.
Comment retracted. My bad, missed some details.
I think your comment is a bit unfair.<p>> no reasoning comparison<p>Benchmarks against reasoning models:<p><a href="https://www.inceptionlabs.ai/blog/introducing-mercury-2" rel="nofollow">https://www.inceptionlabs.ai/blog/introducing-mercury-2</a><p>> no demo<p><a href="https://chat.inceptionlabs.ai/" rel="nofollow">https://chat.inceptionlabs.ai/</a><p>> no info on numbers of parameters for the model<p>This is a closed model. Do other providers publish the number of parameters for their models?<p>> testimonials that don't actually read like something used in production<p>Fair point.
Just to clarify one point: Mercury (the original v1, non-reasoning model) is already used in production in mainstream IDEs like Zed:
<a href="https://zed.dev/blog/edit-prediction-providers" rel="nofollow">https://zed.dev/blog/edit-prediction-providers</a><p>Mercury v1 focused on autocomplete and next-edit prediction. Mercury 2 extends that into reasoning and agent-style workflows, and we have editor integrations available (docs linked from the blog). I’d encourage folks to try the models!
You are right edited my post (twice actually). Missed the chat first time around (though its hard to see it as a reasoning model when chain of thought is hidden, or not obvious. I guess this is thd new nirmal), and also missed the reasoning table because text is pretty small on mobile and I thought its another speed benchmark.
Reading such obvious LLM-isms in the announcement just makes me cringe a bit too, ex.<p>> We optimize for speed users actually feel: responsiveness in the moments users experience — p95 latency under high concurrency, consistent turn-to-turn behavior, and stable throughput when systems get busy.
Genuine question: what kinds of workloads benefit most from this speed? In my coding use, I still hit limitations even with stronger models, so I'm interested in where a much faster model changes the outcome rather than just reducing latency.
I think it would assist in exploiting exploring multiple solution spaces in parallel, and can see with the right user in the loop + tools like compilers, static analysis, tests, etc wrapped harness, be able to iterate very quickly on multiple solutions. An example might be, "I need to optimize this SQL query" pointed to a locally running postgres. Multiple changes could be tested, combined, and explain plan to validate performance vs a test for correct results. Then only valid solutions could be presented to developer for review. I don't personally care about the models 'opinion' or recommendations, using them for architectural choices IMO is a flawed use as a coding tool.<p>It doesn't change the fact that the most important thing is verification/validation of their output either from tools, developer reviewing/making decisions. But even if don't want that approach, diffusion models are just a lot more efficient it seems. I'm interested to see if they are just a better match common developer tasks to assist with validation/verification systems, not just writing (likely wrong) code faster.
I've tried a few computer use and browser use tools and they feel relatively tok/s bottlenecked.<p>And in some sense, all of my claude code usage feels tok/s bottlenecked. There's never really a time where I'm glad to wait for the tokens, I'd always prefer faster.
Once you make a model fast and small enough, it starts to become practical to use LLMs for things as mundane as spell checking, touchscreen-keyboard tap disambiguation, and database query planning. If the fast, small model is multimodal, use it in a microwave to make a better DWIM auto-cook.<p>Hell, want to do syntax highlighting? Just throw buffer text into an ultra-fast LLM.<p>It's easy to overlook how many small day-to-day heuristic schemes can be replaced with AI. It's almost embarrassing to think about all the totally mundane uses to which we can put fast, modest intelligence.
multi-model arbitration, synthesis, parallel reasoning etc. Judging large models with small models is quite effective.
[dead]
I am little underwhelmed by anything diffusion at the moment - they didn't really deliver.
Please pre-render your website on the server. Client-side JS means that my agent cannot read the press-release and that reduces the chance I am going to read it myself. Also, day one OpenRouter increases the chance that someone will try it.