I'm not sure what people are on in the comments. It doesn't <i>beat</i> the other models, but it sure competes despite its size.<p>GLM 5.1 is an excellent model, but even at Q4 you're looking at ~400GB.
Kimi K2.5 is really good too, and at Q4 quantization you're looking at almost ~600GB.<p>This model? You can run it at Q4 with 70GB of VRAM. This is approaching consumer level territory (you can get a Mac Studio with 128GB of RAM for ~3500 USD).<p>For the Claude-pilled people, I don't know if you only run Opus but when I was on the Pro plan Sonnet was already extremely capable. This beats the latest Sonnet while running locally, without anyone charging you extra for having HERMES.md in your repo, or locking you out of your account on a whim.<p>Mistral has never been competitive at the frontier, but maybe that is not what we need from them. Having Pareto models that get you 80% of the frontier at 20% of the cost/size sounds really good to me.
I didn't know about HERMES.md ... (??) - found information here for others who are curious <a href="https://github.com/anthropics/claude-code/issues/53262" rel="nofollow">https://github.com/anthropics/claude-code/issues/53262</a>
This github thread is incredible, thanks for sharing. This link should be its own HN topic.
That is insane, if you billed me an extra $200 for a bug in your system I'd flat out cancel my subscription. If you're not going to credit that back to me, you don't deserve anymore of my money. I'm a Claude first guy, but if you're going to bill me incorrectly, that's on you, own it, fix it.
Let's not forget Qwen 35B A3B MoE. It gets better performance than this in all the metrics for a fraction of the memory / compute footprint.<p>Sad to see all the non Chinese open source models being at least one generation behind.
> This model? You can run it at Q4 with 70GB of VRAM. This is approaching consumer level territory (you can get a Mac Studio with 128GB of RAM for ~3500 USD).<p>The one thing I would want everyone curious about local LLMs to know is that being able to run a model and being able to run a model fast are two very different thresholds. You can get these models to run on a 128GB Mac, but we need to first tell if Q4 retains enough quality (models have different sensitivities to quantization) and how fast it runs.<p>For running async work and background tasks the prompt processing and token generation speeds matter less, but a lot of Mac Studio buyers have discovered the hard way that it's not going to be as responsive as working with a model hosted in the cloud on proper hardware.<p>For most people without hard requirements for on-site processing, the best use case for this model would be going through one of the OpenRouter hosted providers for it and paying by token.<p>> This beats the latest Sonnet while running locally<p>Almost every open weight model launch this year has come with claims that it matches or exceeds Sonnet. I've been trying a lot of them and I have yet to see it in practice, even when the benchmarks show a clear lead.
> The one thing I would want everyone curious about local LLMs to know is that being able to run a model and being able to run a model fast are two very different thresholds. You can get these models to run on a 128GB Mac, but we need to first tell if Q4 retains enough quality (models have different sensitivities to quantization) and how fast it runs.<p>Very valid. This is an active area of research, and there are a lot of options to try out already today.<p>- People have successfully used TurboQuant to quantize model weights (TQ3_4S), not just the context KV, to achieve smaller sizes than Q4 (~3.5 bpw) with much better PPL and faster decoding.<p>- Importance-weighted quantization (e.g. IQ4) also provides way better PPL, KDL, etc. at the same size as a Q4.<p>- DFlash (block diffusion for speculative decoding) needs a good drafting model compatible with the big model, but can provide an uplift up to 5x in decoding (although usually in the 2-2.5x range)<p>- Forcing a model's thinking to obey a simple grammar has been shown to improve results with drastically lower thinking output (faster effective result generation) although that has been more impactful on smaller models.<p>We should be skeptical, but it's definitely trending in the right direction and I wouldn't be surprised if we are indeed able to run it at acceptable speeds.<p>> Almost every open weight model launch this year has come with claims that it matches or exceeds Sonnet. I've been trying a lot of them and I have yet to see it in practice, even when the benchmarks show a clear lead.<p>This hasn't been my experience. After Anthropic's started their shenanigans I've switched to exclusively using open-weights models via OpenRouter and OpenCode and I can't really tell a difference (for better or for worse).
Cloud hardware is not inherently more "proper" than what's being proposed here, there's nothing wrong per se about targeting slower inference speeds in an on prem single-user context.
> Cloud hardware is not inherently more "proper" than what's being proposed here<p>Cloud hardware can run the original model. Quantization will reduce quality. The quality drop to Q4 is not trivial.<p>Cloud hardware is also massively faster in time to first token and token generation speed.<p>> there's nothing wrong per se about targeting slower inference speeds in a local single-user context.<p>If that's what the user wants and expects then it's fine<p>Most people working interactively with an LLM would suffer from slower turns.
> Cloud hardware can run the original model. Quantization will reduce quality.<p>New models are often being released in quantized format to begin with. This is true of both Kimi and the new DeepSeek V4 series. There is no "original model", the model is generated using Quantization Aware Training (QAT).
The quantization for some models can be very detrimental and their quality can drop considerably from the posted benchmarks which are probably at bf16, this is why having considerable RAM can be important.
> For the Claude-pilled people, I don't know if you only run Opus but when I was on the Pro plan Sonnet was already extremely capable.<p>Before February I was able to use Opus on High exclusively on my Max plan no problem. Now I've shifted to just using Sonnet on high and yeah, its pretty capable. I love that, Claude Pilled. ;)
Yeah, you can run it locally if you have enough VRAM, but the reports trickling in are saying about 3 tok/sec. This was on a Strix Halo box which definitely has the needed VRAM, but isn't going to have as high mem bandwidth as a GPU card, it's going to be similar on a Mac - that's the dilemma... the unified memory machines have the VRAM, but the bandwidth isn't great for running dense models. This size of a dense model is only going to be runnable (usefully) by very few people who have multiple GPU cards with enough memory to add up to about 70GB.
I don't think this is quite correct, a Strix Halo box usually has 256 GB/s memory bandwidth. An M5 Max has 614 GB/s. An M3 Ultra (no M4 or M5 Ultra) has 820 GB/s. It's still not GDDR or HBM territory, but still significantly faster.<p>That's the edge of Apple Silicon for AI. When they scale up the chip they add more memory controllers which adds more channels and more bandwidth.<p>But yeah in the end it's still going to be only a handful of people that can run it.<p>What I meant is that I think researching and developing smaller more powerful model is more interesting than chasing the next 3T parameter model while burning through VC money and squeezing your customer base more and more aggressively.
The competition is on DeepSeek v4 Flash for similar size / deployment target.
Eh. Those results would be noteworthy if it was a a MoE. A 120B dense? Firmly in meh territory.
It has similar SWE bench score to qwen 3.6 27b[1]. No one is comparing it to frontier.<p>[1]: There is no other common benchmark in the blog.
Isn't Kimi K2.6 natively INT4?
>This model? You can run it at Q4 with 70GB of VRAM.
>This beats the latest Sonnet while running locally<p>Not sure it will beat Sonet at Q4.<p>>This is approaching consumer level territory (you can get a Mac Studio with 128GB of RAM for ~3500 USD).<p>For $3500 I can get 7-8 years of GLM using coding plans, have a faster model and much better code quality.
> Not sure it will beat Sonet at Q4.<p>Very valid. Importance-weighted quantization and TurboQuant on model weights can reduce loss a lot compared to "traditional" Q4 so one can be hopeful.<p>> For $3500 I can get 7-8 years of GLM using coding plans, have a faster model and much better code quality<p>But you will own no computer, and that's also assuming prices stay what they are. Anyway my point was not whether or not it makes financial sense for everyone. A lot of people are very happy not owning their movies, software, games, cars or house. I'm just happy there is a future where the people can own and locally run the tech that was trained on their stolen data.
> For $3500 I can get 7-8 years of GLM<p>mind sharing where's the go to place to pay for open models?
I recommend using OpenRouter (openrouter.ai). Basically a broker between inference providers and you which allows you to pick, try, and switch models from a massive catalog, extremely transparent about usage and pricing.
You can get GLM coding plans from Z.ai and Ollama Cloud and OpenCode Go.
I would love to be able to run frontier locally, but I think the larger importance of open weight models is price accountability.<p>In the US with our broken system of capitalism, it’s the only way we can tether these companies to reality. Left to their own devices, I’m not convinced they would actually compete with each other on price.<p>Buy nobody like to talk about how “moat” building is fundamentally anti-competitive, even in name.<p>Funny that self proclaimed capitalists hate the system in practice. Commodity pricing is what truly terrifies them.
The point is it's open weight and is tiny compared to a lot of it's competitors. 4gpus for world class performance - sweet!
It’s 128b <i>dense</i> model. Good luck getting more than 3t/s out of a mac. It doesn’t matter if it fits or not.
You could run it on a single Mac Studio with M3 Ultra, or two Mac Studios with M4 Max at higher perf than that. And lightly quantizing this could give us modern dense models in the ~80GB size range, which is a very compelling target.
Wouldn't matter much still. M3 ultra has 819GB/s unified memory bandwidth. That means theoretical max tokem rate is 819/128 =~ 6.39 t/s. At 80 GB (5 bit quantization), its still near about 10 t/s ... far from a good coding experience. Also, these are theoretical max.. real world token generation rates would be at least 15-20% less.
[dead]
I was hoping a lot from it... but this one, is not up to that mark. For example, here is it's comparion with 4.7x smaller model, qwen3.7-27b.<p><a href="https://chatgpt.com/share/69f239e8-7414-83a8-8fdd-6308906e5f73" rel="nofollow">https://chatgpt.com/share/69f239e8-7414-83a8-8fdd-6308906e5f...</a><p>Tldr: qwen3.6-27b, a 4.7x smaller model, have similar performance.
That's a chatgpt summary. Actual usage would a better test.
To be fair MoE from Qwen itself had the same "problem". 3.5 122B MoE was same or worse than 3.5 27B. Yet to see 122B 3.6.<p>UPD. NVM, Mistral Medium 3.5 is dense. So yes, it is worse in every way.
As always, rooting for these guys — model and national diversity is great. This looks like a solid foundation to build on; hopefully the 3.6/3.7 will dial in more gains. It looks like maybe from the computer use benchmarks that their vision pipeline could use improvement, but that’s just speculation.<p>The different results on some benchmarks vibes as if this is truly an independently trained model, not just exfiltrated frontier logs, which I think is also really important - having different weight architectures inside a particular model seems like a benefit on its own when viewed from a global systems architecture perspective.
Compared to all other hosted LLMs that I have tested, Mistral seems to be the only one with rather strict CSP headers. When you ask them to create a website with some javascript library it will not preview, even though le chat offers canvas mode.<p>Sometimes when a new release comes around from any provider I just want to test it a bit on the web. without paying and using an agent harness.<p>Why are they like this ;_;<p>Edit: Christ on a bike it's bad at drawing SVGs <a href="https://chat.mistral.ai/chat/23214adb-5530-4af9-bb47-90f52192f274" rel="nofollow">https://chat.mistral.ai/chat/23214adb-5530-4af9-bb47-90f5219...</a>
<i>> Edit: Christ on a bike it's bad at drawing SVGs </i><p>On the bike would be an improvement. Geez.<p>I know SVGs may not be the best benchmark, but that matches my experience of trying to run a (previous) Mistral model in Mistral Vibe, asking it to help me configure an MCP server in Vibe. It confidently explained that MCP is the MineCraft Protocol and then began a search of my computer looking for Minecraft binaries.
I have never wanted, needed or hoped to draw svgs with an LLM. All of the models suck at it, some are just more fun or something.
It's okay, nothing exceptional, but any news from non US and non Chinese models is still good news.
It's funny that 128B is now considered Medium. I remember back in the day when 355<i>M</i> parameters was considered medium with GPT-2.
The Vibe CLI is really bad on Windows, sure they don’t officially support it, so can’t blame them, but a FYI for anyone wanting to try it. It can’t get find and replace right.
I can't figure out if this is available in the official Mistral API or not.<p>Their model listing API returns this:<p><pre><code> {
"id": "mistral-medium-2508",
"object": "model",
"created": 1777479384,
"owned_by": "mistralai",
"capabilities": {
"completion_chat": true,
"function_calling": true,
"reasoning": false,
"completion_fim": false,
"fine_tuning": true,
"vision": true,
"ocr": false,
"classification": false,
"moderation": false,
"audio": false,
"audio_transcription": false,
"audio_transcription_realtime": false,
"audio_speech": false
},
"name": "mistral-medium-2508",
"description": "Update on Mistral Medium 3 with improved capabilities.",
"max_context_length": 131072,
"aliases": [
"mistral-medium-latest",
"mistral-medium",
"mistral-vibe-cli-with-tools"
],
"deprecation": null,
"deprecation_replacement_model": null,
"default_model_temperature": 0.3,
"type": "base"
},
</code></pre>
So that has the alias "mistral-medium-latest", but the official ID is "mistral-medium-2508" which suggests it's the model they released in August 2025.<p>But... that 1777479384 timestamp decodes to Wednesday, April 29, 2026 at 04:16:24 PM UTC<p>So is that the new Mistral Medium?
Some poking around in the source code for <a href="https://github.com/mistralai/mistral-vibe" rel="nofollow">https://github.com/mistralai/mistral-vibe</a> got me to this:<p><pre><code> curl https://api.mistral.ai/v1/chat/completions \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $(llm keys get mistral)" \
-d '{
"model": "mistral-medium-3.5",
"messages": [
{"role": "user", "content": "Generate an SVG of a pelican riding a bicycle"}
]
}'
</code></pre>
Which did work: <a href="https://gist.github.com/simonw/f3158919b18d2c47863b0a5dc257a355#response" rel="nofollow">https://gist.github.com/simonw/f3158919b18d2c47863b0a5dc257a...</a> - it's pretty disappointing.<p>Weird that it doesn't show up in the model list:<p><pre><code> curl https://api.mistral.ai/v1/models \
-H "Content-Type: application/json" \
-H "Authorization: Bearer $(llm keys get mistral)" | jq</code></pre>
This is a very interesting strategy that might pay off. This model is a very good option for enterprise self host. I would argue a lot of companies are VRAM constrained rather than compute constrained. You could fit 4-5 running instances on one H100 cluster where you can only fit 1-2 Kimi K2 or GLM5.
This release Mistral really reminds you of the gap between the frontier labs and everyone else.<p>Pre-agent, there wasn't always an obvious difference between models. Various models had their charms. Nowadays, I don't want to entertain anything less than the frontier models. The difference in capability is enormous and choosing anything less has a real cost in terms of productivity.<p>I've been a big fan of the smaller labs like Mistral and especially Cohere but it's been a while since I've been excited by a release by either company.<p>That said, I'm using mistral voxtral realtime daily – it's great.
Can't agree at all. Productivity gap just 1 year ago was much larger for frontier model vs non-frontier. Let alone 2 years ago.
Same the gap is almost paper thin for anyone who hasn't gone full uninformed vibe code.
When I was thinking pre-agentic, I was actually thinking more pre-"coding seen as the main use case for these models".
Coding has always been the main real-world business usecase since day one. There has been no point since the very first public availability of GPT 3.5 in November 2022, that it wasn't.<p>A lot of us have been agentic coding since almost 2 years ago, mid-2024. I have. The productivity gap of "best vs 2nd vs 3rd best model" was biggest back then and has slowly been shrinking ever since.
> Pre-agent, there wasn't always an obvious difference between models. Various models had their charms. Nowadays, I don't want to entertain anything less than the frontier models. The difference in capability is enormous and choosing anything less has a real cost in terms of productivity.<p>It's just apples to oranges.<p>There is not a clear, across the board, winner on non-agentic tasks between Gemini, ChatGPT, and Claude - the simple chatbot interface.<p>But Claude Code is substantially better than Codex which itself is notably better than Gemini-cli.<p>In this vein, it should not be surprising that Claude Code is way better than non-frontier models for agentic coding... It's substantially better than other frontier models at specialized agentic tasks.
I’ve been comparing Claude Code and Codex extensively side by side over the past couple of weeks with my favorite prompting framework superpowers…<p>From my perspective, Claude Code is decidedly not better than Codex. They’re slightly different and work better together. I would have no issues dropping CC entirely and using codex 100%.<p>If you’re working off of “defaults”, in other words no custom prompting, Claude Code does perform a lot better out of the box. I think this matters, but if you’re a professional software developer, I’d make the case that you should be owning your tools and moving beyond the baked in prompts.
I think there's a fair amount of evidence that the heavy harnesses actually drag down performance compared to bare harnesses.
CC is not better than Codex, nor is it better than OpenCode, Crush, Pi etc…
> Pre-agent, there wasn't always an obvious difference between models. Various models had their charms. Nowadays, I don't want to entertain anything less than the frontier models.<p>This is a very naive and misguided opinion. In most tasks, including complex coding tasks, you can hardly tell the difference between a frontier model and something like GPT4.1. You need to really focus on areas such as context window, tool calling and specific aspects of reasoning steps to start noticing differences. To make matters worse, frontier models are taking a brute force approach to results which ends up making them far more expensive to run, both in terms of what shows up on your invoice and how much more you have to wait to get any resemblance of output.<p>And I won't even go into the topic or local models.
[dead]
Given what Vibe already did in the previous versions with codestral-v2, that's great news. Keep up the good work ! I don't want to depend on the world's two hungry superpowers.
I'm using mistral-medium-2508 for some text transformation operations. It's giving me better results than mistral-large for my use cases.
Looking forward to testing this new model, although I'm not sure if it's really meant at replacing the previous medium model since it's a lot more expensive and presented more as a coding / agentic model (mistral-medium-2508 was priced $0.4/$2 per 1M tokens, mistral-medium-3.5 is $1,5/$7.5).
With most OSS releases being MoEs, and modern GPUs optimized for MoEs, can somebody with knowledge of the topic explain or speculate why Mistral might have opted for a dense model?
Modern GPUs aren't optimized for MoEs though?<p>The advantage to a dense model like this Mistral one is that it is as smart as a much larger MoE model so it can fit on less GPUs. The tradeoff is that it is much slower since it has to read 100% of its weights for every token, MoE models typically only read about a tenth (though sparsity levels vary).
I like the idea of Mistral, but the last time I evaluated Mistral Vibe it was really nice for $15/month but not as effective as Gemini Plus with AntiGravity and gemini-cli. I am currently running Gemini Ultra on a 3 month 'special deal' and AntiGravity with Opus 4.7 tokens is pretty much fantastic.<p>That said, when I stop spending money on Gemini Ultra, I will give Mistral Vibe another 1-month test.<p>I like the entire business model and vibe of Mistral so much more than OpenAI/Anthropic/Google but I also have stuff to get done. I am curious if Mistral Vibe for $15/month is a stable business model (i.e., can they make a profit).
How do you feel about the responsiveness of gemini-cli? I tried it on a paid plan and the 10-minute hang-ups (per step, not the whole plan execution) really break the illusion of performance gains, unless you run it in the background and do something else in the meantime. It's more noticeable when Americans are awake.
I'm testing it right now and it seems very buggy and unstable, just like before.
I use Mistral Le Chat quite a bit.<p>One thing in particular I was disappointed in was its bad explanations when asking about French grammar. It made multiple mistakes and the other models got it right, even Qwen 3.6 27b!<p>Anyway, I'm hoping they catch up some more.
There's a good chance that they'll catch up. The "AI race" is a race to the bottom, with the leaders blowing huge wads of cash on capabilities that get replicated months later by the competition at a fraction of the cost.<p>The only benefit of leading is mindshare. OpenAI is doubling down on that, by investing in communication companies. That's their pathetic attempt at a "moat".
A 1000B model, can we call it 1KB model?
I'm rooting for Mistral. It seems they are making a big bet that smaller models will win over larger ones and I can see it happening. I was running some simple chat and tool-calling benchmarks for small models and Mistral Small 4 performed well for it's price ($.15/$.60). Seeing this today got me excited, benchmarks seems solid compared to models much larger, but it's priced higher than Haiku, 5.4 mini, and all the the Chinese models it's comparing itself too. It's not even winning those benches either, just being competitive with them, which is great, those models are 5x+ the size, but they are also 1/2 the price. Hard to be excited about that.
TLDR: Mistral Medium 3.5, text-only, 128B dense model, 256k context window, modified MIT license. Model is ~140G ...<p><a href="https://huggingface.co/mistralai/Mistral-Medium-3.5-128B" rel="nofollow">https://huggingface.co/mistralai/Mistral-Medium-3.5-128B</a><p>They more or less claim this exceeds Claude Sonnet 3.5 on most things, but is worse than Sonnet 3.6, and exceeds all other open models.<p>Oh and they have a cloud service that will code your apps "in the cloud". But, yeah, at this point, so does my cat.<p>And, yes, unsloth is on it: <a href="https://huggingface.co/unsloth/Mistral-Medium-3.5-128B-GGUF" rel="nofollow">https://huggingface.co/unsloth/Mistral-Medium-3.5-128B-GGUF</a> (but 4bit quant is 75G)
Sonnet 4.5 and 4.6*<p>There is no way it exceeds “all other” open models - but it does exceed all of Mistral’s past models.<p>You can see it getting blown past by GLM 5.1 and Kimi in this.<p>Still excited to give it a try
Unfortunately they only compare to old “all other open models”. There are probably over 10 other open models better than it by now.
You mean Sonnet 4.5 and 4.6 riight
Oh they are still a thing?! Completely forgot about Mistral. I am assuming they are still burning trough investor money.
Think they’re positioned pretty well. They’ve got an edge in the European corporate space and don’t have ungodly large numbers to hit
I believe they'll get profitable sooner than their frontier competition. Their operating costs seem to peanuts compared to the providers they're compared to most often while having the local advantage of not being Chinese nor American.
> they are still burning trough investor money<p>Difficult to say, this information is not really public. That said, those investors include EU agencies and European multinational companies and governments. It’s not as flashy as the ridiculous sums OpenAI is getting but it should be enough to keep them going for a while.<p>They also have a different business model. They are selling their expertise to fine tune and adapt their models to on-premises computers (which they can help you build) to handle confidential data and information. I would not be surprised that the revenue they get from normal people is negligible in comparison.
What's better than Voxtral for locally processed voice input? More competition is always better.
I want to believe it's gonna be good, but after trying GPT-5.5 even the most advanced Chinese models seem depressing.
This is a French model sir
Then you’ll be happy to learn it’s not Chinese
GP is stating that the second best in the field, the Chinese, is so far behind the best in the field, GPT 5.5, that it is not even worth testing anything else.
Thanks for the translation, I did not express it very clearly. Anything that I try is so much worse.
Is GPT 5.5 the best in the field? I think Opus is still better despite Anthropic's recent stumbling.
I am not following this obsession with SOTA and benchmark rankings<p>I have been using DeepSeek and GLMnmodels with OpenCode and Codex and Claudr side by side.<p>I have not found the Chinese models lacking. I enjoy for coding and like to maintain full control of my codebade and deeply care about the GOF patterns. So I am very stringent in terms of what I want the LLM to code and how to code.<p>So from my perspective, they are all about the same.
That I agree with, but for more complex autonomous changes the differences are considerable. However, it seems that most models will reach the saturation time in which they will be useful for almost everything and the difference will be in more and more niche and specialized tasks.
Honestly I depends on the context which this performance matters. Mistral is quiet cheap
Looks at first graph. It's SWE-Bench Verified. A benchmark Open-AI stopped using two months ago due to contamination.<p>Doesn't look to promising. Is there any reason to consider Mistral other than it's not US?
They did not stop using it due to contamination. They said it's flawed and indirectly said anthropics results were impossible. It's very possible they are sore losers
If it's not US and it's within a few percent of SOTA that might be good enough for a lot of people (eg Europeans)
Gemma has been better for us at EU languages than mistral (for comparable sized models) :/ so ... dunno. What mistral does well and others are lagging behind is deploying on prem with their engineers and know-how, offering tuned models for your tasks and finetuning on your own data. (I expect google to start offering this next)
It's sad that despite their strength in this for onprem, they're so behind on this in the cloud. No publicly available cloud SFT at all. Meanwhile Google has been offering that for years - though remains to be seen if they will for Gemini 3 when GA.<p>And on top of it a range of providers like Fireworks and so on that offer it for Chinese models. This seems such an obvious thing for Mistral to offer.
Price and speed.