I've been saying efficiency is the next "frontier" in AI, at least for LLMs that people use daily. Companies have started to really balk at token costs from the major providers, and there's some evidence that the cheaper Chinese models are chipping away at Anthropic/OpenAI dominance from below (as cheaper Chinese products have done in other industries for many years). I continue to think that the vendor that figures out how to efficiently serve a Good Enough model at a much lower price will be the one to win in the end.<p>DeepSeek is remarkably efficient at caching and their cached token rates are crazy cheap; using it with Reasonix is free real estate, like 97% cached tokens, ends up costing like 30 cents an hour to use DeepSeek V4 Pro. I hadn't dug into MiMo's caching behavior as I haven't used it as heavily as DeepSeek, but this indicates it's close to DeepSeek.<p>At this point I don't see a reason to use Sonnet, Haiku, or the smaller GPT models, because their API rates are much higher than the best models from MiMo and DeepSeek.<p>We're still figuring out the upper bounds of capability and I am still finding next generation models are unlocking things I couldn't readily accomplish before and I'm willing to pay more for them (at least, I'll pay the $100 or $200 subscription rates for them, I couldn't justify the token expense for most of my dev work), but we're already at a point where someone building standard CRUD web apps doesn't need the top models and probably doesn't benefit much from using them.
It's really cool and interesting to see the kind of engineering that goes into Xiaomi (and Deepseeks) inference optimizations. Z.ai has also published some interesting papers although I haven't had a chance to go through them yet.<p>It does inspire hope that the Chinese labs seem to be so open although the sceptic in me does wonder what their end game is.<p>Surely, from a purely economic perspective it would be wiser to keep this proprietary and benefit from the increased API traffic?
Their game? Sell me tokens instead of me buying them from an American lab for a higher price.<p>Publishing open weights gives me more confidence in the model, and ironically makes me less anxious about making sure I can replace the cloud usage with a local alternative. Whereas I’m very nervous right now with relying on 5.6-Sol - what if they triple the price, nerf it, etc.?
> Publishing open weights gives me more confidence in the model<p>Why? It's not like you can audit weights like you can with code.<p>> what if they triple the price, nerf it, etc.?<p>What if an open weights infra provider does that? What's the difference?
Because I can run Qwen 3.6 or DeepSeek V4 until the end of time if I want to? The model is on HuggingFace; anyone can download it. I have Qwen and Gemma on my laptop right now if I want to use them, even if I decide to go be a hermit who never interacts with the outside world again.
you keep the model. it's never deprecated. with closed ai, you are forced into a new more expensive model every few months. if an open model infra provider does that, you simply switch to another one. it's not in their interest to do that.
The bet could be that they’ll ultimately be able to sell hardware capable enough of running local models comfortably.
What Chinese firms are doing makes perfect sense from the commercial perspective actually because they understand how a classic commoditization spiral works. The reality is that models themselves are general commodities and there's just not enough difference between them. A company can get ahead of others by a few months, but then the rest quickly close the gap. It's a really low margin business because there's no way to differentiate yourself.<p>Chinese companies know that there's no profit in general purpose models in the long run, and they're treating models as shared infrastructure akin to Linux. They're amortizing the cost of research by keeping models open, and rapidly closing the gap and driving prices towards the marginal cost of inference. The money is going to be in customization niches. Companies will charge to tune models for specific use cases and charge support for that. There's also going to be money at the bottom for hardware vendors making chips and memory. But the middle tier of generic LLMs is seeing involution where there's relentless competition driving profits towards the bottom.
> <i>the kind of engineering that goes into Xiaomi (and Deepseeks) inference optimizations</i><p>At Xiaomi, MiMo is now led by Luo Fuli. She is a former Alibaba & DeepSeek employee: <a href="https://newsen.pku.edu.cn/news_events/news/people/15385.html" rel="nofollow">https://newsen.pku.edu.cn/news_events/news/people/15385.html</a> (<a href="https://archive.vn/I8Pmu" rel="nofollow">https://archive.vn/I8Pmu</a>) / <a href="https://e.vnexpress.net/news/tech/personalities/who-is-luo-fuli-the-30-year-old-ai-genius-girl-behind-deepseek-s-success-4953445.html" rel="nofollow">https://e.vnexpress.net/news/tech/personalities/who-is-luo-f...</a> (<a href="https://archive.vn/sb3B6" rel="nofollow">https://archive.vn/sb3B6</a>)<p>Don't know if it is due to Luo, but it is striking how similar performance & pricing of the models, DeepSeek v4 Pro & MiMo v2.5 Pro, is.
Standard commoditize your complement.
its a governement mandate that states that AI research must be open source ,that's one benefit of communism
This is really neat. They've done some really impressive engineering here ngl with the ~95% KV cache hit rates. MiMo and Deepseek both do seem to get the job done for me. Hope they can keep this pace while staying open source.
Will they lower the price or is this documenting past work
Such a well written article, refreshing to read in between all the slop.<p>I've used Mimo extensively in the past few months, can't wait to see what v3 will bring.
[dead]