Subtle quote from their website:<p>> Advancing the Frontier of Open Intelligence.<p>Too bad you probably need something like an 8x B300 node to run this. Black Friday is coming soon though, but I fear this open intelligence will be pacing anyone's liquidity frontier so I'll probably keep trailing the frontier for a while with nvidia/qwen3.8-27B-NVFP4 on dual RTX PRO 4000 in TP=2. I can <i>almost</i> fit 450k context in there.
The benefit is that you can use these models on a variety of different providers. You're not stuck with a single proprietary provider. Aggregators like OpenCode Zen and OpenCode Go can shop around providers on these. OpenRouter shows you the costs of a bunch of the providers, and you can route to the cheapest ones.<p>I regularly use together.ai with models like this, where they're fully self hosting them and I can turn off all data retention, at a fraction of the cost of proprietary models. I recently discovered abliterated models (models edited to not refuse to work on security stuff) and hosts for them like refuseless.com, and that's an option for open source models, not for proprietary ones.
Models this size aren't targeting hobbyists running models in their bedrooms on gaming/surplus hardware bought with disposable income.<p>This model is for researchers who already have racks of equipment, inference providers who sell access to models, and for businesses who must (or choose to) use on-prem models. All of which have to pay to use the model if their revenue rises beyond certain thresholds: <a href="https://huggingface.co/moonshotai/Kimi-K3.1-Pro/blob/main/LICENSE" rel="nofollow">https://huggingface.co/moonshotai/Kimi-K3.1-Pro/blob/main/LI...</a>
You could easily run it with SSD streaming on as little as 192GB RAM (The dense portion is around 115GB or so at native precision, and the remainder would fit KV and a cache for the most commonly referenced experts.) This fits Gorgon Halo, which is a reasonably priced platform. It would be slow, but still usable for high-value tasks.
Possible data center bans incoming + tons of GPUs sitting in warehouses due to lack of power infrastructure to power them.<p>Perhaps a flood of 8x B300 will be homelab ready in a few months
> a flood of 8x B300 will be homelab ready in a few months<p>Haha, that's beyond wishful thinking, I'm afraid. We're not even seeing a flood of A100s (6 yo and several generations old). Every GPU is still being squeezed right now, and the trend will probably continue for the foreseeable future.
The world is bigger than the US. If the political climate in the US obstructs the build out, the build out will occur somewhere else.
No, they'll be shipped off to Asia where technology is embraced.
>Possible data center bans incoming<p>A lot of the data center build out has been bipartisan. Unless Dems realize they need to listen to their constituents to avoid a 2028 bloodbath (they probably won't), I wouldn't count on that to save us from RAM/GPU prices.
AND developer cloud has an image for K3 running it on a single MI300x. Offloading to system ram can be quite decent.<p>I cant say how well that image works as I'm still waiting for them to approve my gpu access
Do you need 450k context?<p>I usually keep under 200k if I can help it for Claude/ open ai<p>And lower locally on my Mac to save on prefill.