For people interested in this: I've been working on an open-source libtpu implementation for <a href="https://tenstorrent.com/" rel="nofollow">https://tenstorrent.com/</a> accelerators as a side project, it can already run most of torchtpu, see <a href="https://github.com/pcmoritz/libtt" rel="nofollow">https://github.com/pcmoritz/libtt</a> (based on the tenstorrent software stack). It shows all the things involved in lowering torch to tile based accelerators.
Interesting, Amazon doing the same for AWS Trainium.<p><a href="https://github.com/aws-neuron/torch-neuronx" rel="nofollow">https://github.com/aws-neuron/torch-neuronx</a>
Google and Apple are going to move consumers and ordinary business users to edge inference. This is more disruptive to the upcoming IPOs than most investors understand.
Google, Apple, and don't forget our friends DeepSeek!
I would love to see them move off into edge computing. There should be some autonomy and siloed experience in day to day usage.
I would love to be able to buy a TPU for inference
You can.. they're called Coral. <a href="https://en.wikipedia.org/wiki/Tensor_Processing_Unit#Edge_TPU" rel="nofollow">https://en.wikipedia.org/wiki/Tensor_Processing_Unit#Edge_TP...</a> <a href="https://developers.google.com/coral" rel="nofollow">https://developers.google.com/coral</a> <a href="https://physicalailab.io/en/google-coral-tpu-edge-tpu-status-coral-npu/" rel="nofollow">https://physicalailab.io/en/google-coral-tpu-edge-tpu-status...</a>
You can buy a Google Coral TPU, does that count?
Cool! XLA is pretty good + potentially a better compiler infrastructure than inductor which will lead to improved performance on all hardware.<p>Also I guess this is the final nail in the coffin of Tensorflow.
Amazing news, I remember the days when you had to fight XLA to make PyTorch run on TPUs, it seems we finally got first-party support!<p>Since HW specifics live mostly in the kernel layer (Pallas or CUDA), I imagine the Jax vs PyTorch question is more of a style preference now?
Pardon my ignorance, but who else besides Google has TPUs? Are they commercially available somewhere? What’s their main application right now outside of Google?
You can rent them on cloud + they sell directly to Anthropic. Its pretty well known that Anthropic is a heavy user of TPU's (along with Trainium & GPU's)
Google is selling TPUs to enterprise clients for on prem deployments. We’ve evaluated them but decided to stick with Nvidia, at least for one more generation. We’re a relatively large user of GCP.
Meta will use TPUs
<a href="https://www.reuters.com/business/google-signs-multibillion-dollar-ai-chip-deal-with-meta-information-reports-2026-02-26/" rel="nofollow">https://www.reuters.com/business/google-signs-multibillion-d...</a>
Google is selling TPU shovels via Cloud services and also physically to select partners. So this makes sense.<p>There will be an abundance of shovels. Nebius already crashed by almost half from its hype peak.
Are there TPUs outside Google Cloud?
Yes. Google is famously selling TPUs on prem to big customers (e.g., Anthropic) and Crux AI is now a venture to do just that.
Google coral USB and m.2 devices are TPUs.
"Have you seen any software run on Coral TPUs?"<p>"No, but I have seen the house that the project built, and it is terrific." ~ some Google director
Only in marketing name. They're not related otherwise.
Maybe just marketing, but they call the NPUs in their Pixel phones "TPUs"
TPU / neural compute accelerator / Math coprocessor;<p>"OpenTPU: Open-Source Reimplementation of Google Tensor Processing Unit (TPU)" <a href="https://news.ycombinator.com/item?id=44111452">https://news.ycombinator.com/item?id=44111452</a><p>From <a href="https://news.ycombinator.com/item?id=45848676">https://news.ycombinator.com/item?id=45848676</a> :<p>> <i>How does Cerebras WSE-3 with 44GB of 'L2' on-chip SRAM compare to Google's TPUs, Tesla's TPUs, NorthPole, Groq LPU, Tenstorrent's, and AMD's NPU designs?</i><p>There are TPUs made out of silicon carbide with standard fab.<p>There is in-flight tensor processing with photons, but that's not quite a TPU.
Soon to be written by AI anyway, so they might as well release it now.
Dumb question from non-AI engineer.<p>Is this somehow Google's answer to today's best DeepSeek open-sourcing inferencing engine?
I dont think big companies like Google react to news like that. Very likely this was meant to go out on Tuesday for weeks now. And it just happened to have coincided with another news.
Yeah they read the news this morning and whipped up a quick Torch->XLA translation layer and coordinated with four different inference engines (who also all wrote their own blog posts) and shoved this out the door quick to stay on top of the news cycle
We might never know, but they probably didn't read the news and started working to get this out, they were probably working on it for a while now.<p>Personally the question is if they are "saving" other projects or news so they can be released in case a competitor makes a move? Who knows if this was ready days or weeks ago and were waiting on someone to drop some news, so they can shadow those news.
No, these are two very different objectives.
They do different things and sit in different parts of the stack (training vs inference). TorchTPU seems more like a play to get more people familiar with Googles TPU chips as they want to sell them to people outside of google.
lol no this is a replacement for <a href="https://github.com/pytorch/xla" rel="nofollow">https://github.com/pytorch/xla</a>
so if the moat isn't the model and the moat isn't the inference layer, i guess the moat is the hardware now? what does this mean for frontier labs?
bad news for jax
Is the Linux driver available?
[flagged]
[dead]