3 comments
Isn't this turning a GPU into a slower CPU? It's not like CPUs are slow, in fact they're quite a bit faster than any single GPU thread. If code is written in a GPU unaware way it's not going to take advantage of the reasons for being on the GPU in the first place.
I don’t understand why this is a useful effort. It seems like a solution in source of a problem. It’s going to be incredibly easy to end up with hopelessly inefficient programs that need a full redesign in a normal gpu programming model to be useful.
Founder here.<p>1. Programming GPUs is a problem. The ratio of CPUs to CPU programmers and GPUs to GPU programmers is massively out of whack. Not because GPU programming is less valuable or lucrative, because GPUs are weird and the tools are weird.<p>2. We are more interested in leveraging existing libraries than running existing binaries wholesale (mostly within a warp). But, running GPU-unaware code leaves a lot of space for the compiler to move stuff around and optimize things.<p>3. The compiler changes are not our product, the GPU apps we are building with them are. So it is in our interest to make the apps very fast.<p>Anyway, skepticism is understandable and we are well aware code wins arguments.
Do you foresee this being faster than SIMD for things like cosine similarity? Apologies if I missed that context somewhere.
It depends. At VecorWare are a bit of an extreme case in that we are inverting the relationship and making the GPU the main loop that calls out to the CPU sparingly. So in that model, yes. If your code is run in a more traditional model (CPU driving and using GPU as a coprocessor), probably not. Going across the bus dominates most workloads. That being said, the traditional wisdom is becoming less relevant as integrated memory is popping up everywhere and tech like GPUDirect exists with the right datacenter hardware.<p>These are the details we intend to insulate people from so they can just write code and have it run fast. There is a reason why abstractions were invented on the CPU and we think we are at that point for the GPU.<p>(for the datacenter folks I know hardware topology has a HUGE impact that software cannot overcome on its own in many situations)
<i>> because GPUs are weird and the tools are weird.</i><p>Why is it also that terminology is so all over the place. Subgroups, wavefronts, warps etc. referring to the same concept. That doesn't help it.
All the names for waves come from different hardware and software vendors adopting names for the same or similar concept.<p>- Wavefront: AMD, comes from their hardware naming<p>- Warp: Nvidia, comes from their hardware naming for largely the same concept<p>Both of these were implementation detail until Microsoft and Khronos enshrined them in the shader programming model independent of the hardware implementation so you get<p>- Subgroup: Khronos' name for the abstract model that maps to the hardware<p>- Wave: Microsoft's name for the same<p>They all describe mostly the same thing so they all get used and you get the naming mess. Doesn't help that you'll have the API spec use wave/subgroup, but the vendor profilers will use warp/wavefront in the names of their hardware counters.
I think they've taken the integration difficulty into account.<p>Besides, full redesign isn't so expensive these days (depending).<p>><i>It seems like a solution in source of a problem.</i><p>Agreed, but it'll be interesting to see how it plays out.
Is this proprietary, or something I can play around with? I can't find a repo.