I don’t think it would be surprising that people want to write their own kernels.<p>A big problem with the existing engines like llama or sd is that they don’t support optimal graph compilation. Usually this means about a real 2 or 3x multiplier loss relative to optimal. Cuda graphs do okay but they still leave a lot on the floor<p>It’s usually worth it to optimize in that context if you are willing to peer into the mechanics.<p>Of course that’s expensive. You need to know how to appropriately pipeline and merge your kernels.
Should have started with writing your own blog posts.
> Should have started with writing your own blog posts.<p>While the page looks vibe-coded[1], the content itself does not have any AI tells. What are the tells you are seeing?<p>[1] Too many sites I find on HN frontpage these days slow my PC to a crawl. I assume they are all using the same autogenerated HTML, Javascrip and CSS to make animated backgrounds :-( On this specific site scrolling is laggy.
I stopped reading almost immediately. The stylistic choices in the writing just felt like LLM to me. Examples:<p>"depth estimation that beats PyTorch on CPU in half the memory" — "…beats X in Y…"<p>"Most LocalAI backends wrap somebody
else’s engine, and that is the right default." — "…and that is the right"<p>"MLX and the rest are maintained by people who are better at those models than we are" — "better at those models than we are" — it's this thing that LLMs do where they are kind of weirdly confident but overly deferential<p>"This post is about what those ports buy" — "…buy" used in this context<p>"Same model, 1.31x the speed" — "Same X, something Y" — it's this overconfident yet deferential writing style<p>The further I read, the more tells there are. I find it incredibly tiring to read LLM generated prose and I'm not sure why. Is it because I'm aware it's not human written and have an unconscious bias? Or is it because the style is just full-on, "Not X but Y. Those performance gains are bought, not earned. This stops, that starts. Read on, or don't, that's the follow-up"
For me, it's that AI writing is always trying to be clever for every single point it makes (and constantly uses the same language patterns when doing so).<p>Its like listening to an insufferable clever dick, who is not as bright as they think they are. You would also find it incredibly irritating if a human talked like that
Now that you point it out, there are quite a few tells, still not as many as most of the slop that gets posted here.<p>I think it's because of the laggy scrolling that I didn't read the whole thing anyway, just the first few screens.
Also, “honest reading” — without any context explaining why one would contemplate a dishonest reading.
[dead]
Why are you using an em-dash though?
[flagged]
I found the post insightful and interesting. I'm not sure it was written with AI assistance, but even if it was, I don't see that as a reason to dismiss it. For what it's worth, I spend hours everyday reading AI output and summaries.
Same could be said for all that talk about having Claude do their work.
I've had this debate before on HN. The excuses are generally "well it can write better code than most developers, but an LLM can't write better prose than most people" (I strongly disagree with this) and, what I think is at the heart of the matter, "text is for the reader to read directly, code is hidden." Or in other words, "as long as I can't tell it's AI, it's fine."
Came here to say the same. Really tiring to read these slop-infested posts, where everything has the “right shape”.
The thing is... although the writing is unmistakably full of LLMisms, I can't fault the `author` for having produced a slop readme.
The content earns its keep, it only grates because of the robotic personality. We need another word than "slop" for this.<p>"blland", "llame",... ?
I went through the post because of your comment but it really doesn't look like AI slop. Can you please share why you feel like its slop and not written by a human? I can also say "should have started writing your own comments" to you and its unfalsifiable. Blanket accusations with no proof is not a good move really.
The post is full of signs. Here's only a couple of examples:<p>> The method, the measurements, and what it costs us.<p>> That is the general shape of these wins.<p>> Parity is the gate, speed is the follow-up<p>I could go on and on, but you probably get the point. If you don't find anything funny with the above, you might have not been enough-exposed to slop.
What do you mean you could go on and on? Why do you think those sentences are AI written.<p>And okay, your second argument is that I just don't know slop because I am not exposed to it? But you don't know anything about me or what I am exposed.<p>You're just making random claims and stating they are correct without any evidence or arguments.
What kind of evidence do you expect beyond "random claims" here?<p>This post is incredibly obviously AI generated, to the extent that I doubt a human author edited it at all. Not "written with AI assistance" but full on "give Claude some bullets and hit publish." It contains tons of tropes that show up in all AI writing and which people are highlighting here.<p>What would convince you of that?
<a href="https://www.pangram.com/history/125358dc-9f80-4fe2-8540-85a9c314aa6c" rel="nofollow">https://www.pangram.com/history/125358dc-9f80-4fe2-8540-85a9...</a>
They follow the tropes I get when I ask AI to do docs or summaries. Very Opus style, this one.
It's always hyping up something and throwing punch lines in every sentence. Normies love this shit.
I had a similar success with Model2Vec static embedder and NER inference (both GGUF, compiled for WASM), ported to plain C from ONNX Runtime.<p>Wasm size from 30Mb to 300kb and 1.5x speedup. It's definitely worth it for performance or distribution size.
I did took a native c++ approach when writing a relational transformers engine (RelativeDB). My journey was pytorch -> c++ -> Triton (lang). While C++ was more performant than Triton, I couldn't afford to optimize on every gpu. I just accepted the ~15% throughput loss for my cloud service, which honestly wasn't bad for the amount of flexibility I got out of it.<p>But the cpp port of vllm looks great, that'd be great if you'll maintain that. I hit the same limitations with vllm.
Could this vllm port be faster to install? Im starting gpu machine multiple times a day and it takes 5 minutes to set vllm up. If Inise this port that time is minimized?
What you get: X is the A, Y is the B.
[flagged]
[dead]