Apple Neural Engine: Architecture, Programming, and Performance

(arxiv.org)

83 points by Jimmc4141 day ago

3 comments

g42gregory21 minutes ago
It does not seem to cover the Neural Accelerators, Apple's equivalent of the Tensor Cores. They only got released on M5 platform. This is probably the most important part to cover.
throwa35626241 minutes ago
Is there a non-slop version of this information available?I am reading up on GPU / ML micro architecture and am looking for some good sources.
carbocation2 hours ago
This scans very much as AI-written.
- thx671 hour ago
 This is obvious Claude slop writing, the author would be advised to use vale [1] with samples of their own writing as a guide.> Performance begins with the roofline. On the M1 the engine holds about 12 fp16 TFLOP/s of compute against a DRAM-bandwidth ceiling. The roofline has a ridge point near 141 FLOP per byte, a 2 MB working-set threshold, a 0.23 ms floor under any single dispatch, and efficiency near 0.37 picojoules per FLOP at the compute optimum. On a 256-channel 3x3 convolution it runs about 3.8 times faster than the same chip’s GPU and 9 times more energy-efficient. The roofline pairs the engine’s throughput ceilings with its measured power.> Reaching the engine is not the same as running an arbitrary graph on it. The operations the engine executes are distinct from the ones a capability bit only advertises. A feature attested in the hardware tables or accepted by the compiler frontend counts only once a compile-and-run confirms it, and several advertised operations, three-dimensional convolution among them, never lower to the engine at all. Weight compression on the direct path cuts bandwidth, not only stored size. On the unentitled engine, int4 lookup-table weights run about 2.37 times faster than fp16, and structured sparsity 1.55 to 1.64 times faster at 0.43 times the bytes.<a href="https://vale.sh/" rel="nofollow">https://vale.sh/</a>
- dkdcdev1 hour ago
 why?
 - labcomputer49 minutes ago
 1. It uses non-idiomatic terminology in several places.2. It repeats the same finding over and over (141 flops per byte, for example), without going deeper.3. I stopped reading about a quarter of the way through because it felt like it was never going to stop teasing me about what it was going to tell me and actually tell me it.4. It seems to assume the reader has a lot of context that isn't explicitly laid out (and which the reader wouldn't get just from reading the prior work, which is cited).For example, I understand some of what it is saying because I used some similar techniques to benchmark things in the past (running at multiple scales to estimate overhead + marginal gains with a linear regression), but I wouldn't expect anyone who hasn't personally done that to follow the prose.
 - dylan60430 minutes ago
 > 4. It seems to assume the reader has a lot of context that isn't explicitly laid out (and which the reader wouldn't get just from reading the prior work, which is citedI've had this complaint well before LLMs were used. People writing about topics they have a lot of knowledge in the subject tend to make the assumption only other subject knowledgeable readers will read it. Or that it never edited by a real editor that would enforce rules like spelling out acronyms on first use. Or forcing additional information when too many details have been left out on the assumption it would already be known.There's plenty of this type of writing to have trained the bots that way
 - hbn1 hour ago
 Cmd-F for "AI" has 1000+ hits!
 - thomspoon1 hour ago
 The burden of proof should be with the beholder. Must be so easy to scream AI when you don’t want to read an article.
 - thx671 hour ago
 You obviously haven't read it, because it is clunky garbage.> 19.4 Pacing compiles after a failure> A failed compile is not free of side effects on the shared compile service. A compile that fails restarts the service, which takes a few seconds to come back, and failures that keep arriving faster than the service can restart between them keep it from making progress, so unrelated compiles slow down until the failures stop. The effect is a function of how fast failures arrive, not how many occur: failures spaced out past the restart interval cause no degradation at all. On detecting a failed compile, wait at least one restart interval, roughly 15 seconds, before the next compile, so a burst of failures cannot accumulate. No hard failure-count cap is needed.The whole document is less nutritious than a wonderbread miracle whip sandwich.