Shocking to me how little interest the general public has in mechinterp given the alien capabilities demonstrated by LLMs - this and the subsequent transformer-circuits.pub publications will be seen as classic, foundational work in a few years
Nothing about B-H curves or magnetization current, must be that other kind of transformer, in that other kind of circuit.
> A previous project, the Distill Circuits thread, has attempted to reverse engineer vision models, but so far there hasn’t been a comparable project for transformers or language models.<p>How successful was that?
I’ve tried reading this many times. But it’s sooooo long. Is it worth it?
Neal Nanda's walkthrough of it is pretty good.<p><a href="https://youtu.be/KV5gbOmHbjU?is=GSiv0rSHBocNdU8l" rel="nofollow">https://youtu.be/KV5gbOmHbjU?is=GSiv0rSHBocNdU8l</a><p>Also long, but it's detailed and complicated, so there is no escaping that.
Depending on your field, degree of interest, general curiosity, its a must. Its up there with word2vec.-