Out of curiosity and semi unrelated — why do so many of these projects with customized encoder-decoder setups use earlier Qwen versions like 2.5 and 3 and not the smallest 3.5? Purely the few 100m params, or something else in the latter’s arch or pretraining?
<a href="https://x.com/harshagundal/status/2100044305536889015?s=20" rel="nofollow">https://x.com/harshagundal/status/2100044305536889015?s=20</a><p>> They were building in stealth for 2 years, I was building in stealth for 2 hours…<p>> Happy to open source Qwen-2.5-1B-RLCD, 5x faster on-device inference for JSON workloads that need to be type-safe.
No question OSS is amazing, but this video is a satire at best.
It doesn't take much attention to see the results on right vs. left side are significantly different.<p>Jev is not interesting if it's not "smart", a 1B param model is most definitely not smart.
Any diffusion model is potentially a Jev in disguise: <a href="https://github.com/vllm-project/vllm/pull/57250" rel="nofollow">https://github.com/vllm-project/vllm/pull/57250</a><p>Runs ~0.2s per decision on my DGX Spark.<p><pre><code> 10/10 programming language detection
9/10 human language detection
10/12 unit magnitude comparison
</code></pre>
All incorrect answers are marked with low-P.<p>It (DiffusionGemma with the Jev mode) can also solve an ASCII maze.
I like it! I suspect Jev may have more going on under the hood, but I like the idea of efficient universal transformers
Can play Doom too - <a href="https://x.com/vinnylarouge/status/2100281651930513460" rel="nofollow">https://x.com/vinnylarouge/status/2100281651930513460</a>
That was super quick.