This is interesting and all, but “LoRA” is painfully close to “LoRa” (which is related to radio networking, not AI) when just scanning a list of topics. We’re never going to beat the Shannon limit on acronyms and initialisms.<p>I’m glad the rest of the anchor text gave some context.
You can try out TinyLoRA in PEFT main now: <a href="https://huggingface.co/docs/peft/main/en/package_reference/tinylora" rel="nofollow">https://huggingface.co/docs/peft/main/en/package_reference/t...</a>
Not sure if I buy it. First, SVD decomposition to obtain U, Σ, V is computationally expensive, so it would work only if we are not finetuning very big models.<p>But my real concern comes at the results. The "13 parameters" looks like bait, because it is one result of finetuning a model on a very simple math benchmark, grade-school-math (GSM8K), an already very saturated benchmark on every model. Besides, it seems to happen only for the qwen family model... It looks like GSM8K was part of the training set of the qwen model, and this tinylora finetuning did the last adjustments to perfectly reflect that overtraining.
Fair points, especially on GSM8K saturation and Qwen possibly already sitting close to the solution. That said, even if this is mostly "last-mile alignment", the fact that it can be done with such a tiny signal is still interesting, it suggests the gap between capability and behavior might be much smaller (and cheaper to bridge) than we assume.
They're using the truncated SVD, not the full variant, that's computationally cheaper.
Yeah, my big problem with the paper is it just might be an artifact of qwen's training process.
the '13 parameters' framing is misleading in both directions. the base model's billions of parameters do the heavy lifting; the 13 just steer existing circuitry. but that's actually the interesting finding: the behavior gap between a capable model and a reasoning model is geometrically tiny. you can find it with gradient descent in 13 dimensions of a multi-billion-dimensional space. that's a strong claim about the structure of task-relevant representations in transformers.
Is it an Aprils Fools publication?
>One theory is that the knowledge required to solve the task is already stored in the parameters of the model, and only the style has to change for task success<p>>In particular, learning to generate longer outputs may be possible in few parameters<p>Reminded me of: <a href="https://arxiv.org/abs/2501.19393" rel="nofollow">https://arxiv.org/abs/2501.19393</a><p>>we develop budget forcing to control test-time compute by forcefully terminating the model’s thinking process or lengthening it by appending “Wait” multiple times to the model’s generation when it tries to end. This can lead the model to double-check its answer, often fixing incorrect reasoning steps<p>Maybe, indeed, the model simply learns to insert the EOS token (or similar) later, and the capability is already in the base model
If i understand it correctly, the analogy could be:<p>Let's say we have a low level programmer expert and we try to teach him algebra either we:<p><pre><code> - (SFT): give him algebra book with new nomenclature, definitions, syntax
- (RL): let him learn algebra using C syntax</code></pre>
I don't think so.<p>Fine tuning works on an input/output basis. You are rewarded for producing a plausible output _now_.<p>RL rewards you later for producing the right output now. So you have to learn to generate a lot of activity but you are only rewarded if you end up at the right place.<p>In SFT you are rewarded for generating tokens plausible to the proof, one token at a time. In RL you are expected to generate an entire proof and then you are rewarded or punished only when the proof is done.
Most data in the training set of most reasoning models is crap I guess.
The quality of custom models trained with proper reasoning datasets[0] even with small parameters (3-7B is sweet spot) is incredible now<p>[0]: cartesien.io or Salesforce's WebscaleRL
With four parameters I can fit an elephant, and with five I can make him wiggle his trunk so there is still room for improvement.
[dead]
[dead]
[dead]
[dead]
Such low dimensionality of the LoRA vector must surely result in a close-to-linear modification to the KV calculation. This seems to me to imply that what we call "reasoning" is latent within the model. Pretty clear I didn't read the paper, I'm sure the authors address this.
If 13 parameters can unlock better reasoning, then we will not be "training" models, we'll be steering them. Most of the capability is already there.<p>The real unlock isn’t TinyLoRA, it’s what this implies: ultra-cheap, continuous adaptation. The bottleneck shifts from compute to having a good reward signal.