6 comments

  • amluto25 minutes ago
    I’ve been contemplating this situation. I think that what the author wants out of a Jev-like model is not at all what I want out of it.<p>&gt; I decided to check this on questions where the answer is well understood. For example:<p>&gt; A classical particle of mass m is embedded in a system at thermodynamic equilibrium with temperature T. What is its velocity v?<p>If I feed that into a model, the answer I want is: “the combination of the model and the provided state has nothing useful to add to your prior”.<p>If I want to know the Maxwell-Boltzmann distribution, I can look it up or I can derive it or I can ask a fancy LLM to do it for me (at the cost of some reasoning tokens and some time - unless I’m using an ultraspeed inference system, I’m not getting this answer in 50ms). [0]<p>Similarly, if I want to know that 73% of incoming customer support requests are spam&#x2F;fraud, I should measure that - it’s a property of my system, it takes some manual classification and a database query, and it will be a <i>different percentage</i> than your customer support system would see. I neither expect nor want my <i>classifier</i> to know this (unless I’m using a conventional classifier manually trained on my data, and the whole point of Jev is to avoid this).<p>What I <i>want</i> out of a system like Jev is to tell me how the probabilities <i>change</i> as a result of the per-sample data I provide. Which, is the case of this Boltzmann distribution question, is nothing: I provided no data and the classifier can infer nothing.<p>[0] A <i>really</i> good answer would observe that the answer depends on the dimension of the system (probably 3, but 2D systems are a thing) and also on whether the particles are hot enough for relativistic effects to matter (probably not). And maybe a good answer would check whether the material is a gas - the answer for a solid is not the same, but I suppose that’s not <i>classical</i>. Oh, and one shouldn’t forget drift: if you have a classical particle in a moving fluid or a classical charged particle in an electric field, you will again get a different answer.<p>Yes, I’m being pedantic. But if you want good answers you should be pedantic, and the Jev-like model is not where the pedantry should go.
  • dvt35 minutes ago
    &gt; So Jev does actually know which distributions are correct, it just fails to produce them<p>Claims like this needs to be deeply analyized. Models do have some emergent capabilities[1], and I think there&#x27;s a lot of evidence to show that semantics is actually <i>learned</i> (Word2Vec), and <i>some</i> math seems like it also might be learned (e.g. modular arithmetic). But it&#x27;s hard to exactly say where there&#x27;s some internal mechanism generating a true answer and where we&#x27;re just getting lucky with some distribution so the answer just <i>seems</i> right.<p>[1] <a href="https:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;2502.00873" rel="nofollow">https:&#x2F;&#x2F;arxiv.org&#x2F;pdf&#x2F;2502.00873</a>
  • cannedbread33 minutes ago
    IMO by asking Jev underspecified questions like this, you&#x27;re essentially using it as a random number generator (similar to the dice example). On actual NLP problems (including ones with uncertainty under human review) it does appear to be well calibrated: <a href="https:&#x2F;&#x2F;leonardgrazian.com&#x2F;blog&#x2F;jev-calibration&#x2F;" rel="nofollow">https:&#x2F;&#x2F;leonardgrazian.com&#x2F;blog&#x2F;jev-calibration&#x2F;</a>
  • singularity200139 minutes ago
    Also, it&#x27;s a very small and dumb model. That&#x27;s the only reason why it&#x27;s so fast.
  • popalchemist49 minutes ago
    not very
  • aaryan__verma4 hours ago
    [flagged]
    • dang52 minutes ago
      Can you please not post AI-generated or AI-edited comments to HN? It&#x27;s not allowed here - see <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;newsguidelines.html#generated">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;newsguidelines.html#generated</a> and <a href="https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=47340079">https:&#x2F;&#x2F;news.ycombinator.com&#x2F;item?id=47340079</a>.<p>Of course, it&#x27;s impossible to know for sure what was LLM processed or not, but this post got classified that way.
      • exe341 minute ago
        That&#x27;s interesting - unless the numbers are all made up, this didn&#x27;t read like slop to me. Somebody is claiming to have done this and that and concluded something. Maybe it&#x27;s your slopatron that&#x27;s miscalibrated! Have you considered jev :-D