Show HN: Shoehorn – Quantize any model down to run on your machine

(notactuallytreyanastasio.github.io)

16 points by rhgraysonii2 hours ago

2 comments

  • hmokiguess57 minutes ago
    Reminds me of <a href="https:&#x2F;&#x2F;github.com&#x2F;AlexsJones&#x2F;llmfit" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;AlexsJones&#x2F;llmfit</a>
  • mbuchel-hn1 hour ago
    does this work similar to airllm? i am wondering how it would handle something like quantizing kimi k3 on a budget of 8 gbs, or is that something you are not attempting to solve yet?