Show HN: Reame – a CPU inference server that gets faster as it runs

7 comments

• JPLeRouzic3 hours ago
  Thanks for sharing your work!<p>Please how to select the model? I downloaded tinyLlama, put it in .&#x2F;models, changed reame.conf but I get:<p>(No such file or directory)<p>Otherwise putting the model in &#x2F;opt does not please me much, I fear to forget a model is there, if it is in reame folder its much easier to notice and manage.
• K0IN2 hours ago
  Sick that you can get 2 arm cores and 12 GB ram for free at Oracle cloud, did not know that
  • GTP2 hours ago
    You can actually get more. You have an amount of resources for ARM servers, if you allocate all of that to a single one you ger 24GB RAM and 4 vCPU.
  • esseph2 hours ago
    Yeah but it&#x27;s Oracle
  • CryptoBanker2 hours ago
    If you have good luck perhaps. That tier of server is virtually always sold out
• ohadkr5 hours ago
  The persistent KV cache is interesting; I’d love to see how much of the speedup remains when requests share less structure.
• tyzoid4 hours ago
  Why qwen 2.5 everywhere? Why not 3.5?
  • TheGoddessInari2 hours ago
    Because llms reflect qwen 2.5 heavily in training data, and that&#x27;s who is giving the recommendations.
• gaurang4042 hours ago
  looks really interesting
• UncleOxidant24 minutes ago
  [dead]