6 comments
As a tangent, there has recently been <i>drama</i> with AI and Pokemon Showdown because someone trained a bot (using standard reinforcement learning + a rewrite of the Pokemon battle engine to Rust for max self-play parallelism) that is apparently good enough to get to the top of the leaderboard.
Did you try out different reasoning levels? Wondering if messaging is basically a proxy for additional reasoning
what was the selection for the teams? random per round? how does it play against people in ladder?
Pokémon Showdown isn't a game so much as it is competitive Wiki reading.
Jev will give better results on this kind of thing.
For sure there are many better models for this use case[0]<p>At the same time, it’s interesting seeing how generally capable language models/LLMs/agents/harnesses are getting<p>We’ll probably see a lot more experiments and benchmarks like these in the near future<p>0: simple classifiers can play quite a few game, including Doom <a href="https://playground.jeffyclassify.com/#doom" rel="nofollow">https://playground.jeffyclassify.com/#doom</a>
Yeah Jev can play it but Jev cant communicate with its teammate in the 2v2 + it cant reason and is just a decision model