Ha, interesting. I wasn't aware of Sutton's blog post, but if I might make a shameless plug, we demonstrated [1] exactly this problem (see section 4.4.3), and how multi-step world models (using diffusion models as the substrate) could be one potential answer.<p>Since then, I have come to like temporally-abstract models more and more. Rolling out in time -- either step-by-step or many steps at once -- suffers from the <i>tyranny of the specific</i>. For long horizon planning with agents, I care (often only approximately) about where I can end up, and seldom about <i>exactly when</i> I end up there. Successor features, GVFs, Forward-Backward representations, and the like seem like they have an elegant approach for structuring thinking at a "high level", instead of generating exponentially large search trees by rolling out microscopic world models.<p>[1] <a href="https://arxiv.org/abs/2410.05364" rel="nofollow">https://arxiv.org/abs/2410.05364</a> (funnily, from around the same time / few months after Sutton's blog post)