One trick I've found that works well is to tell it to refactor, e.g for Python:<p><pre><code> Refactor the Python code to make it more Pythonic, e.g. fewer classes/singletons, especially if it will provide a speedup. The Python code **MUST** follow code organization standards expected of popular open-source Python packages code without causing any benchmark performance regressions.
</code></pre>
A variant I've used for Rust code:<p><pre><code> The Rust codebase in `/src` has become bloated with several files >1k LoC. Refactor the Rust codebase to fit code organization standards expected of popular open-source Rust code without causing any benchmark performance regressions.
</code></pre>
Those types of prompts appear to a) reorganize the code logically and b) do seem to get better performance from the agents because the files now provide hints to where relevant code resides. For bloated 5k LoC files, the agent has to Read several chunks to find relevant code which is inefficient.<p>In terms of benchmark performance it generally <i>improves</i> after the refactor which I suspect is coincidental but I'm not complaining.
In my experience, the delta in agent performance is substantial if the codebase is littered with dead code, redundant code, unreachable fallbacks, leaking abstractions and half-baked design patterns vs if the code is well-organized, with clear data flow, with good encapsulation and clean architecture. Like, I've seen all the frontier models have to do several rounds of code review / QA and fix when the code is bad vs just getting it right at the 1st/2nd attempt.
I’ve been working with these things for quite some time now and every time I simply “treat it like I would a human” it seems to perform better. I can’t imagine agents wouldn’t perform better in a clean codebase than a giant mess of one. Just like it performs better when it has well formed specs and access to documentation.
Some of the issues mentioned above like dead code removal, code duplication, unreachable code are already solved using deterministic linters for quite a while now for most language ecosystems.<p>You can get the LLM to run a script which checks for all of these and also enforce them by running the same script as a pre-commit hook. Setting this up religiously in every code base I work on has been what's given me the most mileage with agentic coding.<p>I wrote down a more detailed post of the various linters I use here:<p><a href="https://www.balajeerc.info/Use-Deterministic-Guardrails-for-your-LLM-Agents/" rel="nofollow">https://www.balajeerc.info/Use-Deterministic-Guardrails-for-...</a>
Feel the same way myself when working in messy codebases… At some point, the horrible patterns start to rub off…
I was reading your comment, agreeing with it but still feeling why this is a bad comment. It just occurred to me that an anecdotal statement like this is the antithesis of scientific discourse. We have a paper here, trying to answer a question, and anecdotal testimonials can only harm the discussion by biasing readers without adding anything of value to let anyone objectively conclude anything on the problem.<p>The most useful discussion would be if we all read the paper and critique its methodology or results.
i mean this is <i>feeling</i> too but im too paranoid and frequently do refactoring and code organization passes and never don't do it, so i cant say i know for sure there is a delta.<p>though people who complain that llms aren't that great strike me as the type to have messy code bases
I can't imagine how it wouldn't. None of them can fit a real codebase in context and have to browse the code the same way a person would. Doing searches and reading files. If the files are in the places they would be expected to be and things are called what the model or a person would first guess to search then it gets found in the first attempt rather than requiring a deep search and multiple attempts.
Even if agents can learn to navigate all the stubs and WET crap they leave behind do we really want a code base that no human can follow what's actually going on?
"agent pipelines that [...] clean a messy [repository]"<p>This feels like a terrible approach, sufficient to condemn the entire study.<p>Apparently half of the "minimal pairs" in this work were constructed in this way. I simply am not going to trust any conclusion that requires assuming these AI "cleaned" repos are in any way representative of actually-good codebases.
agents are pretty good at cleaning up a codebase, finding dead code, fixing bad abstractions, etc. You just have to spend some focusing the agents on that goal.
Another consideration: written by hand, the trade-off of development velocity vs well organised code means that it can be worth taking some tech debt now in order to deliver some value now. (Especially when prototyping etc.).<p>With coding agents, agents can produce code quicker. The same trade-off still applies.. but, the time it takes an LLM coding agent to write well organised code is still going to be quicker than the time it takes me to write scrappy code.
Yes this is one reason IMO I think of AI code as instant legacy code.<p>You take on a lot of tech debt. Then you need to do the same work you would do with any legacy app: finding where the brittle points are, what needs better testing, which leads to breaking apart the big ball of mud into cleaner components.
[flagged]
"Across 660 trials with Claude Code, code cleanliness does not change the agent's pass rate. However, it substantially alters the agent's operational footprint: agents working on cleaner code use 7 to 8% fewer tokens and reduce file revisitations by 34%. Our findings suggest that traditional maintainability principles remain highly relevant in the era of AI-driven development [...]"