Not sure, but if shared in the same docs as what we would read, it also allows us to skip over it with the assumption that it's a special set of instructions for AI.
I’ve noticed something similar with coding agents. Specific rules seem to work better than simply saying “this is for AI.” I wonder if files like AGENTS.md work differently because the tools are designed to read from them first.
Indeed. Another thing worth exploring is how much the position of instructions matter; Claude claims to read docs as a whole unless they're very large, though it might be that it only grabs the first lines of a doc sometimes, in which case there <i>is</i> an "above the fold" effect.
The experiment in TFA is saturated so there's no signal there, unfortunately.
Documentation is only useful for agents if the source is unavailable, and if you are trying to save on tokens and trust that the docs are accurate.<p>If none of that is relevant to you, or rather <i>accuracy</i> is your concern, it's much better just to search the source code for your answers, from how to set up the application to configuration questions to troubleshooting. Bonus points if you strip out all the comments programmatically via linting before you even let the agent view the code, to avoid possible context poisoning.
[flagged]