Small paper on the usefulness of AGENTS.md files: https://arxiv.org/abs/2602.11988
Findings boil down to
- It costs more to have them - makes perfect sense, since we are putting more into context
- Human curated versions of AGENTS.md outperform auto generated - again, makes sense. The AI can't always know what is useful, it is just summarising and guessing.
- The paper tested only solving tasks and results vary based on model, but seldom is there a big swing either way for it being good or bad - this is kinda surprising to me.
- Testing is improved by having AGENTS.md - makes sense, we're telling the agent what to do each time. AGENTS.md is augmenting our prompt
- Not in this paper, but a prior worked cited in this paper tested security and that saw improvements from AGENTS.md again - makes sense, it is augmented the prompt with the security things
Ultimately, my takeaway is that AGENTS.md is not magic (no one should be surprised by that) - all the AGENTS.md is doing is just augmenting your prompt with more info. More info costs money with AIs, and more info can help or hinder if it is good or bad info. Like with anything you use, know how the tool works to get the best results and do not blindly trust AIs (even /init is an AI)
Where do I land after reading this? Keep it - not because your individual LLM use is better or worse, but as a team you commit the AGENT.md to your repo and your teams experience with AI tools is more consistent.
Also, I am way too old and forget things, so having a AGENTS.md with things I should tell the agent... rather the individual prompt costs more than I have to run the same thing 3 times and pay a lot more in totality because I forgot something.