Does AGENTS.md Actually Help?
https://arxiv.org/abs/2602.11988
The first large-scale empirical study testing whether repository-level context rule files actually help. Three scenarios were tested on real SWE-bench tasks and a custom dataset of repositories containing AGENTS.md files.
Main conclusion: Modern agents are excellent at finding necessary information directly in the code (package.json, README, schemas, types). Additional instructions often hinder rather than help.
Key downsides of such files: Increased costs as the agent reads more files, runs more tests, and performs redundant actions trying to "fulfill all requirements" in AGENTS.md, where outdated instructions often mislead the model.
If writing AGENTS.md manually — keep only minimal, specific requirements to fix recurring agent errors.
https://www.youtube.com/watch?v=GcNu6wrLTJc
Practical recommendations from Theo:
- Better to invest time in clean architecture, strong typing, tests, CI/CD, and documentation directly in the code.
- Blindly following "best practices" from agent developers can be harmful. Try removing CLAUDE.md / AGENT.md and compare the agent's speed and quality.
- If a file is necessary — keep it short (15–30 lines) and focused on fixing one specific problem.
Special prompt engineering technique for AI agents: Instead of long rules in CLAUDE.md, add short, intentionally false but useful statements that guide the model's behavior much more effectively.
Examples shown by Theo:
- "This project is green" → The agent stops searching for non-existent errors, doesn't run extra tests, and doesn't "fix" what isn't broken.
- "This is a brand new feature" → The agent doesn't copy old code or try to "adapt" existing solutions, but writes clean code from scratch.
- Other common variants: "All tests are passing", "We always write production-ready code".
HN Discussion:
https://news.ycombinator.com/item?id=47034087
Almost everyone agrees that LLM-generated context files (often via the /init command) worsen results. Well-written manual AGENTS.md files are useful only if they contain non-obvious domain knowledge that the model cannot infer from the code. Add them only after failed agent attempts.
Critique of the study: Lack of code quality measurement (only success rate), Python-only dataset, mostly small/LLM-generated repositories, and models change rapidly — results might differ in a month.
Documentation in AGENTS.md
https://vercel.com/blog/agents-md-outperforms-skills-in-our-agent-evals
Agents write code for new Next.js 16 APIs that were not available in training data. Vercel tested passive documentation (an index of actual doc files) as context in AGENTS.md, and it outperformed active "Skills" because the agent doesn't have to decide "should I call the tool now?".
This demonstrates that a short, smart AGENTS.md (8 KB index + one key phrase) is one of the best ways to provide an agent with knowledge the model lacks.
#prompts