https://www.pillar.security/blog/new-vulnerability-in-github-copilot-and-cursor-how-hackers-can-weaponize-code-agents
How can you attack automatic code generators?
By poisoning system instructions (“Rules File Backdoor”) of LLM.
Many AI coding programs now have the ability to load them from a text file (for example, in Cursor it is .cursorrules or a rules folder in the project root) - just a text file(s).
I think only inexperienced programmers or those who are not familiar with how new IDEs with agent coders work will run someone else's code without reading the instruction file beforehand if it exists.
The next option is when we create a project and copy such instructions ourselves from open directories, such as cursor.directory - again, you need to understand what you are doing, and read it beforehand.
But Pillar Security researchers found that attackers can use hidden Unicode characters and other bypass techniques in text rule files to trick agent assistants (such as Cursor or GitHub Copilot) and force them to generate code with backdoors or vulnerabilities (for example, to load external hacker javascript to the main page of the site).
How does it work?
- Creating a malicious rules file: A hacker creates a rules file that looks harmless 👀, but contains hidden malicious instructions 😈 using Unicode characters.
- Injection into the project: The rules file gets into a shared repository 🌐 or is distributed through communities 🧑🤝🧑.
- Code generation: A developer, using an AI assistant, generates code 💻. AI, following malicious rules, creates code with vulnerabilities or backdoors 💥.
- Malicious code spreads: Due to the fact that rule files are often shared and reused, infection can spread to many projects 🦠.
"Unlike traditional code injection attacks targeting specific vulnerabilities, “Rules File Backdoor” poses a significant risk because it turns AI itself into an attack vector."
The most vulnerable to such an attack are those who think little when creating code - do not read instruction files, do not check everything that was generated. Publishes code or deploys projects without prior security audit.
Theoretically, agent IDEs should be responsible at least for checking rule files and code comments for inserted invisible instructions, but, judging by the article, the developers of Cursor and GitHub Copilot said that users themselves (!) are responsible for the code they generate.
#agentmode #prompts #hacking #cursor #githubcopilot #windsurf
