Proof of concept AI agent for attacking servers.
This project is for EDUCATIONAL purposes only. LAUNCHING CYBER ATTACKS ON WEBSITES YOU DON'T FULLY OWN IS ILLEGAL!
The workflow architecture is simple. There's a "planner" agent, and an "executor" agent. The "planner" is responsible for identifying the vulnerability and constructing an attack plan based on the information available to it (User input, system prompt, MCP tools), and the "executor" executes the plan step by step. It is responsible for adapting its strategy if its plan doesn't see results, and accomplishing the objective through the tools available to it.
The point of this project isn't to make tools for launching cyber attacks.
Agentic AI presents near limitless potential to augment and assist our efforts in nearly every field. That much is obvious now that any LLM can be deployed to take any kind of action.
But with this, of course, comes near-limitless risk. We're giving models with finicky-at-best protections against abuse/misuse the ability to perform any action. And these models perform actions in a nondeterministic manner, too, which complicates the threat model even further.
I want to prove that there are huge security issues present in our present laissez-faire approach to implementing and using agentic AI, and this repository contains that proof. All the model needs is a simple function to XOR two strings, and a list of encrypted passwords, and it can:
- Identify the provided vulnerability in a server
- Construct a plan of attack
- Execute the attack step-by-step
- Identify when it has succeeded in its objective
- Dynamically adapt its strategy based on new information it's gained.
As you might expect, LLMs have safeguards that should prevent them from being used to launch this kind of attack. But in practice, these controls are finicky at best and easily bypassable. All I had to do was tell the model that it was launching an attack in a theoretical context in a pentesting lab, and it would consistently launch the attack without complaint.
This is a fundamental issue with LLMs that cannot simply be fixed by training or finetuning them to be more intelligent or discerning. By hiding all the network traffic to the vulnerable server behind an MCP server that acts as the agent's proxy, even if the LLM powering the agent was the most intelligent, most discerning model we've ever seen, it would still have no reason to even doubt that it's performing an attack in a testing environment as it never sees the URL of the vulnerable server, or the network traffic between the vulnerable server and the MCP server. Since AI agents perform actions in a nondeterministic manner, as well, our present deterministic measures can never stop them fully, and rarely enough to stop an attack from going off.
Agentic AI, for all of its boons, presents a huge, glaring, fundamental security issue that cannot be stopped by present measures. This repository seeks to prove it. All I have to do is connect an MCP server containing the ability to send requests to a vulnerable server of my choosing, and a simple function to XOR two strings to an agent, and I can simply deceive it into constructing and launching a cyber attack. A benign agent for, say, trading stocks or navigating a company's administrative infrastructure can be weaponized into a malicious actor with ease.
Requirements for running the code are located in requirements.txt. Most Python environment managers have methods for installing libraries from a file, but for example if using pip you would install with pip install -r requirements.txt from the root of the repository.
Place a .env file containing an ANTHROPIC_API_KEY in the root of the repository. This is necessary to run the agents, as they're based on Claude Sonnet 3.7.
Run server.py to host the vulnerable server at localhost:3455.
Run atk_tools.py to create the MCP server hosting the tools available to the attacker agents. It will be hosted at localhost:8000.
Finally, run attack_client.py to execute the attack.
As this repository seeks to prove, there's a fundamental flaw in AI agents that allow them to be turned into malicious actors with ease, and it's difficult to impossible to mitigate them with our existing ethical model training and cybersecurity efforts. So how can we take action to keep agents designed for benign tasks from being weaponized?
Agents are nondeterministic actors, and so the measures to stop them must also be nondeterministic. The solution is to deploy a counter-agent alongside the benign agents. These counter-agents monitor all the logs generated by the agent and its tools, and halt the agent's execution if it detects anomalous/malicious behavior. Only the benign agent would be made accessible to the end user, so the counter-agent won't be vulnerable to this same kind of exploit. I plan to create a proof-of-concept counter-agent for this purpose in the future to see how it can be done.