AI Agent Security

Introduction

A lot is being made of recent LLM-driven security breaches. Not without good reason, but the discourse generally falls into a few categories:

1. Level-headed warnings about the potential threat,
2. Expert CEOs claiming “We don’t know how to control this technology.”, and
3. Widespread panic about Skynet – both at the semi-technical level, as well as the legislative.

This brief article aims to sit in the space left over, breaking down the wider issue into smaller more manageable problems to solve, and laying out one solution to the largest case, while setting the scene for future updates on additional solutions.

Security isn’t an on/off switch, or a downloadable plugin or product. It is the process of reducing attack surface drastically using a variety of tools and techniques.

Note too that what’s being proposed below isn’t a silver bullet. It is a start that will allow us to reduce the alarms from going off due to accidental breaches, allowing us to follow up with subsequent solutions for malicious cases.

Not The Current Path

The current path sees Western Governments looking to legislate, without being able to even vaguely articulate what that might look like. These are the same governments currently passing various forms of age verification legislation that doesn’t work and is in some cases more harmful. Regardless of country, or political party, they’re not qualified.

What is most likely to happen is that the “experts” will end up in some form of self-regulation, we’ll claim victory and be worse off. Even if we assume 100% sincere motive, the most likely outcome is monopolisation and the death of innovation. Remember, these are the same folks that said that a) this tech will take all our jobs, and b) they have no idea how to control it.

Breaking The Problem Apart

Before getting into any of the mechanical and technical details, let start out by differentiating between the two types of threats we are facing:

1. Cases where these LLMs and agents are being deployed with malicious intent.
2. Cases where it’s accidental or potentially negligent.

Both are problems that we need to be concerned with, but how you tackle each is different and warrants different approaches.

Looking at the evidence, it’s clear that the vast majority of cases that have been thrown around in the media have been the latter case. OpenAI didn’t instruct their LLM+agent to “break into” Hugging Face. But they did leave the gate to the sandbox open and a bunch of credentials lying around that enabled it. Security doesn’t make the stock price go up, and Engineers would rather be working on something more sexy.

I make the assertion that if we can make the friction low enough to implement agents with the right guardrails, it’ll become standard practice and that will take care of the vast majority of non-malicious issues we’re seeing. This then leaves more room for resources, media hyperbole, and general panic to be focussed on the smaller, but more dangerous threat – the malicious actors.

The malicious actors will, and are, using this tech against us. Just as they have for every piece of tech before it. I don’t actually know that solving the problem is any different from a malicious actor without AI. An LLM+Agent is definitely capable of researching and trying things faster – narrowly edging out a teen in a basement jacked up Jolt Cola (if that’s still the stereotype).

I also argue that if we reduce the friction for getting the defense right, we can tackle that too.

I’m going to use this article to go into more detail on the former. If the villagers with their pitchforks and torches don’t get me after this one, I’ll take up the malicious actor part of this in another installment.

Who’s The Problem?

In discussions around this stuff, there’s a lot of emphasis put on the model. A model cannot do anything by itself. Model Henry 8 can be as syphilitic as you please. Model Hannibal can be as evil as you can imagine. It doesn’t matter. An LLM is modelled to give us the impression it’s the equivalent of a human brain, and that’s not a bad analogy. When was the last time a brain walked into a bank with a gun and a bunch of sacks with dollar signs on them?

An Agent is even dumber. It’s like the robot body for the bodyless brain that is the AI LLM. It can give the LLM the instructions it heard from whoever prompted it, gives those instructions to the LLM and performs bodily functions on behalf of the agent. If the model needs to read something off the internet, it asks the model to grab that text and feed it back. If the model wants to scan a filesystem for credentials for Hugging Face, it needs to ask the Agent. If the model wants to scratch its ass, it needs to ask the Agent.

The way this LLM+Agent interaction works is through a loose standard of “Tools”. An Agent starts a model and feeds it its various prompts (tasks, persona) and a list of tools available to it.

Many (many) Agents out there offer a small number of these Tools, with one of those being a generic “give-me-any-system-command-in-the-world-and-I’ll-do-it-no-questions-asked” Tool. Literally anything goes.

Do you remember the last job you started, where on the first day, the company gave you the ability to run any system command on their servers you like? I’ve been working in the industry since the 1990s and it only happened when I was hired as the experienced guy who had to run all the stuff. That was 1998. Even with job titles like “VP” or “Director” and I’ll still only be granted access to what I need.

So why are we giving these agents the keys to our organisation unchecked?

Sandboxing is also important, but even that wasn’t done with any real intent.

Treating AI Like A Human

If a human is employed as an editor for your documents and articles, is it not true that, no matter how malicious their brain may or may not be, the only data and function you’ll give them access to is to:

1. Be able to read your documents and articles and supporting documents and materials, and
2. Be able to mark up your documents with suggested corrections and changes.

You wouldn’t give them access to anything else, and would be given strict instructions about the job (albeit with the inclusion of previous job experience and shared morals – something that AI doesn’t yet really have).

Many organisations, often by mandate, also create audit logs of document accesses and changes. Especially when sensitive data is concerned.

So why would we treat AI (an LLM+Agent) any different?

The same goes for software developer creating software, or a security researcher looking for vulnerabilities in existing software. In both cases, we do currently give both of those roles more rope to hang themselves, but this is a result of a proven track record and common-ish set of morals. And it’s generally been successful. But we still don’t grant them access to the production database, the financials of the company and the data HR has on everyone in the company when they walk through the door. AI should be the same and, for now at least, probably with less assumption about morals and understanding of the job.

The example of a literary editor is a very simple one, but the same security process applies to something more technically complicated (from a security perspective) such as containing a software development process.

Actions

Make Agent Tooling More Standard and Auditable

Two technical points about LLMs and Agents.

1. Anyone can train an LLM and anyone can build an agent. You can even have an LLM write you the source code for an agent. Google Gemini will quite happily write you a few dozen lines of human-readable Python code that is a functional AI Agent with whichever model or provider you like. These genies are out of the bottle.
2. The interface between Agents and Models is loosely defined, and largely a conduit of somebody else’s problem. An LLM can describe almost any set of systems commands for the agent to run, and it will do so. Putting intelligence into the Agent to guess whether something is good or bad, is an arms race we will lose.

So why do many internal and external agents just blindly run commands on behalf on an untrusted LLM?

Because it’s the easy path. Spending money on it doesn’t help your stock price (and never has). Writing the code isn’t sexy either. It’s a lot of boilerplate code that isn’t interesting to a lot of Engineers, which is the fastest way to have something labelled as too complex. Wherever will we find a piece of technology that will gladly take on a project of boring boilerplate code, and deliver it to spec in hours? Let’s ask an LLM if it knows anyone…

The solution to this is a knowledgeable and agile standards group that:

• Maintains guidelines and standards around fine-grained LLM+Agent tooling
• Includes reference implementations for any project wanting to include it – making it the easy path for agent developers
• The tooling layer of the stack is an ideal place to do an amount of security auditing. Any standards or implementation would include auditing, taking data privacy into consideration (ie, not logging sensitive data), digital signing, etc.
• Would include complex cases, such as remote network access, safe execution of malicious code, etc. I can go into more detail in the right forums.
• This could be extended to include stub systems or networks or even honeypots. A model could ask an agent to attempt a brute-force or fuzz attack against a remote web service (let’s call it Mugging Race), and these requests could transparently be sent to a honeypot location.

With reference implementations and tooling, there’s no need to legislate or force adoption. If it’s easy enough for any agent to plug into and utilise, it becomes an easier path than implementing it badly yourself. Agent developers would have the option of implementing things themselves (especially where the reference implementation was unaware of emerging technology becoming available). The guidelines would cover how best to approach that.

With all of the tools and guidelines being available, it would be hard at this point for a company to claim they weren’t negligent if things went awry. Ideally, this would give everyone enough flexiblilty and control, without the right guardrails, and a clear line drawn when it comes to deciding whether something was malicious (on the part of the driver) should that come up.

A reference implementation also means that one team can develop it and make it available to all, further reducing implementation friction. This isn’t likely to be a commercially viable software product, but should be thought of more as a public good. Much like the Netfilter, SELinux or AppArmor projects.

Privilege Separation

I’ve given a few examples of limiting LLM access to tools. This oversimplification was deliberate. There are many cases that are more complex than an editor reading a document and marking it up.

The ability to for one or more Agents to collaborate, build and test a piece of software, a lot of flexibility is needed. Reading library docs, writing the code, executing the tests, fixing bugs, testing the final result. Each of these tasks typically involves running a whole bunch of code that would need to be treated as untrusted, buy an author who is untrusted.

The standards and tooling would include ways for this to happen (including self-hosted) in a way that severely limited the ability to break containment. One set of tools for writing the code, another tool for selectively assembling the dependencies, another tool for performing the build, and other tools in other environments for execution of the result and any intermediate tests. Some might have network access, others would not. There wouldn’t be any keys to other systems left lying around for a start.

Existing Approaches

A lot of effort has been put into trying to train good behaviour into the models and impose “morals” onto them. There’s no reason why this shouldn’t continue. There will be no single solution. Training the models adds extra guardrails.

Making it happen

If this were turned into a software development project, taking requirements from government and the industry, it wouldn’t take a large independent team. All of the standards and source code would be made open. Without wanting to put any words in the mouths of an CEOs claiming to not be able to control their tech, I’m sure that a few of the major players would be able to fish around behind their couch, contribute some spare change and fund it for a few years without breaking sweat. Especially considering the positive media impact involved.

The planning and day-to-day specifics would be made open too, giving private companies and governments the insight they’d be looking for when declaring victory.

An Aside on Precedent

In 1998, Robert Morris executed some software automation he wrote. By his account, not maliciously. It used existing trusts and keys and open gates, to get into other remote systems on the internet. It worked better than expected and happened so fast and so far widespread that it brought thousands of vulnerable servers (essentially a big percentage of the Internet at the time) down. Fast.

He wasn’t trying to steal anything or deny service to anything. It wasn’t malicious. He was convicted of a felony for the first Internet Worm. He wasn’t the only one – The Hacker Crackdown of the 1990s saw many people made an example of without necessarily having done any real damage.

What does it say about CEOs in debt for trillions, who used the greatest set of computer resources ever gathered to unleash the most competent piece of automation software ever conceived, only to have them shrug their shoulders and suggest that they don’t know how to control it? There’s a case to be made that if they don’t know how to control it, that maybe they shouldn’t be in charge of it.

I’m not necessarily suggesting taking legal action in these cases. But if we make it easy for AI and other tech companies to do the right thing, there’s fewer excuses in cases of gross negligence.

AI Foot-Guns

With the sporadic, yet increasing sound of Artificial Intelligence Agent Foot-Guns¹ going off in the distance, I’m left wondering whether we’ve forgotten the lessons of the past. Or perhaps in the rush to use Large Language Models (LLMs) for everything we’re forgetting to put up the same guard rails we did to protect ourselves from Organic Intelligence “hallucinations”. What am I misssing?

It seems like every other day, we’re reading about someone’s experience with LLM-driven automation, such as Agentic AI, ending in tears and with fewer toes. Entire datasets and backups gone. Whole software repositories left a smoking crater.

This isn’t an anti-AI post by any stretch. I’m not an AI naysayer, preaching about the impending AI-pocalypse. But I’m also not a member of the AI cult, drinking the Kool-Aid without checking the ingredients. Like any tool we’ve developed in the industry ever, once you see past the hype, there’s value to be had, but also caution to be exercised.

I started my first software development job in 2000. I already had some years of running production systems in previous roles at that point, all using the only thing we had back then – Organic Intelligence. But my employer at this software development company didn’t, on my first day, hand me the keys to their customers’ production databases to drop it as I see fit. So why are folks out there granting these agents the ability to wipe out a whole dataset?

A few years further into my career, I was running some security and other courses for telecommunications companies, utility companies and government. One of the key security concepts we used to drive home was to grant the least privilege necessary for a given user. So why are folks out there granting agents permission to wipe out entire software repos. In most organisations I’ve worked for, we generally all have to create a pull request and have it reviewed before it’s merged.

My (admittedly relatively new) experience with self-hosting LLMs and agents suggests that there are numerous ways to separate different concerns for different agent functions. It’s also definitely possible to have LLMs write test cases that would also act as an extra set of checks and balances.

A previous employer and mentor of mine once described there being a difference between a software engineer or software developer, and someone who can write code. In some ways, the code written is less important than some of the other things that the job entails.

LLMs can write code, but there’s a lot they’re missing that an engineer brings. The same likely applies for other automated tasks.

So whilst we’re offloading the writing of code (arguably the most fun part) to AI, perhaps we need to remember to take care of the rest of the job that AI can’t, and to supervise it with the same sorts of guardrails we would any human with Organic Intelligence.

Linux Process State

Overview

The Linux kernel exposes a great deal of per-process information through the /proc filesystem interface. This can be very helpful in things like:

• Forensics – What is this process that’s running and where did it come from? What files/resources does it have open?
• Automation – Being able to programmatically retrieve information about processes without trying to parse the output of commands.

We’ll look at a few common, useful objects under the proc filesystem, but you can do no harm looking around, so please do.

Under /proc, there is a subdirectory for each process ID (PID) currently running on the system. There is also a special symbolic link called self that points to the PID subdirectory for the current process (your shell). For example:

$ ls /proc/self/
arch_status         fd                 net            setgroups
attr                fdinfo             ns             smaps
autogroup           gid_map            numa_maps      smaps_rollup
auxv                io                 oom_adj        stack
cgroup              ksm_merging_pages  oom_score      stat
clear_refs          ksm_stat           oom_score_adj  statm
cmdline             limits             pagemap        status
comm                loginuid           patch_state    syscall
coredump_filter     map_files          personality    task
cpu_resctrl_groups  maps               projid_map     timens_offsets
cpuset              mem                root           timers
cwd                 mountinfo          sched          timerslack_ns
environ             mounts             schedstat      uid_map
exe                 mountstats         sessionid      wchan
$

We’ll take a look at few examples below, and potentially look at others in future articles.

What is that process?

Let’s say, for example, we see a running process called ./foo.

$ ps ax
 ...
 269792 pts/2    Sl     0:00 ./foo
 ...
$

We can see that it was started in the directory containing the executable, but we can’t see from this output which directory that is, or the full path to the executable started. There are multiple ways to do this, but we can quickly find answers to both these questions in a way that is also automation-friendly.

Under /proc/<pid> virtual directory (where <pid> is the process ID of the process we’re interested in), we can see two symbolic links of interest:

1. The exe symlink, which points to the binary used to instantiate this running process, and
2. The cwd symlink, which points to the current working directory of the process.

Using the PID 269792 belonging to the process in the output above, we can see both of these files and what they point to:

$ ls -l /proc/269792/{exe,cwd}
lrwxrwxrwx 1 matthew matthew 0 Oct 25 15:38 /proc/269792/cwd -> /home/matthew/src/rust/foo
lrwxrwxrwx 1 matthew matthew 0 Oct 25 15:38 /proc/269792/exe -> /home/matthew/src/rust/foo/foo
$

From this, we can tell that the program foo was started from the binary /home/matthew/src/rust/foo/foo, and that its current working directory (the default location for reading and writing files) is the same directory the binary was executed from, /home/matthew/src/rust/foo.

As a side benefit, the ls -l output above also shows the owner of the files. The Linux proc filesystem uses this to tell us which user the process is running as.

Which files does it have open?

Now that we know where this process was started from, we likely want to know more about what it’s doing. One potential avenue of investigation would be to see what files it has open on the filesystem. The fd subdirectory contains symbolic links that point to files open by that process. Using the above process as our example again:

$ ls -l /proc/269792/fd/
total 0
lrwx------ 1 matthew matthew 64 Oct 25 15:56 0 -> /dev/pts/2
lrwx------ 1 matthew matthew 64 Oct 25 15:56 1 -> /dev/pts/2
lrwx------ 1 matthew matthew 64 Oct 25 15:56 2 -> /dev/pts/2
lrwx------ 1 matthew matthew 64 Oct 25 15:56 3 -> /tmp/.data/log
$

We can see that file descriptors 0, 1 and 2 point to a terminal, which probably makes them the stdin, stdout, and stderr for the process. We can also see that it has a file open called log in the /tmp/.data directory. That might be a good place to continue our search.

Summary

We’ve taken a quick look at a few objects under the /proc directory that can be used for forensics or in automation to find out information about currently running processes in real time. We’ll likely look into others in the future. Suggestions welcome.

Why Wasm At Versatus

Overview

At Versatus, we’ve chosen Web Assembly (WASM) as a target for our Smart Contract execution, as well as other general compute use cases such as Serverless Functions. As with any technology, it’s easy to get caught up in all of the hype. This quick article aims to lay out some of the reasons why we’ve chosen to use Web Assembly for these use cases.

Common Programming Languages

The Web3 space is littered with single-purpose, custom languages for writing things like Smart Contracts. There are technical reasons why this is the case for some chains, but it increases the barrier to entry for new developers and isn’t necessary. By targetting Web Assembly for compiled Web3 components, developers are free to use the language of their choice in the IDE of their choice with the debugging and testing tools of their choice.

Common Execution Target

Just as we shouldn’t necessarily need to dictate which programming language a Web3 developer writes their code in, we feel that they shouldn’t care what hardware the code is eventually executed on. They certainly shouldn’t have to compile their code for one architecture when testing their code and for another architecture entirely when they wish to deploy and run their code.

Using the ever-growing set of tools for compiling many common (and some uncommon) languages to the common Web Assembly instruction set gives us a common execution target regardless of the programming language and regardless of the underlying hardware.

Controlled Security

By default, Web Assembly is just a simple instructions for a virtual CPU. There are no networks or files or devices. This alone gives us a very small surface area to secure. We want to secure the network and the general population from malicious software, and we want to protect the resources of the hosts running the software from misuse or overuse. At the same time, we also want to protect the running software from outside attack. Out of the box, Web Assembly makes it easier for us to contain and secure the executing software.

The Web Assembly Systems Interface (WASI) is a set of extensions to WASM that allow us to selectively and carefully enable some basic I/O functionality, including potential file access or potential network access. Rather than these being a simple on/off switch, we can strictly limit these calls and resources and whitelist only what we know to be necessary and safe.

Extensible

It is possible for someone like Versatus to extend a WASM runtime to include new functionality. This can be done in a number of ways, including adding new WASI calls, modifying the underlying implementation of existing WASI calls, or making use of the proposed component model to add new functionalities.

Consider the case where a developer is fairly new to Web3, understands how Content Addressable Storage (CAS) works in Web3, but doesn’t have the bandwidth to totally come up to speed with the inner workings of IPFS and the APIs needed to interact with it. With some extension to the WASM runtime shipped by Versatus, the following Rust code excerpt could potentially be all that is needed by that developer to retrieve the contents of a file from IPFS:

let data = std::fs::read("/ipfs/QmRgKAxcwi6aJUUA3DGnEUVXNEEocyhqweWRqvxjHh3i5z");

The virtual path /ipfs could be intercepted and handled by Versatus, allowing us to handle the complexities of IPFS and making it trivial for a developer to work with Web3 content.

Summary

Web Assembly (WASM) isn’t just for web browsers. It does help us to solve a number of issues around portability, security and simplicity, as described above. It will change the way a number of types of applications and services are deployed more widely in the industry. It isn’t perfect for every situation, and Versatus ourselves have other use cases where we won’t be using Web Assembly. We don’t believe that it will change the way operating systems work any time soon, as is suggested in this great article, but it will certainly have a lot of impact on the way some types of applications are developed and deployed and mtaintained in the future.

Jump Hosts and OpenSSH

The Problem

If you have used ssh to create a remote terminal session into a Linux or Unix machine, or if you have used related file transfer tools such as scp or sftp to transfer files between machines, you’ll be familiar with the idea that you’re establishing a point-to-point network connection, using the SSH protocol, from one machine (the client) to another (the server).

In the picture below, the laptop on the right is on the same network as the three servers (alvin, simon and theodore). It is able to establish that client-server connection, authenticate, and you have your ssh session.

But what about the laptop on the left? Due to the firewall and probably some stuff called NAT, it’s unlikely that the laptop on the left is able to establish network connectivity to the servers (alvin, simon and theodore) directly. When you’re at home or at the office, you can access those local resources, but when you’re remote, you can’t.

In these cases, it may be that the manager of the firewall might enable ssh on the firewall (or a machine near it in a DMZ) to act as a jump host. This would allow you to ssh from the laptop on the left, into the firewall machine and from there ssh to alvin, simon or theodore behind the firewall.

Here’s what an example session might look like:

mg@laptop$ ssh mg@jump-host.incomplete.io
mg@jump-host.incomplete.io's password:

mg@jump-host$ ssh mg@simon.incomplete.io
mg@simon.incomplete.io's password:

mg@simon$

This approach works, but does have some serious limitations:

• Using ssh public key authentication becomes more difficult (more below).
• Transferring files using scp or sftp might mean copying to the jump host first and then to the real destination, where the jump host may not have the capacity or a writeable filesystem.
• Escaping ssh ~ commands needs a ~ per ssh hop, which can be difficult to keep track of after a few hops.
• It’s very interactive, making it difficult to use with automation.

A ProxyJump Tunnel

It is possible to have the ssh command handle the extra step for us by wrapping the SSH session we want inside an SSH session to the jump host. Using the ProxyJump configuration option (or the -J short option on the command line), we can have ssh automatically establish the session to the jump host. From there, ssh will tunnel our session to the servers over the jump host session. For example:

mg@laptop$ ssh -J mg@jump-host.incomplete.io mg@simon.incomplete.io
mg@jump-host.incomplete.io's password:
mg@simon.incomplete.io's password:

mg@simon$

This has many advantages over the two manual steps above, many stemming from the fact that the session to simon in the above example is from laptop to simon, being tunnelled through jump-host. This allows us to use tools like scp to transfer files without having to store them on the jump host as part of the process. For example:

mg@laptop$ scp -o 'ProxyJump mg@jump-host.incomplete.io' ./some-file mg@simon.incomplete.io:
mg@jump-host.incomplete.io's password:
mg@simon.incomplete.io's password:

....

The scp command doesn’t have the short -J option, so we specify the long-form configuration option using -o.

You can use the -J option multiple times if you have multiple jump hosts to pass through.

Tunnelling Your Browser

As mentioned, the -J option establishes a session to a jump host and then tunnels our SSH session to the remote server. OpenSSH is able to tunnel all kinds of things, including web browser traffic. By using the -D option and specifying an unused port number, ssh can listen on your local client machine (laptop in the example above) as a SOCKS5 proxy. It will then tunnel any web requests it receives over the SSH session to the jump host, and make the request on the remote network.

For example:

mg@laptop$ ssh -D3128 -J mg@jump-host.incomplete.io mg@alvin.incomplete.io
mg@jump-host.incomplete.io's password:
mg@simon.incomplete.io's password:

mg@simon$ 

Then simply configure your web browser to use localhost as a SOCKS5 proxy with the port number specified with the -D option. From there, you ought to be able to browse content inside your private network (perhaps hosted on simon, alvin and theodore), with the web browser traffic being tunnelled (encrypted) over your SSH session through the jump host.