Give an agent access to an S3 bucket with grep, cat, and Python

To give an agent access to an S3 bucket with grep, cat, and Python, expose the bucket as a filesystem. Let the agent work with files using the tools it already knows, rather than making it write S3 API code before it can inspect the data. Archil provides that bridge by connecting a bucket to a disk once, so commands see objects as files without a separate download step in the agent’s workflow.

How to give an agent access to an S3 bucket with grep, cat, and Python

The setup has two parts, namely a disk backed by the bucket and a bash tool that runs commands with that disk mounted. The disk makes the objects available as paths. The bash tool lets the agent explore those paths with shell commands it knows, then gets the command’s output and exit code. This keeps bucket access out of the agent’s code without asking it to learn a new way to read a file.

Connect the bucket once, then let the agent run commands on the mounted directory. A search can use grep, a quick inspection can use cat and head, and a small calculation can use Python. None of those commands needs to know how to page through an object listing.

A filesystem view of a bucket is not a promise that changes appear instantly in both places. Archil synchronizes the disk and the bucket, with timing and writing constraints covered below. This way of reading files is simple, but using it means knowing where permissions apply and when new data becomes visible on the disk or in the bucket.

Why agents are better with files than with S3 API calls

Coding agents are trained on shell sessions. They reach for ls, grep, cat, head and short Python scripts because those tools fit the work of finding files, looking inside them and calculating something from their contents. Instead of changing how the agent explores data, the filesystem makes the bucket available in a form it already knows.

Against a raw bucket, the agent has to write S3 API code, page through listings and download objects before it can look inside them. Those steps cost time and tokens, even when the question just asks for a word or a few lines. Giving the agent file paths removes that API work from its code.

Downloading the entire bucket first takes a different route to the same familiar tools, but it introduces another problem. Big buckets take a long time to download, and the copy goes stale. The agent can use the shell, but it is now exploring a snapshot that has to be kept current. Exposing the bucket as a directory instead gives it access to live data through the disk, subject to synchronization delays.

Connect the bucket as an Archil disk

Create an Archil disk with the bucket as its data source. Bring your own bucket is available on paid plans and works with Amazon S3, Google Cloud Storage, Cloudflare R2 and other S3-compatible stores. The agent’s later commands use the disk, so they can get another object without setting up the connection again.

For Amazon S3, the recommended setup is a bucket policy that lets Archil’s IAM role list, get, put and delete objects. No long-lived keys change hands, and an archilSessionId condition scopes the grant to a single disk. Static IAM credentials also work. The example below uses those credentials.

Bucket permissions and the agent’s mount serve different purposes. The policy grants Archil access to the backing objects, while a read-only mount restricts what the agent’s commands can do through the disk. A connection can allow writes while the agent that explores it is kept from overwriting the source data.

Each file on the disk remains an ordinary object at the matching key in the bucket. So other tools can keep using the bucket directly. There is no special object format for those tools to decode. However, Archil tracks POSIX metadata, including mode bits and ownership, itself, and that metadata is not visible when reading the bucket directly. The contents are available through both interfaces, but the filesystem metadata belongs to the disk view.

Example: let the agent run grep, cat, and Python over the bucket

Archil’s serverless execution runs a bash command in a Firecracker microVM with the disk mounted under /mnt/archil. It returns stdout, stderr and the exit code, which supplies a complete bash tool for an agent loop. The runtime image includes common Unix tools. So grep, cat and python3 work without having to set them up inside the command environment.

The following example creates a disk backed by the reports bucket and wraps execution in a bash function. The mount name data makes the bucket available at /mnt/archil/data, and readOnly: true prevents the commands from changing it. The wrapper returns the exit code and command output. Its sample calls then search for matching files, inspect a CSV and count its records.

import * as archil from "disk";

// Once: a disk synchronized with your bucket (files stay ordinary objects in S3)
const { disk } = await archil.createDisk({
  name: "reports-bucket",
  mounts: [{
    type: "s3",
    bucketName: "my-reports-bucket",
    accessKeyId: "AKIA...",
    secretAccessKey: "..."
  }],
});

// The agent's bash tool: the bucket appears at /mnt/archil/data, read-only
async function bash(command: string) {
  const { stdout, stderr, exitCode } = await archil.exec({
    disks: { data: { disk: disk.id, readOnly: true } },
    command,
  });
  return `exit ${exitCode}\n${stdout}${stderr}`;
}

console.log(await bash(
  "grep -rl 'refund' /mnt/archil/data/reports/ | head"
));
console.log(await bash(
  "cat /mnt/archil/data/reports/q3/summary.csv | head -5"
));
console.log(await bash(
  "python3 -c \"import csv; print(sum(1 for _ in csv.reader(open('/mnt/archil/data/reports/q3/summary.csv'))))\""
));

The first command searches recursively for refund under the reports directory and uses head to limit the displayed matching paths. The second command opens the named summary file and shows its first five lines so the agent can see a little of the data. The Python command then reads that file with csv.reader and counts the records it gets, including a header record if there is one.

These commands illustrate different jobs: the shell handles discovery and inspection, while Python handles a calculation over parsed CSV records. The disk connection takes care of the work needed to read from the bucket. That leaves the agent’s commands focused on the question in front of it, rather than mixing data exploration with object retrieval code.

Execution has limits that affect how results should be returned. Each command can run for up to 5 minutes, and stdout and stderr are each capped at the last 128 KiB. Large outputs should therefore be written to a file on the disk. Since this example mounts the source read-only, those outputs go on a separate writable disk.

For searches across many files, Disk.grep offers another path. It fans a regular-expression search out across many microVMs and returns structured matches, rather than making a single bash command do all the searching. For an agent using the AI SDK, createDiskTools from @archildata/ai-sdk provides read, write, list, grep and bash tools on a disk in one call. With those tools, there is no need for the custom bash wrapper shown here.

Keeping the agent read-only

Pass readOnly: true for the disk in archil.exec when the agent is exploring source data. The execution restriction prevents the command from modifying that mounted disk, so it cannot delete or overwrite the bucket’s objects through it.

To narrow what the agent can access, use the subdirectory option to limit the mount to one prefix of the bucket. Read-only access controls whether commands can change files, while a restricted prefix controls which part of the bucket appears in the mount. Together, they let the source view match the portion of the data the agent needs to explore.

Use a separate writable disk, mounted next to the read-only disk, for scratch files and outputs. That gives calculations somewhere to put results without opening the source data to changes. It also provides the destination needed when output is too large for stdout or stderr.

How changes reach the bucket

For writable disks, changes reach durable storage before they reach the backing bucket. Archil stores writes durably first, then synchronizes them to the bucket asynchronously, combining them into fewer PutObject calls. Those are separate stages. A write being durable in Archil does not mean the matching object has already been updated for a tool reading the bucket directly.

The delay also applies in the other direction. Objects added directly to the bucket can take minutes to appear on the disk. An agent using the mounted path may have to wait to see a new object that another tool can already read from the bucket. This qualifies the idea of live access.

There is also a writing constraint to respect. Writing to the same file from the bucket side and the disk side at the same time is not supported, so keep those writers on different paths. Other tools can continue using the bucket directly, but that compatibility is not permission to have both interfaces update the same file concurrently.

For a new data set that needs both file and S3 access with strong consistency, Archil’s own S3-compatible API on a managed disk is the better fit. That is a different requirement from exposing an existing bucket to an agent for exploration. Choose the storage to fit that consistency need, then give the agent the file tools it needs for the work. An agent asked to read a CSV should get to spend its effort reading the CSV.