# The buyer’s guide to storage for AI agents

Source: https://archil.com/guides/storage-for-agents

What an agent’s work needs from storage, the obvious ways to build it and where each one breaks, and why getting it right is hard.

## Contents

1. [The work outlives the machine](https://archil.com/guides/storage-for-agents/the-work-outlives-the-machine.md): An agent’s sandbox is disposable. Providers stop it when it sits idle, and its host can be retired with little warning. The workspace, meaning the repository, packages, databases, and notes, has to outlive every one of those machines, which sets six requirements that no classic storage design meets at once.
2. [The obvious ways to build it, and where each one breaks](https://archil.com/guides/storage-for-agents/the-obvious-ways-to-build-it.md): Teams that build storage for agents themselves tend to climb the same ladder, from the sandbox’s own disk to uploads to S3, a mounted bucket, a network volume, NFS, and JuiceFS. Each rung fixes the failure of the one before it and meets a new one. Each puts the authoritative copy of the workspace in one place, and that place decides what breaks.
3. [Why it’s hard: where the truth lives](https://archil.com/guides/storage-for-agents/where-the-truth-lives.md): Every storage system decides which machine may answer a read or a write without asking anyone. Speed wants that to be the machine doing the work, safety wants copies elsewhere before anything is acknowledged, and sharing wants a single authority. The design that satisfies all three keeps the truth remote and replicated and lends the right to answer to the working machine, and building it is a long list of hard problems.
4. [Copies versus references](https://archil.com/guides/storage-for-agents/copies-and-references.md): Forking an attempt, handing a workspace to another machine, and leaving a vendor each cost either a copy of the data or a new pointer to it. A copy takes time and storage in proportion to the workspace, and a pointer doesn’t. On Archil-managed disks the first two are pointers, but leaving is a copy, because those disks use our internal format.
5. [What you pay for](https://archil.com/guides/storage-for-agents/what-you-pay-for.md): Storage bills for one of three things: capacity you reserve, the machine the files live on, or what you use. An agent’s workspace waits far more than it works, so which of those you pay for matters more than any rate. Billing by use needs storage that stands apart from both a reserved size and the machine.
6. [What Archil does, and where it falls short](https://archil.com/guides/storage-for-agents/what-archil-does.md): Archil gives each agent workspace one disk that any machine can mount, replicated across availability zones before `fsync()` returns, with checkpoints and branches in milliseconds at any size, billed for the data in use. It’s slower than Amazon EFS on large sequential writes, and shows other machines a new file only when their cache expires or they ask for it.

## 1. The work outlives the machine

Source: https://archil.com/guides/storage-for-agents/the-work-outlives-the-machine · Part of The buyer’s guide to storage for AI agents · Facts checked 2026-11-04

**The short answer:** An agent’s sandbox is disposable. Providers stop it when it sits idle, and its host can be retired with little warning. The workspace, meaning the repository, packages, databases, and notes, has to outlive every one of those machines, which sets six requirements that no classic storage design meets at once.

Follow one coding agent through a day. In the morning it clones a repository into a fresh sandbox, installs its packages, and creates a SQLite database for its tests. Then it asks a person to review its plan, and while it waits, the sandbox’s idle timer stops the machine. The person replies before lunch, and the agent picks up on a different host. That host is retired early in the afternoon, so the agent moves again. Later it tries three fixes at once, each in its own sandbox, and keeps the one whose tests pass. The next morning, it starts where it left off.

Count the machines, and there are seven. Count the workspaces, and there is one. Everything the agent did lives there: the repository and its history, the installed packages, the build output, the test database, and the notes the agent wrote for its next run. Every machine did its job and was thrown away. If the workspace had gone with any of them, the day’s work would have gone too.

**One workspace, a day of machines**

One workspace across a day of machines, from 09:00 to 18:00 and on to the next morning. Sandbox A clones a repository and runs `npm ci`, writing `repo/`, `node_modules/`, `app.db`, and `notes.md` into the workspace, and its auto-stop ends it at about 09:40. Sandbox B resumes on a second host at 11:00 and runs until that host is retired at 13:30. Sandbox C picks up on a third host. At 15:00, three attempts run at once on sandboxes D, E, and F, each on its own branch of the workspace, and at 16:00 the attempt that passes its tests is kept and the other two are thrown away. The workspace waits overnight, and sandbox G picks up again at 09:00 the next day. Seven machines, one workspace.

This guide is about that workspace: what it needs from storage, the designs teams build for it and where they break, why the problem is harder than it looks, and what it costs. This chapter starts with the two facts the rest of the guide depends on. The sandbox is disposable, and the workspace isn’t.

### A machine built to be thrown away

A sandbox exists to run code that nobody has reviewed, somewhere that code can’t do harm. The cheapest clean room is a new one, so sandboxes are built to be created on demand and thrown away afterward. A sandbox is a machine rented for the length of a task. The provider places it on whichever host has room, and each of the providers in the next figure builds in a timer that stops it. E2B’s default timeout ends a sandbox and its state, unless you set the sandbox to pause instead. Modal’s sandboxes end at a default timeout too, and Daytona stops a sandbox that sits idle. Ours pause when they sit idle, and no session runs longer than a day. (From each provider’s documentation, as summarized on our comparisons with [E2B](https://archil.com/compare/e2b), [Modal](https://archil.com/compare/modal), [Daytona](https://archil.com/compare/daytona), and [Vercel Sandbox](https://archil.com/compare/vercel-sandbox), and from our own [limits](https://docs.archil.com/reference/limits). These are defaults, and most of them can be changed.)

Stopping is often gentler than it sounds. E2B can pause a sandbox and keep its memory and files, and Daytona keeps a stopped sandbox’s files, archiving them after 7 days. But a paused sandbox keeps its files with that one sandbox. When the agent’s next step runs on another machine, or as three attempts at once, the files have to be somewhere that every one of those machines can reach.

Underneath every sandbox is a physical host, and hosts fail. AWS schedules an EC2 instance for retirement “when AWS detects irreparable failure of the underlying hardware,” and warns that by the time you hear about it, the instance “might already be unreachable.” Data on the host’s own drives doesn’t survive a retirement, a stop, or a failed disk. (From [Instance retirement](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-retirement.html) and [Data persistence for Amazon EC2 instance store volumes](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-store-lifetime.html). What this means for a program that calls `fsync()` is in [Where fsync returns from](https://archil.com/guides/how-file-systems-work/where-fsync-returns-from).) Neither the agent nor you decides when that happens.

**Every sandbox has a timer**

A clock runs from 1 minute to 30 days on a log scale. E2B’s default timeout ends a sandbox, and its state, at 5 minutes. Modal’s default timeout is also 5 minutes. Daytona stops a sandbox after 15 idle minutes, Archil pauses one after 30 idle minutes, and a Vercel Sandbox session on the Hobby plan ends at 45 minutes. Where a longer session is allowed, it ends at 24 hours: on E2B Pro, Modal, Archil, and Vercel Pro. In the last row, the agent’s work picks up again after a pause of almost 15 days, the longest pause in our agent loops.

### Where an agent’s hours go

The timers exist because agents spend most of their time waiting. We read the timestamps on 30 days of our engineering team’s coding-agent sessions: 384 agent loops and about 37,000 tool calls. The median tool call finished in 0.5 s, and the model’s median turn took 2.7 s. Between bursts of work, the agent waits on a person, and some of those waits last overnight or longer. At least 26 of the loops picked up again more than a day after their previous tool call. (The method is in [Do AI agents need a sandbox running for the whole session?](https://archil.com/article/do-ai-agents-need-a-sandbox-running-for-the-whole-session) We read only when each model turn and tool call started and ended, not what they contained. The count of 26 comes from re-running the analysis a day later, on the 363 loops whose timestamps we still had.)

That pattern defeats every timer setting. Suppose each loop had started a sandbox at its first tool call and stopped it after 15 minutes without one. That sandbox would have been idle for 65% of the time it ran. A shorter timer idles less, but the sandbox stops more often, and each new start needs the workspace on a new machine.

**Drag the auto-stop timer**

An explorable chart of 31,422 pauses between tool calls, from 363 of our agent loops, on log scales from 0.1 s to about 3 weeks. Most pauses last 1 to 10 s, while the model writes its next step. A long, thin tail runs through minutes and hours to pauses of days. Pauses of 10 minutes or more are 1.5% of all pauses but hold 98% of the waiting time. Dragging the auto-stop timer splits the pauses: shorter ones keep the sandbox up and idle, and longer ones stop it. Over all 384 loops, a 1-minute timer leaves the sandbox idle 28% of the time it runs (54% in the median loop), for 243 hours of sandbox time against 174 hours of tool work, and 1,303 pauses outlast it, about 4.6 starts per loop. A 5-minute timer: idle 48% (75% in the median loop), 337 hours, 669 pauses, 2.8 starts per loop. A 15-minute timer: idle 65% (88%), 493 hours, 384 pauses, 2.1 starts per loop. A 1-hour timer: idle 83% (96%), 1,019 hours, 226 pauses, 1.6 starts per loop. Pause and start counts come from the 363 loops.

Neither end of the timer is a good home for the work. A long timer pays for a machine that mostly waits, and a short one throws the machine away several times in a loop. Either way the machine is temporary, so the workspace has to live somewhere that isn’t. What that waiting costs in storage is the subject of [What you pay for](https://archil.com/guides/storage-for-agents/what-you-pay-for).

### What the workspace needs

Read the day again, and each thing that happened to the workspace sets a requirement for the storage under it:

- **It survives a lost machine.** When the host is retired mid-task, every write that the agent was told was saved has to be there on the next machine. That makes a write safe when `fsync()` returns, not when an upload finishes later. Storage that acknowledges from the host’s own drive and uploads afterward can lose its newest writes with the host. See [When is a write safe?](https://archil.com/guides/evaluating-storage/when-is-a-write-safe)
- **It’s as fast as a local disk on thousands of small files.** An `npm ci` of one Next.js app writes 18,602 files. Add 1 ms of network waiting to each file, an assumption for the arithmetic, and the install takes about 19 s longer. Agents run steps like this all day, and wait on every one. See [How fast is it on small files?](https://archil.com/guides/evaluating-storage/how-fast-is-it-on-small-files)
- **It moves to other machines, and is shared with them.** The next tool call can land on any host. Copying a 5 GiB workspace onto each new machine at an assumed 500 MB/s takes about 11 s, more than 20 times the median tool call. So the workspace has to be reachable from wherever the call runs, and sometimes from two machines at once. See [Can many machines use it at once?](https://archil.com/guides/evaluating-storage/can-many-machines-use-it-at-once)
- **It forks cheaply.** Three attempts from one starting point need three writable copies of the workspace, made in about the time it takes to start the sandboxes. See [Can you fork it?](https://archil.com/guides/evaluating-storage/can-you-fork-it)
- **It costs little while idle.** Most of a workspace’s life is waiting, and storage that bills by the machine or by a reserved size bills all of it. See [What does it cost when nothing is running?](https://archil.com/guides/evaluating-storage/what-does-it-cost-when-nothing-is-running)
- **It’s a real file system.** `git commit` renames a lock file over the index, SQLite writes pages in place, and `npm install` creates symbolic links. The agent’s tools assume POSIX, and the agent didn’t write them. See [Is it really a file system?](https://archil.com/guides/evaluating-storage/is-it-really-a-file-system)

**From the day to the requirements**

Six events from the agent’s day, each with the requirement it sets. The host is retired mid-task, so the storage survives a lost machine. `npm ci` writes thousands of files, so it needs local-disk speed on small files. The next call lands on another host, so the workspace moves to, and is shared with, other machines. Three attempts start from one state, so it forks cheaply. It waits overnight, or for days, so it costs little while idle. Git, npm, and SQLite run as they are, so it is a real file system.

The storage you already have meets some of these requirements, never all of them. The sandbox’s own disk is fast and is a real file system, and it disappears with the host. An S3 bucket outlives any machine and bills only for what it stores, but it isn’t a file system, and every operation is a request over the network. The requirements pull in opposite directions. Speed and POSIX want the data on the machine doing the work, and surviving, moving, and idling cheaply want it somewhere else.

A team that runs agents ends up building something between those two poles. [The next chapter](https://archil.com/guides/storage-for-agents/the-obvious-ways-to-build-it) climbs through the designs teams reach for, starting with the sandbox’s own disk, and shows which requirement each one gives up.

## 2. The obvious ways to build it, and where each one breaks

Source: https://archil.com/guides/storage-for-agents/the-obvious-ways-to-build-it · Part of The buyer’s guide to storage for AI agents · Facts checked 2026-11-04

**The short answer:** Teams that build storage for agents themselves tend to climb the same ladder, from the sandbox’s own disk to uploads to S3, a mounted bucket, a network volume, NFS, and JuiceFS. Each rung fixes the failure of the one before it and meets a new one. Each puts the authoritative copy of the workspace in one place, and that place decides what breaks.

Every team that gives its agents a workspace starts with the storage it already has. Most then climb the same ladder. Each rung is a sensible answer to the rung before it, and each breaks somewhere new.

To compare the rungs fairly, this chapter runs one day through each of them. An agent clones a repository and runs `npm ci`. It commits, `fsync()` returns, and then the host it ran on is lost. A sandbox on a second machine picks up the work, forks three attempts at the next change, and then the workspace sits untouched for a week. [The work outlives the machine](https://archil.com/guides/storage-for-agents/the-work-outlives-the-machine) explains why that day is ordinary for an agent.

Where we have measurements, the numbers are ours, and each section says what we measured on. Where we don’t, the chapter says what the vendor documents, and doesn’t guess.

### The sandbox’s own disk

The first workspace is the disk the sandbox came with. It’s the best storage on the ladder for the first half of the morning. Every tool works, every call is answered by the kernel on the same machine, and nothing later on the ladder is faster. Every other rung is measured against it.

It fails at the day’s third step. A sandbox runs on whichever host has room, so its disk is that host’s disk. Data on the host’s own drives doesn’t survive a retirement, a stop, or a termination. (From AWS’s page on [instance store data persistence](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/instance-store-lifetime.html). [The work outlives the machine](https://archil.com/guides/storage-for-agents/the-work-outlives-the-machine#a-machine-built-to-be-thrown-away) covers how hosts are retired.) The clone, the install, and the commit go with it, and the sandbox that replaces it starts on an empty disk.

**The disk goes with the host**

A sandbox on host 1 clones a repository, installs its packages, and commits, all on its own disk. Host 1 is retired, and the disk goes with it. The sandbox starts again on host 2 with an empty disk, so the clone, the install, and the commit are gone.

Even without a failure, the disk lasts only while the sandbox runs. An idle sandbox with 2 vCPUs and 4 GiB costs about $0.17 an hour at E2B’s list price. (E2B bills $0.000014 per vCPU-second and $0.0000045 per GiB-second ([E2B pricing](https://e2b.dev/pricing)).) Keeping one up through a week of idling, so that its disk survives, costs about as much as storing a 5 GiB workspace in S3 Standard for 20 years.

### An upload when the agent stops

The second rung keeps the speed and adds a copy. The agent still works on the sandbox’s disk. The workspace goes to S3 at the end of each session as a tarball, or every few minutes with `aws s3 sync` or rclone. S3 stores objects across at least three availability zones, and a week of idling costs almost nothing. (From AWS’s page on [S3 data protection](https://docs.aws.amazon.com/AmazonS3/latest/userguide/DataDurability.html).) Products sold to agent builders polish this design. Fly.io writes a Sprite’s data “to fast NVMe storage” and backs it up “to durable object storage” when the Sprite goes idle. Vercel describes its Drives as having “NVMe speed writes.” (From the [Sprites overview](https://docs.fly.io/sprites/) and Vercel’s [Drives documentation](https://vercel.com/docs/sandbox/concepts/drives).)

The failure is the gap between a write and its upload. `fsync()` returns once the local drive has the data, and the host can be lost before the copy is made. We measured that gap on Vercel Drives. In 21 of 21 snapshots taken the moment `fsync()` returned, the file wasn’t there.

**A Vercel Drive after fsync returns**

A sandbox wrote a 4 KiB file and a 64 MiB file, called `fsync()` on each, and synced the directory. After a delay, a new sandbox read the Drive through a snapshot or a fork. Taken the moment `fsync()` returned, 10 of 10 snapshots, 11 of 11 snapshots taken while the writer kept writing, and 11 of 11 forks were missing the file. At 1 second, every snapshot and fork still lacked it. At 30 seconds, 7 of 10 snapshots and 9 of 10 forks lacked it, and 4 of 10 snapshots taken while the writer kept writing held a 64 MiB file whose last 12 to 28 MiB read as zeros. From 60 seconds on, the file was intact in every trial but one. An Archil disk mounted in a new sandbox the moment `fsync()` returned had the file in 20 of 20 trials.

Drives and Sprites are reasonable trades for speed. But a database that tells a user their transaction committed has called `fsync()` and believed the answer. For more on where that answer comes from, see [When is a write safe?](https://archil.com/guides/evaluating-storage/when-is-a-write-safe)

The rest of the day costs copies. The second machine downloads the last upload before it can start, and loses whatever came after it. Three attempts mean three downloads. A tarball keeps modes and links but moves the whole workspace every time. A sync tool moves only what changed, but as one object per file, which can’t hold everything a file can, as [A file system on an object store](https://archil.com/guides/how-file-systems-work/a-file-system-on-an-object-store) explains.

Every team that builds this rung patches it, and each patch is sensible. The following list shows where the common ones lead:

- **Upload more often.** Syncing every minute instead of at the end shrinks the gap, but it never closes it. A busy host always has writes in flight when it dies, and the program was already told they were safe. In our Drive test, 7 of 10 snapshots taken 30 seconds after `fsync()` were still missing the file.
- **Leave out `node_modules` and the build output.** The uploads shrink, and every new machine reinstalls before it can work. The `npm ci` in our measurements took 10.7 s on a local disk, paid again on every start.
- **Snapshot with `tar` instead of syncing.** A snapshot is consistent and keeps modes and links, but it moves the whole workspace each time. So it runs less often, and the gap grows.
- **Make the sync safe to run.** `rclone sync` makes the destination match the source, “including deleting files if necessary,” and its documentation warns that “this can cause data loss.” If a restore fails quietly and the next upload runs from a nearly empty workspace, the good copy is what gets deleted. (From the [rclone sync documentation](https://rclone.org/commands/rclone_sync/).)
- **Lock the workspace so that two machines can’t both write it.** Now you run a lock service, and you have to decide what happens to a lock whose holder’s host was retired. [Chapter 3](https://archil.com/guides/storage-for-agents/where-the-truth-lives) is about that decision.

Follow the patches far enough and they meet at one requirement. The copy has to be made before `fsync()` returns, on every write, and the next machine has to start without a download. That isn’t a sync job anymore. It’s a file system whose storage lives somewhere else, which is the next rung.

### A bucket mounted as a file system

If the gap is the problem, remove it and write straight to the bucket. Mountpoint for Amazon S3 and s3fs mount a bucket so that each file is an object. Mountpoint uploads a file before `fsync()` returns, so a lost host takes nothing with it. Any machine can mount the same bucket, and every file is an ordinary object that other tools can read. In our measurements, a second machine saw a new file in under a third of a second without being told to look.

It fails at the first step of the day. S3 can’t rename an object, and git depends on renames. It publishes every change by writing a lock file and renaming it over the original. npm links package binaries with symbolic links. Mountpoint supports neither rename on a general purpose bucket nor symbolic links, and its README says not to use it for a Git repository. s3fs fakes a rename with a copy and a delete, so git runs, but its README lists atomic renames among what it can’t provide. (From the [Mountpoint README](https://github.com/awslabs/mountpoint-s3), its [semantics document](https://github.com/awslabs/mountpoint-s3/blob/main/doc/SEMANTICS.md), and the [s3fs README](https://github.com/s3fs-fuse/s3fs-fuse).)

**Five calls, five failures**

Five commands and the call each one depends on. `git commit` renames `index.lock` over `index`, and without an atomic rename git can’t update its index. SQLite writes a 4 KiB page at an offset in the database file, and without in-place writes each commit rewrites the whole file. `npm install` creates symbolic links in `node_modules/.bin`, and without symlinks the install fails. `pnpm install` hard-links files from its store into `node_modules`, and without hard links every file is copied. `git checkout` sets the executable bit on `build.sh`, and without mode bits `./build.sh` fails with `Permission denied`.

Where it does work, every small file pays for a round trip to S3. Creating a 4 KiB file and syncing it took 244 ms on Mountpoint on our EC2 host. A mounted bucket suits training data that is written once and read many times. An agent’s workspace is the opposite.

### A network volume

The fourth rung goes back to a real disk, one that lives on the network. To the kernel, an EBS volume is a block device with ext4 on it, so git, npm, and SQLite run at local-disk speed. The local disk in our benchmarks is in fact an EBS gp3 volume. And the volume outlives the machine: “EBS volumes persist independently from the running life of an EC2 instance,” and EBS replicates each one within its availability zone. (From the AWS pages on [Amazon EBS volumes](https://docs.aws.amazon.com/ebs/latest/userguide/ebs-volumes.html) and [EBS features](https://docs.aws.amazon.com/ebs/latest/userguide/EBSFeatures.html).) It’s the first rung that runs the whole morning and keeps the commit.

It breaks at the handoff. The file system on the volume keeps its map of blocks in one kernel’s memory, so the volume attaches to one machine at a time. And “the volume and instance must be in the same Availability Zone.” Taking it from a failed instance means a forced detach, which AWS calls a “last resort.” The instance “doesn’t get an opportunity to flush file system caches or file system metadata,” so you check the file system before you use it. (From the AWS pages on [attaching](https://docs.aws.amazon.com/ebs/latest/userguide/ebs-attaching-volume.html) and [detaching](https://docs.aws.amazon.com/ebs/latest/userguide/ebs-detaching-volume.html) a volume. Multi-Attach lets an `io1` or `io2` volume attach to several instances in one zone, but ext4 and XFS aren’t built for that ([Multi-Attach](https://docs.aws.amazon.com/ebs/latest/userguide/ebs-volumes-multi.html)).)

**A volume outlives its machine, inside one zone**

Machine A commits to an EBS volume in availability zone a, and then machine A is lost. The volume survives. Machine B, in the same zone, force-detaches the volume, attaches it, and checks the file system, and the commit is there. Machine C, in zone b, can’t attach the volume at all.

The rest of the day follows from the same fact. Three attempts mean a snapshot and three new volumes, and a volume restored from a snapshot fetches its blocks from S3 as they’re read, so it starts slow. (From AWS’s [Amazon EBS snapshots](https://docs.aws.amazon.com/ebs/latest/userguide/ebs-snapshots.html) and [Initialize Amazon EBS volumes](https://docs.aws.amazon.com/ebs/latest/userguide/ebs-initialize.html) pages.) And a volume bills the size you gave it, idle or not. A 20 GiB volume holding a 5 GiB workspace bills four times what it holds, and EBS can grow a volume but can’t shrink it.

### A shared file system over NFS

The fifth rung lets every machine mount the same files at once. On Amazon EFS, any number of machines mount one file system over NFS. A Regional file system stores a write across availability zones when the program calls `fsync()` or closes the file, and you pay for what you store. (From the AWS page on [EFS features](https://docs.aws.amazon.com/efs/latest/ug/features.html).) The host can be lost, and the second machine mounts the file system and finds the commit. It’s the first rung that runs the morning’s tools and gets through the middle of the day without a forced detach or a download.

It breaks on the morning’s small files. When two machines create the same name, only the server can decide which one wins. So an NFS client waits for the server on every create, every change of mode, and every close. A small file costs about four round trips. For how those calls add up, see [Anatomy of a file create](https://archil.com/guides/how-file-systems-work/anatomy-of-a-file-create). Amazon S3 Files, which serves a bucket over NFS, shows the cost on the install.

**npm ci, timed until durable**

`npm ci` of a Next.js app writes 18,602 files. On one EC2 host, timed until every write was durable, it took 10.7 s on the instance’s own EBS volume, 12.4 s on Archil, and 71.4 s on Amazon S3 Files. The animation plays the three runs side by side, 10 times faster than they ran.

The install takes almost seven times as long over NFS as on the volume. EFS itself took 86.3 s for `npm ci` of a create-next-app project in a separate benchmark, 11 times as long as Archil in the same run. (That benchmark stopped the clock when each command returned. For the hosts, versions, and commands behind both benchmarks, see [File systems on one host](https://docs.archil.com/reference/benchmarks/file-systems).) NFS clients also cache what they’ve seen, so a second machine sees a new file only when its cached copy of the directory expires. Archil’s clients do the same by default.

**Until another machine sees a new file**

We created a file on one client and measured how long a second client took to see it. Archil, when the reader runs `archil invalidate-cache` first: about 70 ms. Mountpoint for Amazon S3: 284 ms, with no metadata cache by default. A Daytona Volume: about 360 ms, the median of 10 runs. Amazon S3 Files: 30 s, the length of its NFS directory attribute cache. Archil with default client caching: 30 s, its client cache expiry. A Modal Volume: not within 60 s, because a running sandbox sees another sandbox’s commit only after it reloads the Volume.

Forks are the last cost on this rung. EFS has no clone, so each attempt copies every file over NFS and pays the round trips of the install again.

### JuiceFS, and a database you run

The sixth rung stops asking one system to do everything. JuiceFS keeps names, directories, modes, links, and the map from each file to its blocks in a metadata database that you run, such as Redis or PostgreSQL. It keeps file contents as numbered blocks in your bucket. (From the JuiceFS [architecture page](https://juicefs.com/docs/community/architecture). The Community Edition is Apache 2.0.) A rename changes one row, a write in the middle of a file uploads only the new bytes, and any machine can mount it. `juicefs clone` copies only metadata, so forks share blocks. It’s the most complete of the do-it-yourself designs.

**Two systems to keep alive**

A machine running the JuiceFS client commits. The names, modes, and map of blocks go to a metadata database you run, such as Redis or PostgreSQL, which you have to back up, fail over, and size. The file contents go to your bucket as numbered blocks. Lose the database, and the bucket holds blocks with no names.

The cost is the database, which you now operate, and the file system is only as safe as it is. JuiceFS’s own guide to Redis says the default log sync means “you may lose the writes from the last second.” It also says failover “does not guarantee that acknowledged writes are retained during failures,” and “Redis data needs to be backed up regularly.” (From JuiceFS’s [Redis best practices](https://juicefs.com/docs/community/redis_best_practices/).) Lose the database, and JuiceFS’s docs say “you cannot find the original file directly in the object storage.”

The database also sits in the path of every small file. With Redis on ElastiCache in the same availability zone, our `npm ci` took 161.6 s with JuiceFS’s defaults, 15 times as long as on the volume. Its write-back mode is faster because it stages blocks on local disk and uploads them later. Its docs warn that “if write cache data suffers loss before upload is complete, file data is lost forever.” (From JuiceFS’s guide to [cache](https://juicefs.com/docs/community/guide/cache/). Our setup is in the [file system benchmark](https://docs.archil.com/reference/benchmarks/file-systems#archil-vs-juicefs-vs-amazon-s3-files).) That is the second rung’s gap, back again.

### One day on every rung

Run the day yourself. Pick a design, and each step shows whether the workspace comes through it and how long it took where we measured it. The idle week is priced at list price for a workspace holding 5 GiB. Where we have no measurement, the step says what the vendor documents instead.

**An agent’s day**

One workspace through one day on seven designs, judged step by step: clone a repository, run `npm ci`, lose the host after `fsync()` returns, hand off to a second machine, fork three attempts, and sit idle for a week. Times are our measurements, and idle costs are for a workspace holding 5 GiB at us-east-1 list prices.

On the sandbox’s disk, the clone took 15.5 s and the install 10.7 s, measured on an EBS gp3 volume. The host loss takes the workspace, the second machine starts empty, forks are local copies, and keeping the sandbox up through the idle week costs about $28.

Uploading to S3 works at the same local speed, but the host loss takes everything since the last upload. The second machine and each fork download the last upload, and the idle week costs $0.03 in S3 Standard.

On a mounted bucket, the clone and the install fail, because Mountpoint can’t rename on a general purpose bucket or create symbolic links. The commit survives the host, the second machine mounts the bucket in 299 ms, forks are full copies, and the idle week costs $0.03.

On a network volume, the clone and the install run at the same speeds as the sandbox’s disk. The volume survives the host, but the handoff needs a forced detach, a machine in the same zone, and a file system check. Forks are a snapshot and three volumes that load lazily, and the idle week costs $0.37 for a 20 GiB volume.

On NFS, EFS took 81.5 s to clone React and 86.3 s to install a create-next-app project in a separate benchmark, where Archil took 9.8 s and 7.6 s. The commit survives, the handoff works, forks are full copies over NFS, and the idle week costs $0.35.

On JuiceFS with its defaults, the clone took 247.4 s and the install 161.6 s, or 29.2 s and 53.8 s in write-back mode. The commit is as safe as the metadata database, the handoff works, forks copy only metadata, and the idle week costs $0.03 in S3 plus the database.

On Archil, the clone took 19.8 s and the install 12.4 s. The commit survives. The lost client’s checkout doesn’t expire. Archil marks it orphaned, usually within a minute, and the second machine takes it over with `--force`, losing only writes that were never synced. It then mounts in 44 ms. Forks are one 13 ms checkpoint and three 33 ms branches at the median, and the idle week costs $0.03, billing the archive rate on every byte.

Every rung gets through part of the day. The fast ones lose the host or stay with one machine in one zone. The ones that share are slow on small files, can’t run git, or need a database you run.

### Where the authoritative copy lives

Each rung failed for its own reasons, but the reasons share a shape. Each design decides where the authoritative copy of the workspace lives, and which machine may answer a read or a write without asking anyone else. The following figure puts each rung where it keeps that copy.

**Where each rung keeps the truth**

Where each rung keeps the authoritative copy of the workspace. The sandbox’s disk keeps it on the machine: fast, and gone with the host. Uploading to S3 keeps it on the machine first and in the bucket later: fast, and it loses what wasn’t uploaded. A mounted bucket keeps it in the bucket: safe and shared, without rename or links. A network volume keeps it on a volume: safe and fast, for one machine in one zone. NFS keeps it on a server: safe and shared, with a round trip for each change. JuiceFS keeps the map in a database and the blocks in the bucket: complete, with a database to run.

A design that fixes the authority in one place gets part of the day, and climbing to the next rung only moves the authority somewhere else. The way off the ladder is to stop fixing it in one place: keep the truth remote and replicated, and lend the right to answer to whichever machine is doing the work. The idea is decades old, and [Where the truth lives](https://archil.com/guides/storage-for-agents/where-the-truth-lives) covers why it works and what building it takes.

## 3. Why it’s hard: where the truth lives

Source: https://archil.com/guides/storage-for-agents/where-the-truth-lives · Part of The buyer’s guide to storage for AI agents · Facts checked 2026-11-04

**The short answer:** Every storage system decides which machine may answer a read or a write without asking anyone. Speed wants that to be the machine doing the work, safety wants copies elsewhere before anything is acknowledged, and sharing wants a single authority. The design that satisfies all three keeps the truth remote and replicated and lends the right to answer to the working machine, and building it is a long list of hard problems.

An agent runs `touch notes.md`. Before that call returns, some machine has to decide that the name is free, record it, and answer. On a laptop, the kernel decides in its own memory and returns without waiting for anything. On a network file system, a server decides, and the agent waits for a round trip.

Which machine gets to decide is the most consequential choice a storage system makes. [The previous chapter](https://archil.com/guides/storage-for-agents/the-obvious-ways-to-build-it) showed each familiar design making that choice once and living with the consequences. This chapter is about the choice itself, and about the old idea that gets around it.

### Three demands on one answer

Every read and every write needs a machine that’s allowed to answer it. Call the copy that machine answers from the _authoritative copy_: when two copies disagree, it’s the one that’s right. Three demands pull on where it lives.

**Speed wants the answer here.** A local file system answers `open`, `write`, `chmod`, and `close` from the kernel’s memory, and the disk catches up later. Any call that has to ask another machine waits at least one round trip instead. A job that creates thousands of files pays that wait thousands of times.

**Safety wants copies elsewhere before anyone is told.** In the cloud, losing a machine is routine, and a stop, a termination, or a host retirement erases the drives attached to the host. (See [The work outlives the machine](https://archil.com/guides/storage-for-agents/the-work-outlives-the-machine#a-machine-built-to-be-thrown-away), and the field guide’s [Where fsync returns from](https://archil.com/guides/how-file-systems-work/where-fsync-returns-from#the-machine-as-a-failure-domain).) A write survives that only if it already exists on other machines, ideally in other availability zones. So `fsync()` has to wait for those copies, which costs at least one round trip every time a program asks for safety.

**Sharing wants one authority.** Suppose machines A and B both create `hello.txt` in the same directory. Only one of them can win, and neither knows about the other’s request. The only place that can decide is a place both requests reach, and every other machine has to ask it.

**Three demands on one answer**

Three demands on one answer. Speed: machine A’s kernel answers `open`, `write`, `fchmod`, and `close` from memory, with no round trips. Safety: A’s `fsync()` sends the write to servers in other zones before it’s acknowledged, so when A is lost, the write survives. Sharing: machines A and B both create `hello.txt`, and only an authority that sees both requests can decide. A’s file is created, and B is told the file exists.

Each demand is reasonable alone, and together they conflict. Speed puts the authority on the machine doing the work. Sharing puts it in one place that every machine can reach. Safety says that, wherever it is, no answer goes out until the data is off any single machine.

### Moving the authority

The trade-off is easiest to see by moving the authority yourself. The following explorable has three places for it: the machine where the agent works, a server, and a server that lends the working machine a _checkout_. A checkout is the exclusive right to answer for part of the file system. The distances are round trips we measured: within one AWS region, and from sandboxes on two other providers’ platforms.

**Move the authority**

An explorable with three positions for the authority, three measured server distances (0.5 ms within one region, about 10 ms from a Vercel sandbox, and about 40 ms from a Modal sandbox), and a button that loses machine A. Creating one file takes four calls: open with O\_CREAT, write 4 KiB, fchmod, and close, followed by fsync. When this machine answers, the create and fsync wait for no round trips, and an `npm ci` of 18,602 files waits on no network. If A is lost, everything written since the last copy left it is lost, including writes fsync acknowledged, and machine B can never see the files. When a server answers, each create waits for four round trips (CREATE, SETATTR, and WRITE plus COMMIT on close): 2.0 ms within the region, 40 ms from the Vercel sandbox, and 160 ms from the Modal sandbox. The `npm ci` waits for 74,408 round trips: about 37 s, about 12 minutes, or about 50 minutes. Nothing acknowledged is lost with A, and B sees every change once its cached copy expires; Linux NFS keeps a directory’s attributes for 30 to 60 s by default. B can write too, and the server decides every conflict. When the server lends A a checkout, the create, write, chmod, and close are answered from memory, and only fsync waits, for one round trip to a replicated log: 0.5 ms, 10 ms, or 40 ms. The `npm ci` waits for none of its creates; changes travel in background batches, and a final sync waits once. If A is lost, everything fsync acknowledged survives, writes never synced are lost, and the checkout stays with the lost machine until another machine takes it over. B reads what A has sent, about once a second and on every fsync, once its own cache expires, and can write under A’s checkout only after A hands it back.

With the authority on this machine, every answer is local and nothing waits. That’s the sandbox’s own disk, and it’s also a disk that uploads to object storage in the background, because until the upload finishes, the only copy is here. Lose the machine and the recent work goes with it, acknowledged or not. No other machine can see the files at all.

With the authority on a server, every change asks. That’s NFS, and it’s Amazon EFS. It’s safe and it’s shared, and a create costs four round trips. Multiply that by the files in a clone or an install, and the network sets the wall-clock time, not the disk.

With the authority on a server that lends a checkout, the working machine answers like a local disk, and the truth stays on replicated storage elsewhere. Only `fsync()` waits, once. Measured systems keep the same order, though a real server adds its own work to each round trip. In our measurements, creating a 4 KiB file and syncing it took 0.18 ms on a Vercel Drive, which acknowledges from local NVMe. It took 8.5 ms on Amazon S3 Files, an NFS server in the same zone. On Archil, where the client holds a checkout and only the `fsync()` waited, it took 4.2 ms. (Vercel Drive from a Vercel sandbox in pdx1; S3 Files and Archil from one c7gn.2xlarge in us-east-1. Medians. Methods are in our [sandbox storage benchmarks](https://docs.archil.com/reference/benchmarks/sandbox-storage#vercel-sandbox-and-vercel-drive) and [file system benchmarks](https://docs.archil.com/reference/benchmarks/file-systems#archil-vs-amazon-s3-files-vs-mountpoint-for-amazon-s3).)

The third position is the only one that does well on all three readouts. It’s also the one with the most moving parts, and the idea behind it is older than any product that uses it.

### Lending the right to answer

The Andrew File System, built at Carnegie Mellon, ran into this problem early and measured it carefully. John Howard and six colleagues described it in “[Scale and Performance in a Distributed File System](https://www.cs.cmu.edu/~coda/absdir/s11-abstract.html),” published in 1988. In the AFS prototype, each workstation checked with the server before using any cached file, and two kinds of status call made up nearly 90% of the calls its servers received. The revised AFS turned that around: “When a workstation caches a file or directory, the server promises to notify it before allowing a modification by any other workstation.” They called the promise a callback. On their benchmark, the prototype ran 70% slower than a workstation with a local disk, and the revised system 19% slower.

A callback has no end date, which is a problem when the workstation holding it crashes. In 1989, Cary Gray and David Cheriton put an expiry on the promise in “[Leases: An Efficient Fault-Tolerant Mechanism for Distributed File Cache Consistency](https://dl.acm.org/doi/10.1145/74850.74870)”: “A lease is a contract that gives its holder specified rights over property for a limited period of time.” If a holder vanishes, the server waits out the term, and then lets the write go ahead. A failure costs time, and no client reads a stale copy.

NFS version 4 built both ideas into NFS: its state lives under leases, and the server recalls what it lent with a callback. Its first specification, [RFC 3010](https://www.rfc-editor.org/rfc/rfc3010) in December 2000, introduced “the ability of the server to delegate certain responsibilities to the client.” A client holding a write delegation opens, closes, and writes a file without asking, until the server recalls it. NFSv4.1, in [RFC 5661](https://www.rfc-editor.org/rfc/rfc5661) in 2010, added directory delegations, so a client could list a directory and look names up from memory.

**What a client answers without asking**

What a client may answer without asking the server, by mechanism. AFS callbacks, 1988: reuse cached reads, open and close, and write a file’s data, but not create names; AFS sent directory changes to the server. Leases, 1989: reuse cached reads, and open and close if the lease covers the name, but writes and creates go to the server. NFSv4 write delegations, 2000: reads, opens, closes, and data writes, but not creates. NFSv4.1 directory delegations, 2010: cached reads of the directory only, because they’re read-only. A checkout of a directory tree: all four, including creating names.

Every one of these protocols stops short of creates. AFS made directory changes “directly on the server responsible for that directory.” An NFSv4.1 directory delegation is “read-only,” in RFC 8881’s words, so a client holding one still sends every create to the server. (Both quotations, and the rest of this lineage, are worked through in the field guide’s [Anatomy of a file create](https://archil.com/guides/how-file-systems-work/anatomy-of-a-file-create#callbacks-the-andrew-file-system) and [One file, many caches](https://archil.com/guides/how-file-systems-work/one-file-many-caches#leases).) An agent’s work is mostly creates: a clone, an install, a build. The last row of the figure is the step it needs.

### Truth remote, answers local

The design that follows from this history keeps the authoritative copy remote, in a log replicated across availability zones, and lends the right to answer to the machine doing the work. While machine A holds a checkout of `/repo`, nobody else may change anything under it. So A can answer creates, writes, mode changes, and renames from memory, exactly as a laptop does.

The changes still travel, but not one at a time. A sends them to the log in batches in the background, and immediately when a program calls `fsync()`. That `fsync()` returns once the log has stored the changes on other machines, so an acknowledged write never depends on A alone. When machine B wants to write under `/repo`, it asks the server. The server recalls the checkout, A sends everything it still holds and hands the checkout back, and then B holds it.

**A checkout, lent and recalled**

Machine A checks out `/repo` from the server. It creates `a.js` and `b.js`, changes a mode, and renames a file, each answered in its own memory with no round trips. `fsync()` sends the four changes in one batch to a log that’s replicated across zones before it acknowledges, which takes one round trip. A then creates `c.js`, which isn’t synced yet. Machine B asks to write under `/repo`, so the server recalls the checkout. A sends `c.js` and hands the checkout back, and B holds `/repo` and answers locally.

Each demand gets what it asked for. Speed gets local answers for the machine doing the work. Safety gets every acknowledged write on replicated storage off that machine. Sharing gets one authority per path at every moment, and the server always knows which machine it is.

The design has costs, and they’re worth knowing before you rely on it.

- **Contention.** If two machines take turns writing in the same directory, every turn is a recall, and that’s slower than asking the server each time. A checkout pays off when one machine does most of the work in a path, which is how a clone, an install, or a build behaves.
- **A holder that vanishes.** A lease waits out its term. A checkout without an expiry has to be taken over by force, and the writes the old holder never synced are gone. Either way, a lost machine costs something.
- **First reads.** A machine can answer from memory only what it has already fetched. Reading a large tree for the first time still costs a round trip per miss.

This is the design Archil is built on. Our docs call the grant a delegation, and the commands are `archil checkout` and `archil checkin`. [What Archil does, and where it falls short](https://archil.com/guides/storage-for-agents/what-archil-does) covers our implementation and its limits.

### What building it takes

I spent eight years on the Amazon EFS team. The diagram in the previous section is the part of this work that fits on a whiteboard. The following list is the part that doesn’t, and each item is a project of its own.

**Under the waterline**

An iceberg. Above the waterline: mounting a disk, reading, and writing, which is what a demo shows. Below it, eight pieces of work: handing authority over, a log replicated across zones before `fsync()` returns, recovery after a crash, the corners of POSIX, cache invalidation, copy-on-write metadata, syncing to a bucket in its native format, and isolation between tenants.

**A protocol for handing authority over.** Granting a checkout is the straightforward step. A recall has to collect every change from the holder before the server answers the next machine, or B lists a directory that’s missing A’s files. A holder that stops answering forces a choice between waiting, which needs clocks with bounded drift, and taking over, which loses its unsynced writes. The storage also has to refuse writes from a holder that lost its grant, because a lock the storage never checks protects nobody. The field guide covers [handing a directory over](https://archil.com/guides/how-file-systems-work/anatomy-of-a-file-create#answering-a-create-from-memory) and [why locks across machines are hard](https://archil.com/guides/how-file-systems-work/one-file-many-caches#why-locking-across-machines-is-hard).

**A log replicated across zones before `fsync()` returns.** Every acknowledged write crosses at least one zone boundary, so the log has to batch and append to keep `fsync()` near a single round trip. It also can’t wait for its slowest copy, or one slow server sets the pace for every disk on it. A quorum waits for enough copies instead of all of them. See [Quorums](https://archil.com/guides/how-file-systems-work/where-fsync-returns-from#quorums).

**Recovering half-flushed state after a crash.** The log is eventually folded into cheaper storage, and a server can crash halfway through. Some new objects are written, and the pointer that would make them live hasn’t moved. On restart, the server has to replay the log from the last point it knows is complete, and treat the half-written objects as garbage, not data. See [Write-ahead logs](https://archil.com/guides/how-file-systems-work/where-fsync-returns-from#write-ahead-logs) and [crash consistency in a copy-on-write tree](https://archil.com/guides/how-file-systems-work/copy-on-write#crash-consistency-for-free).

**The corners of POSIX.** Tools rely on them without saying so. `git` updates its index by renaming `index.lock` over `index`, and needs the swap to be atomic. A rename across directories must check that a directory isn’t being moved inside itself, which Linux does under one lock per file system. Across machines, the authority has to play that lock. A hard link is two names for one file. A file that’s unlinked while a process still has it open keeps its data until the last close, which the NFSv3 specification says “is impossible for a stateless server to implement.” See [Rename](https://archil.com/guides/how-file-systems-work/from-a-path-to-an-inode#rename-one-entry-one-step) and [Links, and the last close](https://archil.com/guides/how-file-systems-work/from-a-path-to-an-inode#links-and-the-last-close).

**Cache invalidation.** Every machine that cached a name, an attribute, or a page has to find out when it’s stale, by asking on a timer or by being told. Asking costs round trips and still leaves a window. Being told means the server tracks every copy and reaches each one before a write goes ahead, including copies on machines that have crashed. See [Finding out a copy is stale](https://archil.com/guides/how-file-systems-work/one-file-many-caches#finding-out-a-copy-is-stale).

**Copy-on-write metadata.** Forking a workspace in milliseconds, whatever its size, means the fork shares every block with its parent and stores only what changes. Then every write allocates new blocks and rewrites the path above them. Freeing a block requires knowing that no checkpoint or branch still points at it. See [Deciding when a block is free](https://archil.com/guides/how-file-systems-work/copy-on-write#deciding-when-a-block-is-free), and [Copies versus references](https://archil.com/guides/storage-for-agents/copies-and-references) for why it matters to agents.

**Syncing to a bucket in its native format.** If your files should land in your bucket as ordinary objects, each file operation needs an object-store translation, and some have none. A directory rename becomes a copy and a delete per object, and a reader partway through sees both directories. An object has no mode bits or owner of its own, and a hard link becomes two separate objects. See [What an object can’t hold](https://archil.com/guides/how-file-systems-work/a-file-system-on-an-object-store#what-an-object-cant-hold) and [Rename is a copy and a delete](https://archil.com/guides/how-file-systems-work/a-file-system-on-an-object-store#rename-is-a-copy-and-a-delete).

**Isolation between tenants.** A service shares its servers among many disks, which is what makes storage cheap when most of those disks sit idle. Then one tenant’s install of tens of thousands of files must not slow another tenant’s `fsync()`. A bug triggered by one disk’s state must not take down the server for every other disk on it. See [Reserved capacity and consumed capacity](https://archil.com/guides/how-file-systems-work/how-storage-is-metered#reserved-capacity-and-consumed-capacity) for why services pool.

Two of these items, copy-on-write metadata and the bucket’s format, also decide what it costs to fork a workspace, hand it to another machine, or leave the vendor that stores it. [Copies versus references](https://archil.com/guides/storage-for-agents/copies-and-references) takes those three moments in turn.

## 4. Copies versus references

Source: https://archil.com/guides/storage-for-agents/copies-and-references · Part of The buyer’s guide to storage for AI agents · Facts checked 2026-11-04

**The short answer:** Forking an attempt, handing a workspace to another machine, and leaving a vendor each cost either a copy of the data or a new pointer to it. A copy takes time and storage in proportion to the workspace, and a pointer doesn’t. On Archil-managed disks the first two are pointers, but leaving is a copy, because those disks use our internal format.

Teams that moved large files between services learned, years ago, to pass pointers instead of data. A service uploaded a 1 GB file to S3 once and put the URL in a queue message. Every service after it received the URL, not the gigabyte.

An agent’s unit of work isn’t one file. It’s a workspace: a repository, its installed packages, a SQLite database, and the notes the agent left for its next run. Three moments in a workspace’s life ask you to move it:

- an attempt needs a fork of its own to work in;
- another machine needs to pick up the work;
- you leave the vendor that stores it.

Each moment costs one of two things. A copy reads and writes every byte, so its time and storage grow with the workspace. A reference gives the new owner a pointer to data that already exists, so it costs about the same at 1 GiB as at 1 TiB. This chapter prices all three moments with one workspace, and shows why no storage format we know of makes all three references, ours included.

### Forking an attempt

Take a 100 GiB workspace and a harness that wants five attempts at a failing test. If a fork is a copy, each attempt waits while 107 GB is read and written again. At 500 MB/s, a brisk rate that we assume here rather than one we measured, that’s about 215 seconds per attempt. Five attempts in a row spend 18 minutes copying and store 600 GiB, almost all of it identical.

On an Archil-managed disk, a checkpoint takes 13 ms and a branch 33 ms at the median, and neither depends on the size of the disk. (The p95s are 19 ms and 36 ms, measured with our CLI from a c7gn.2xlarge in AWS `us-east-1`. The branch time includes opening a new TLS connection. For the method, see [Limits](https://docs.archil.com/reference/limits) and the [latency benchmark](https://docs.archil.com/reference/benchmarks/latency) in our docs.)

**Copy or branch**

A full copy and a branch of a 100 GiB workspace start at the same moment, played in real time. After 5 seconds, the copy, at an assumed 500 MB/s, has written 2,500 MB of about 107,000 MB and has about 3.5 minutes to go. The branch was ready in 33 ms, and the agent has been working in it ever since.

A branch is fast because it’s a reference. A file system already keeps a map from each file to the blocks that hold its data. A copy-on-write branch starts a second map that points at the same blocks, so no data moves when it’s made. When the attempt rewrites a block, the storage puts the new data in a new block and repoints only the branch’s map.

**Shared blocks, then one new one**

A disk’s map points at four blocks, `A` to `D`. A checkpoint freezes the map. A branch starts with its own map pointing at the same four blocks, so no data moves. When the branch rewrites `C`, the storage writes a new block, `C′`, and repoints only the branch. The parent still reads `C`, and the two keep sharing `A`, `B`, and `D`.

The reference changes what a fork costs, not only how long it takes. A branch stores what its attempt changed, so five attempts that each rewrite 2 GiB store about 110 GiB in all, not 600 GiB. A harness that pays that little forks more often. It can checkpoint after every turn that succeeds, and branch back to the last good checkpoint when a turn fails. For how snapshots and writable clones work block by block, and why they survive a power cut, see the field guide’s chapter on [copy-on-write](https://archil.com/guides/how-file-systems-work/copy-on-write).

### A fork of a copy that lags

A reference is only as current as the data it points at. Some storage acknowledges `fsync()` once a write reaches the sandbox’s local drive, and uploads it to durable storage later. Its forks are made from the durable copy, which trails what the program was told is safe. [The obvious ways to build it](https://archil.com/guides/storage-for-agents/the-obvious-ways-to-build-it#an-upload-when-the-agent-stops) covers that design. Here, the question is only what a fork is made from.

We tested this on Vercel Drives. A sandbox wrote a 4 KiB file and a 64 MiB file, called `fsync()`, and the script then forked the Drive and read the fork from a new sandbox. Every fork requested in the first second after `fsync()` returned lacked the new file: 29 of 29. At 5 s and again at 30 s, 9 of 10 forks still lacked it. From 60 s on, every fork had it. (The script and every result are in [Vercel Drive fsync durability](https://docs.archil.com/reference/benchmarks/vercel-drive-durability) in our docs. The test ran in Vercel’s `pdx1` region, with 8 to 11 trials per delay.)

Vercel’s docs describe a Drive fork two ways: as “copying the drive’s data”, and as a drive that inherits its parent’s data “but stores its own changes independently”. (Both phrases are from Vercel’s [Drives](https://vercel.com/docs/sandbox/concepts/drives#forking-a-drive) documentation.) We didn’t time Drive forks, so we can’t say which one you pay for. Either way, the fork came from the copy that lagged.

**What the fork is made from**

Two forks requested right after `fsync()` returns. On storage that acknowledges `fsync()` from the sandbox’s local drive, the write reaches the store that forks are made from later, in the background, so the fork lacks the file. In our test of Vercel Drives, every fork requested in the first second lacked it, 29 of 29, and forks were intact only from 60 s on. On Archil, `fsync()` returns after the write is replicated across availability zones, and the checkpoint and branch point at that replicated state, so the branch has the file.

Our branches don’t have that gap, because a checkpoint is taken from the same replicated state that `fsync()` waits for. From inside a Vercel sandbox, checkpointing a disk, branching it, and mounting the branch took about 0.9 s end to end. A 1 GiB file was intact in the branch. Call `fsync()` on the files a turn wrote before you checkpoint it.

### Handing the workspace to another machine

The machine that finishes a task often isn’t the one that started it. Moving the workspace between them is the second moment.

The copy version is the sync-to-S3 design from [the obvious ways to build it](https://archil.com/guides/storage-for-agents/the-obvious-ways-to-build-it). Machine A packs the workspace and uploads it, and machine B downloads and unpacks it. For the 100 GiB workspace at the assumed rate, that’s about 215 seconds up and 215 down, around 7 minutes before B runs a command. B also gets the workspace as it was when the upload started, so anything A wrote afterward stays behind.

The reference version hands over the disk itself. A runs `archil unmount`, which flushes what A has written, and B mounts the same disk in 50 ms at the median. Nothing is copied up front. B fetches each block the first time it reads it: about 0.5 ms for 4 KiB served from our servers’ cache, or 10 to 30 ms when the block has to come from S3. (Both read times are from our [performance](https://docs.archil.com/details/performance) page; the first is the median for a 4 KiB read that misses the client’s cache.) A machine that reads 2 GiB of a 100 GiB workspace moves about 2 GiB.

**Two ways to hand off a workspace**

Two ways to hand a workspace from machine A to machine B. Copying it across uploads the whole workspace to a bucket and downloads it again before B can start: for 100 GiB at an assumed 500 MB/s, about 7 minutes. A file A wrote after the upload began never reaches B. Mounting the same disk takes one unmount on A and one mount on B, 50 ms at the median, and B then fetches only the files it opens.

In the default exclusive mode, one machine writes to a disk at a time, so B mounts it for writing once A has let go. Shared mode lets machines check out separate paths of one disk. [Where the truth lives](https://archil.com/guides/storage-for-agents/where-the-truth-lives) covers how that right to write moves between machines.

A reference hand-off isn’t unique to us. An EBS volume can be detached from one instance and attached to another, as long as both are in the same availability zone. (From AWS’s guide to [attaching a volume](https://docs.aws.amazon.com/ebs/latest/userguide/ebs-attaching-volume.html): “You can attach volumes to instances that are in the same Availability Zone only.”) A Vercel Drive mounts read-write in one sandbox at a time. In our test, a Drive handed over by stopping the writing sandbox had the file intact in 10 of 10 trials. What differs is how far the pointer reaches: one zone, one vendor’s sandboxes, or any Linux machine that runs our client.

### What a branch shares

A branch shares blocks with its parent, and it shares the parent’s access rules too. Three things follow for Archil’s branches today:

- **Space comes back only from what a branch owns.** An attempt that rewrites its whole workspace stores as much as a copy. Deleting a branch would free only the blocks that no other branch or checkpoint still reads.
- **You can’t delete an individual branch or checkpoint.** The data an abandoned attempt wrote stays on the disk. If you fork thousands of times a day, plan for that.
- **A branch isn’t a security boundary.** Credentials cover the whole disk, every branch and checkpoint included. Give each untrusted tenant its own disk, and keep branches for attempts that may see each other’s files.

**What forks use**

An interactive space meter for one 100 GiB checkpoint and several writable forks of it. Every fork references 100 GiB. The disk stores the checkpoint once plus what each fork rewrote: four forks that have each rewritten 10% store 140 GiB, against the 500 GiB the five trees show. Each fork uses 10 GiB, which deleting it would free. If the forks rewrote different blocks, every block of the checkpoint is still read by some fork, so the checkpoint uses nothing and deleting it frees nothing. If they all rewrote the same blocks, only the checkpoint still reads the old versions, and deleting it frees 10 GiB. A fork that has rewritten 100% of the disk costs as much as a full copy.

### Leaving a vendor

The third moment happens once, if at all, and it’s the one buyers check least. Whether leaving is a copy or a reference depends on the format your files are in on the day you go.

If they’re ordinary objects at their paths, in a bucket you own, leaving is a reference. You stop writing, let the last sync land, and point the next system at the bucket. Anything else is a copy: you read every byte back through the product that wrote it, and write it somewhere new.

Leaving an Archil-managed disk, the default, is a copy. It’s stored in our internal format, in a bucket we operate. You export it through the disk’s own S3 API, or mount it and copy it with `rsync -aH` to keep owners and links. Exporting 500 GB costs $35 in transfer, and takes about 17 minutes at the assumed rate.

A disk synced to your own bucket in S3, GCS, R2, or another S3-compatible store is a reference. Each file is already an object at its path, and deleting the disk leaves the objects where they are. The bucket doesn’t hold the whole file system, though. Mode bits, owners, and extended attributes stay in Archil. A symbolic link arrives as a text file holding its target, and a hard link as a full copy.

**What your bucket sees**

The figure lists one agent workspace (a Git repository, a shell script with mode 755, a symbolic link, an installed package, a Parquet file, and a SQLite database) as each kind of storage leaves it in object storage.

An Archil-managed disk, the default, leaves nothing in your bucket: it’s stored in Archil’s internal format, in a bucket Archil operates, and you list it through the disk’s own S3 API, where the symbolic link isn’t readable. A bucket-synced Archil disk leaves every file as an ordinary object at its path; the script’s mode stays in Archil, and the symbolic link becomes a text file holding its target. Mountpoint for Amazon S3 leaves ordinary objects too, but can’t write the Git repository, the symbolic link, or the SQLite database, and can’t change modes. Amazon S3 Files leaves ordinary objects as new versions, keeps the mode as object metadata, and stores the link as an object whose encoding AWS doesn’t document. JuiceFS leaves numbered block objects under `chunks/`, with names and modes in its metadata engine. A block volume leaves a snapshot you can’t read as files, and a sandbox vendor’s disk leaves nothing in your bucket at all.

For how each of these designs maps files onto objects, see the field guide’s chapter on [a file system on an object store](https://archil.com/guides/how-file-systems-work/a-file-system-on-an-object-store).

### The map behind every cheap fork

A reference needs something to point through. A copy-on-write branch is a second map over shared blocks, so the storage has to keep that map, and it has to name blocks by something other than their paths. A bucket of ordinary objects has no such map: each object’s key is its path. Two trees that share most of their files can’t both live at the same keys. A fork there is either a copy under a new prefix, or a map kept somewhere else, which is a format of its own.

The products in this guide that fork without copying each keep that map, and the map is what an exit has to carry out. JuiceFS clones a directory by copying only its metadata, which lives in an engine you run, and its bucket holds numbered blocks you can’t read as files. AWS keeps EBS snapshots incrementally, in S3 buckets you can’t access directly.

We’re no exception. Checkpoints and branches work on Archil-managed disks only, and those disks use our internal format. A bucket-synced disk keeps your files at their paths, and it can’t checkpoint or branch. So you choose per disk. A managed disk suits agents that fork often. A bucket-synced disk suits files that other tools must read where they land, or a workspace whose exit matters more than its forks. Moving from one kind to the other is a copy: create the new disk and copy the data into it.

**Copy or pointer, three times**

An interactive ledger for one workspace and three moments: forking an attempt, handing the workspace to another machine, and leaving. Copy times use an assumed 500 MB/s, and pointer times are our measured medians. For a 100 GiB workspace on a disk that lives on one machine, all three are copies: about 3.6 minutes to fork, about 7 minutes to hand off by upload and download, and about 3.6 minutes to export. On an Archil-managed disk, a fork is a pointer (a 13 ms checkpoint and a 33 ms branch), a hand-off is a pointer (a 50 ms mount, after which only the files read move), and leaving is a copy of the whole workspace, $7.52 in transfer for 100 GiB. On a bucket-synced Archil disk, a fork is a copy, because checkpoints and branches need a managed disk. A hand-off is a pointer, and so is leaving: nothing moves, although modes and links stay behind.

To find out which kind of fork a vendor sells, time a fork of a small disk and of your largest one. A fork whose time grows with the disk is a copy, whatever it’s called. Then fork the moment `fsync()` returns and compare checksums, and ask in writing what format your files will be in on the day you leave. [Can you fork it?](https://archil.com/guides/evaluating-storage/can-you-fork-it) and [Where does your data live?](https://archil.com/guides/evaluating-storage/where-does-your-data-live) have the scripts for both tests.

Between those three moments, the workspace mostly sits idle. [What you pay for](https://archil.com/guides/storage-for-agents/what-you-pay-for) covers what the idle time costs.

## 5. What you pay for

Source: https://archil.com/guides/storage-for-agents/what-you-pay-for · Part of The buyer’s guide to storage for AI agents · Facts checked 2026-11-04

**The short answer:** Storage bills for one of three things: capacity you reserve, the machine the files live on, or what you use. An agent’s workspace waits far more than it works, so which of those you pay for matters more than any rate. Billing by use needs storage that stands apart from both a reserved size and the machine.

Take one agent workspace that holds 5 GiB, whose agent works about 2 hours a day in one session and touches half its data. The following table prices a month of it three ways:

| What the bill counts          | Example                                             | Per month   |
| ----------------------------- | --------------------------------------------------- | ----------- |
| Capacity you reserve          | A 20 GiB EBS gp3 volume                             | $1.60       |
| The machine the files live on | An E2B sandbox with 2 vCPUs and 4 GiB, kept running | about $121  |
| What you use                  | An Archil-managed disk                              | about $0.20 |

The rates don’t explain the spread. Per GiB, our active rate is higher than gp3’s. What differs is what each meter counts during the 22 hours a day the agent isn’t working. Those hours are most of an agent’s life, as [our measurement of 384 agent loops](https://archil.com/guides/storage-for-agents/the-work-outlives-the-machine#where-an-agents-hours-go) showed.

This chapter traces each meter back to the design that produces it, shows where the cheapest-looking option stops being cheapest, and prices a fleet. For how metering works in general, including GB-months, pooling, and storage classes, see [How storage is metered](https://archil.com/guides/how-file-systems-work/how-storage-is-metered). (Every price here is a list price in us-east-1 (N. Virginia), before discounts, free tiers, taxes, and plan minimums. Our Developer plan bills at least $10 a month, and usage counts toward that minimum. AWS bills a GB as 2³⁰ bytes, the same as a GiB, so this chapter applies AWS’s per-GB rates per GiB.)

### The meter follows the architecture

Each design bills for the thing it has to keep for you while you’re away.

A provisioned volume is a block device of a size you choose. The provider promises you that space and its performance, so it bills them as promised. EBS gp3 costs $0.08 per GB-month, charged, in AWS’s words, “by the amount of GB you provision”. The volume couldn’t bill you for less if it tried. A block device stores blocks, and it doesn’t know which of them hold files.

A sandbox’s own disk is part of the machine. The files exist for as long as the machine runs, or for as long as the vendor keeps a stopped copy of it. The price of keeping the files is the price of keeping the machine, which bills CPU and memory by the second.

To bill by use, the storage has to stand apart from both. It can’t be a size that someone reserved, and it can’t live on the machine that runs the agent. A service that pools many customers’ data, the way S3 and EFS do, can bill the bytes you store. The provider carries the empty space across all its customers, whose peaks rarely line up. A service that also tracks which data is in use can bill hot data differently from cold.

That separation is the design [the chapter on where the truth lives](https://archil.com/guides/storage-for-agents/where-the-truth-lives) reached for other reasons. The authoritative copy lives off the machine, and the machine borrows the right to answer through a checkout. The same split is what lets a meter count use.

**Where the meter sits**

Three designs hold the same 5 GiB workspace, drawn as 1 GiB cells. The agent writes its files and then waits. A provisioned volume reserved 20 GiB and holds 5 GiB of data, and its meter bills all 20 GiB, full or empty, every hour: the provider set the space aside, and the volume can’t tell which blocks hold files. A sandbox with 2 vCPUs and 4 GiB keeps the files on its own disk, and its meter bills the whole machine for every hour it stays up, so keeping the files means keeping it running. With decoupled storage, the machine stops once the agent is done. The meter bills the 2 GiB the agent touched in the last hour at the active rate, and the whole disk at a low rate. The quantities are an example.

### Reserved capacity

A volume bills from the moment you create it “until you release the storage”, attached to a machine or not, full or empty. For agents, that has two consequences.

First, you size each volume for the largest workspace you expect, not the typical one. EBS can grow a volume but can’t shrink it. Suppose most workspaces hold 5 GiB and a few reach 20 GiB. The fleet either provisions 20 GiB everywhere or runs its own machinery to grow volumes as they fill. (From [Amazon EBS pricing](https://aws.amazon.com/ebs/pricing/) and [Modify an Amazon EBS volume](https://docs.aws.amazon.com/ebs/latest/userguide/ebs-modify-volume.html): “You can’t decrease the volume size.”)

Second, the empty part of a volume costs as much as the full part. Divide the bill by the data stored, and a volume that’s a quarter full costs four times its list rate per stored GiB. That’s more than EFS Standard charges. A fork made by copying is another volume, billed at its full size. [The chapter on copies and references](https://archil.com/guides/storage-for-agents/copies-and-references) covers what a copy costs in time.

**What a stored gigabyte costs on a rented volume**

A 100 GiB gp3 volume costs $8.00 a month however full it is. Divide that by the GiB actually stored to get the price per stored GiB: at 100% full it is the list price, $0.08; at 25% full it is $0.32; at 10% full it is $0.80. EFS Standard costs $0.30 per stored GiB and S3 Standard $0.023, at any fill. The volume costs more per stored GiB than EFS Standard below about 27% full, and never gets as low as S3 Standard.

### The machine

E2B bills $0.000014 per vCPU-second and $0.0000045 per GiB-second for as long as a sandbox runs, used or not. A 2 vCPU, 4 GiB sandbox kept up for a 730-hour month gives the table’s second row. Almost none of that pays for the files. Priced in storage, an idle machine hour is expensive. (From [E2B pricing](https://e2b.dev/pricing). Other providers’ idle hours, with links to each price list, are in [What does an idle agent sandbox cost per hour?](https://archil.com/article/what-does-an-idle-agent-sandbox-cost-per-hour))

**One idle hour, priced in storage**

One idle hour of a 2 vCPU, 4 GiB sandbox at E2B’s list price, about $0.17, buys 7.2 GiB-months of S3 Standard at $0.023 each, or 2.1 GiB-months of EBS gp3. A 5 GiB workspace kept in S3 Standard for a month costs $0.115, less than that one hour.

Sandbox vendors have started to bill the machine closer to use, and they deserve credit for it. Vercel Sandbox bills memory for as long as a sandbox is provisioned, but CPU only while it’s actively used. Fly.io Sprites bill the CPU and memory actually used, and an idle Sprite moves to a non-billed state within seconds. Daytona bills a stopped sandbox for its disk only. (From [What does an idle agent sandbox cost per hour?](https://archil.com/article/what-does-an-idle-agent-sandbox-cost-per-hour), which links each provider’s pricing page.)

Each of these shrinks the machine’s share of the idle bill. The files still belong to the sandbox, though, kept in that vendor’s storage and reached through it. Whether they survive losing the machine is the question [the chapter on the obvious designs](https://archil.com/guides/storage-for-agents/the-obvious-ways-to-build-it) asked of each one.

### Use

Storage billed by use comes in two grades.

The first grade bills the bytes you store. EFS Standard costs $0.30 per GB-month and S3 Standard $0.023, so an empty workspace costs nothing. A full one bills every hour, though, used or not. EFS moves files to cheaper classes after 30 days without access, but a workspace the agent opens every day keeps restarting that clock. (From [Amazon EFS pricing](https://aws.amazon.com/efs/pricing/), [EFS lifecycle management](https://docs.aws.amazon.com/efs/latest/ug/lifecycle-management-efs.html), and [Amazon S3 pricing](https://aws.amazon.com/s3/pricing/).)

The second grade bills the data in use. Archil charges $0.20 per GiB-month of active data. Data is active if it was read in about the last hour, or written and flushed to the disk’s data source in about the last hour. Active data is the data in our high-speed layer, in front of the object storage that holds the rest. The meter counts nothing else. We measure it every minute and bill the time-weighted average. Reading metadata with `ls`, `stat`, or `find` doesn’t make file data active.

Under the active meter, the whole disk bills at $0.025 per GB-month on an Archil-managed disk. On a disk synced to your own bucket, there’s no archive charge, and your cloud provider bills the bucket at its own rate. (From [Metering](https://docs.archil.com/concepts/metering) and [Pricing & Billing](https://docs.archil.com/administration/billing) in our docs. Branches and checkpoints are copy-on-write, so data they share bills once.)

**Four meters, one day**

The figure follows one agent workspace through a day with three working sessions, about 4 hours of tool calls in all. A sandbox kept running bills all 24 hours, and a provisioned volume bills its full size for all 24. Archil’s active meter counts the data the agent touched, during each session and for up to an hour after, about 7 hours in all. All day, the whole disk also bills at the archive rate, or at your bucket’s rate on a disk synced to your own bucket.

For the example workspace, the active meter runs about 3 hours a day: the 2-hour session, plus the hour after it. Half of 5 GiB, active for 3 hours in 24 at the active rate, comes to about $0.06 a month. Archive on the whole disk, 5 GiB or about 5.37 GB at the archive rate, adds about $0.13. Together they make the table’s third row.

Our price list doesn’t say whether archive also bills data that’s active. This chapter takes the reading that costs the most: archive on every byte, all month, on top of the active rate. Every number and figure in this chapter uses that assumption, including both calculators. If archive applies only to inactive data, a managed disk costs slightly less than shown.

### Where a volume costs less

Billing by use wins when the agent waits. When it doesn’t wait, the higher rate shows. Our active rate is 2.5 times gp3’s, so a workspace that’s busy all day costs more on Archil than on a volume sized to fit it.

Suppose the agent touches all 5 GiB of its workspace around the clock. That meters $1.00 a month active, plus archive, against $0.40 for a 5 GiB gp3 volume. With one session a day, a tightly sized volume costs less once the agent touches all its data for more than about 5 hours. If your agents work most of the day on data that fits a size you know in advance, price a volume first.

Data that nobody touches costs less still in plain object storage. For that reason, a disk synced to your own bucket leaves cold data at your bucket’s rate instead of ours.

**One workspace, two meters**

The figure prices one 5 GiB workspace with one working session a day, by the hours the agent works. A gp3 volume costs the same at any hour: $0.40 a month at 5 GiB, $0.80 at 10 GiB, and $1.60 at 20 GiB. Archil’s cost rises with the hours its active meter runs, the session plus an hour, on the share of data touched, and archive adds about $0.13 on every byte all month. When the agent touches all its data, a 5 GiB volume costs less above about 5.4 hours of work a day, a 10 GiB volume above about 15 hours, and a 20 GiB volume never. When it touches half its data, a 5 GiB volume costs less above about 11.8 hours a day.

### Compute billed by use

The machine can follow the storage. Once the files live apart from the sandbox, nothing has to stay up between tool calls. A serverless tool call runs one command in a short-lived microVM with the disk mounted, returns the output, and shuts down.

**A serverless tool call**

The agent’s harness sends `npm test` to the disk. A microVM starts with the disk mounted, runs the command, returns its output, and shuts down. The call bills $0.00005. Until the next call, nothing runs and no compute bills, and the files stay on the disk.

We charge $0.05 per 1,000 calls, with execution included. Our own agents made about 160 tool calls per hour of activity, so 2 hours a day for a month is about 9,700 calls, or $0.49.

Per-call pricing has limits. At about 3,300 calls an hour, close to one a second sustained, it costs the same as an idle E2B sandbox of the same size. Each call can run for up to 5 minutes. A dev server or a Python kernel has to stay up between calls, so for those, use a persistent sandbox with the same disk mounted. (From [Serverless execution](https://docs.archil.com/compute/serverless-execution), [Limits](https://docs.archil.com/reference/limits), and [Pricing & Billing](https://docs.archil.com/administration/billing). Each call gets 2 vCPUs and 4 GiB. Serverless tool calls and sandboxes run in AWS regions only.)

### Pricing your own fleet

The calculator does the same arithmetic for a fleet and splits each bill into the hours the agent works and the hours it waits. Start from your own agents’ idle share. [What does it cost when nothing is running?](https://archil.com/guides/evaluating-storage/what-does-it-cost-when-nothing-is-running) has a short script that replays your tool-call timestamps against an auto-stop and prints it.

**Price a fleet of workspaces**

The calculator starts with 1,000 workspaces of 5 GiB each on 20 GiB volumes. Each agent works 2 hours a day in one session and touches half its data. At those settings, EBS gp3 costs $1,600 a month, EFS Standard $1,500, and S3 Standard alone $115, and about 92% of each falls in the hours the agent waits. Archil costs about $197 a month on managed disks and $178 synced to your own S3 bucket. A sandbox kept running for each workspace so its disk survives adds about $120,900 a month in compute. One that stops after 15 idle minutes adds about $11,300, before whatever the vendor charges to keep its stopped copy. It leaves EFS in Standard, before throughput charges.

Then check a real bill. Create one workspace with each vendor you’re considering, us included, leave it untouched for 48 hours, and read every line item that kept accruing.

## 6. What Archil does, and where it falls short

Source: https://archil.com/guides/storage-for-agents/what-archil-does · Part of The buyer’s guide to storage for AI agents · Facts checked 2026-11-04

**The short answer:** Archil gives each agent workspace one disk that any machine can mount, replicated across availability zones before `fsync()` returns, with checkpoints and branches in milliseconds at any size, billed for the data in use. It’s slower than Amazon EFS on large sequential writes, and shows other machines a new file only when their cache expires or they ask for it.

So far this guide has argued from the problem. This chapter is about what we built: what a team gets on the first day, in commands and measurements, and then where Archil is slower or narrower than you might need.

### One disk through a day

An Archil disk is a file system that any Linux machine can mount. You create it with one API call, and it stays put while the machines that use it come and go. Here is one agent’s day on one disk.

**One disk, five commands**

Five commands drive one Archil disk through an agent’s day. In sandbox A, `archil mount myorg/work /work` mounts the disk, and `npm ci && npm test` installs and tests a project. Each `fsync()` is replicated across availability zones before it returns. `archil checkpoints create /work ready` checkpoints the whole disk, and `archil branches create` makes three branches of that checkpoint, `try-1` to `try-3`. Machines B, C, and D each mount one branch with `archil mount --branch` and carry on from the checkpoint. Sandbox A stops, and nothing it saved is lost. At the median, a mount took 50 ms, a checkpoint 13 ms, and a branch 33 ms, whatever the disk’s size.

Written out in full, the day is a handful of lines:

```bash
## Sandbox A: mount the disk and work in it.
sudo archil mount myorg/work /work --region aws-us-east-1
cd /work && npm ci && npm test

## Mark a point to return to, then fork it once per attempt.
archil checkpoints create /work ready
archil branches --region aws-us-east-1 create dsk-0123456789abcdef try-1 --from-checkpoint ready

## Machine B: mount the branch and carry on.
sudo archil mount myorg/work /work --region aws-us-east-1 --branch try-1
```

Each step replaces a rung of the do-it-yourself ladder in [The obvious ways to build it](https://archil.com/guides/storage-for-agents/the-obvious-ways-to-build-it). The following table gives our medians, measured from a c7gn.2xlarge in AWS `us-east-1`. (From [Limits](https://docs.archil.com/reference/limits#latency), which links the script that produced them.)

| Step                        | Median                          | What it replaces                                                                                                       |
| --------------------------- | ------------------------------- | ---------------------------------------------------------------------------------------------------------------------- |
| `archil mount`              | 50 ms                           | Downloading the workspace before the agent starts, or detaching a volume from one machine and attaching it to the next |
| A 4 KiB write with `O_SYNC` | 2.0 ms, replicated across zones | An upload after the agent finishes, and the work lost if the machine dies first                                        |
| `archil checkpoints create` | 13 ms, at any size              | A tarball of the workspace, uploaded in a time that grows with its size                                                |
| `archil branches create`    | 33 ms, at any size              | A full copy of the workspace for every attempt                                                                         |

One disk takes up to 5,000 connected clients, so the next step can run on one machine or a fleet. Checkpoints and branches work on Archil-managed disks, the default. [Copies and references](https://archil.com/guides/storage-for-agents/copies-and-references) shows why a pointer beats a copy at this scale.

### Speed and safety, measured

The day depends on two properties that usually trade against each other.

Speed comes from where the work happens. The machine that holds a checkout answers `open()`, `write()`, and `rename()` from its own memory, as a local disk does, and sends its changes to our servers about once a second. [Where the truth lives](https://archil.com/guides/storage-for-agents/where-the-truth-lives) explains how that stays correct when other machines share the disk. On one EC2 host, `npm ci` of 18,602 files, timed until every write was durable, took 12.4 s on Archil and 10.7 s on the instance’s own EBS volume.

Safety comes from what `fsync()` means. When it returns, the write is replicated across availability zones, and we design for 99.999% durability for recent writes before they reach S3. If a sandbox dies, every write that `fsync()` covered is on the disk. Writes the program never synced, and that the client hadn’t sent yet, can be lost, the way unsynced writes on a local machine are lost in a power cut.

A dead sandbox’s checkout doesn’t expire. Archil marks it orphaned, usually within a minute, but doesn’t release it. No other machine can write there until one takes it over with `archil mount --force` or `archil checkout <path> --force`. Taking it over discards the writes the old holder hadn’t sent, much as force-detaching an EBS volume skips its cache flush. (From [Forcing ownership](https://docs.archil.com/concepts/sharing-disks#forcing-ownership) and [Release a stale lock](https://docs.archil.com/guides/platforms/troubleshooting#release-a-stale-lock) in our docs. `--force` revokes every other client’s checkouts, including running ones. Revoking only orphaned checkouts through the SDK can’t interrupt a client that’s still connected.)

We also benchmarked Archil against Amazon EFS, AWS’s managed NFS file system, from one host. (The host, versions, and commands are in [File systems on one host](https://docs.archil.com/reference/benchmarks/file-systems#archil-vs-amazon-efs), and all 15 results are in [Benchmark against Amazon EFS](https://docs.archil.com/details/comparison#benchmark-against-amazon-efs).)

**Archil and Amazon EFS, test by test**

Nine of the 15 tests we ran against Amazon EFS, including all three that EFS won, from one c7gn.2xlarge in us-east-1 with Archil client v0.9.2, medians of 3 runs. Archil was faster on a git checkout of an old react tag (1.06 s against 46.9 s, 44 times), 10,000 SQLite transactions with `synchronous=FULL` (23.9 s against 523 s, 22 times), `npm ci` of create-next-app (7.6 s against 86.3 s, 11 times), a git clone of react (9.8 s against 81.5 s, 8.3 times), cold 4 KiB random reads (4,086 against 1,298 IOPS, 3.1 times), and cold sequential reads (1,197 against 1,020 MB/s, 1.2 times). EFS was faster on a bulk insert of 1 million SQLite rows in one transaction (1.7 s against 11.0 s, 6.5 times), 4 KiB random writes followed by `fsync()` (2,976 against 953 IOPS, 3.1 times), and a 4 GiB sequential write followed by `fsync()` (590 against 194 MB/s, 3.0 times).

Archil wins where agents spend their time: many small files, many metadata operations, and transactions that each wait for `fsync()`. EFS wins when one client writes a lot of data at once, which is the first item in [Where Archil falls short](#where-archil-falls-short).

### Your bill and your bucket

Archil bills for the data you use, not for a size you reserve. Data counts as active for about an hour after it’s read or written, and the whole of an Archil-managed disk also bills at a lower archive rate. Branches and checkpoints bill the data they share once. [What you pay for](https://archil.com/guides/storage-for-agents/what-you-pay-for) gives the rates and works through what those meters do to the bill for a fleet of mostly idle agents.

A disk can instead sync to a bucket you own on S3, GCS, R2, or another S3-compatible store, on any paid plan. Each file lands as an ordinary object at its path, tens of seconds to minutes after it’s safe on Archil. DuckDB, Spark, or the AWS CLI can then read it without our software. You pay your provider for the bucket, and Archil for the data in use.

**Safe now, in your bucket later**

The agent writes `events.parquet` and calls `fsync()`. Archil replicates the write across availability zones, and then `fsync()` returns: the data is safe. Tens of seconds to minutes later, the object lands in your bucket at `data/events.parquet`, where DuckDB on any machine queries it with no Archil client or account.

### Many machines on one disk

Any number of machines can write one disk at once, including the same files. How they share it depends on the mode each one mounts in. Exclusive mode, the default, gives one client the whole disk, and other clients can mount it read-only. Shared mode leaves the disk read-only on every client until it runs `archil checkout <path>`, and opening or creating a file checks that file out automatically. Conditional mode lets any client write wherever nobody holds a checkout, at the speed of a remote file system, and the last write to reach the disk wins. In exclusive and shared mode, a second writer to a checked-out path gets an error instead of silently replacing the first writer’s work.

**What client B can do, by mode**

Client A has checked out `/repo` (in exclusive mode, the whole disk). In exclusive mode, B mounts with `--read-only`: it can read everything, and every change fails as read-only until A unmounts. In shared mode, B can read, can’t change `/repo/plan.md` until A checks it in, can write a file nobody holds (opening it checks it out until B closes it), can’t delete a file until it checks out the parent directory, and writes at local speed after `archil checkout /data`. In conditional mode, B can read, changes and deletes outside `/repo` run on the server at remote-file-system speed with the last write winning, and a write to `/repo/plan.md` can return and then fail when it’s sent: B’s mount turns read-only and its unsent writes are lost.

For which mode fits a fan-out, a reviewer beside a writer, and a handoff, see [Can many machines use it at once?](https://archil.com/guides/evaluating-storage/can-many-machines-use-it-at-once)

### Where Archil falls short

These are the limits we’d want to hear about before choosing storage, and what to do about each.

**Large sequential writes are slower than on EFS.** On a 4 GiB sequential write with `fsync()`, EFS wrote 590 MB/s and Archil 194 MB/s. We have more work to do on throughput. If your agents mostly write large files in one stream, such as model weights or dataset exports, time that workload on both before you choose.

**A new file takes about 30 s to appear on another machine by default.** Each client answers lookups from its cache until the entries expire, so a reader can miss a file that another machine created a moment ago. Have the reader run `archil invalidate-cache` on the mount before it reads. In our test, it then saw the new file in about 70 ms.

**Managed disks use our internal format.** The objects behind an Archil-managed disk can’t be read without Archil. Export them over the disk’s S3 API or by copying from a mount, or use a disk synced to your own bucket wherever other tools need the files.

**You can’t delete individual branches or checkpoints today.** Each one keeps the data it changed, on your bill, for as long as its disk exists. If your agents fan out many short-lived attempts, count what each attempt writes as storage you keep.

**In our public regions, sandboxes and serverless tool calls run in AWS only.** Disks in GCP `us-central1` work for storage, but `disk.exec` there returns `Exec is not enabled`, and Archil sandboxes don’t run there. Create a disk in an AWS region if your agents will run commands through Archil, or mount it from your own compute.

### How to evaluate us, and anyone else

The measurements in this chapter can be checked. Our [limits page](https://docs.archil.com/reference/limits) links the script behind its latency figures, which you can run on your own account. The EFS benchmark lists its host, versions, and commands, including for the tests EFS won. Run the workload your agents actually run, timed until its writes are durable, on us and on anything else you’re considering.

[Evaluating storage](https://archil.com/guides/evaluating-storage) turns the argument of this guide into one page per buyer’s question, each with a test to run against any vendor. [Its scorecard](https://archil.com/guides/evaluating-storage#the-scorecard) rates every architecture in this guide on those questions, Archil included, and each question’s page has the test that checks the rating.