# 3. How fast is it on small files?

Source: https://archil.com/guides/evaluating-storage/how-fast-is-it-on-small-files · Part of Evaluating storage for agents · Facts checked 2026-11-04

**The short answer:** Agent work writes thousands of small files, so its wall-clock time is set by how many network round trips each file costs, not by throughput. Storage that answers file operations on the machine finishes an install within seconds of a local disk. Storage that asks a server about every file takes several times as long.

Installing the dependencies of one Next.js app with `npm ci` writes 18,602 files. We ran that install on one EC2 instance with several file systems mounted side by side, and stopped the clock only when every write was durable. On the instance’s own EBS volume, it took 10.7 s. On Archil, it took 12.4 s. On Amazon S3 Files, which serves an S3 bucket over NFS, the same install took 71.4 s.

The host, the package cache, and the bytes written were the same each time. What differed was how often each file had to wait for a server. An agent that clones a repository, installs its dependencies, and runs its tests pays that difference at every step, and wall-clock time is what you and your users notice.

**npm ci, timed until durable**

`npm ci` of a Next.js app writes 18,602 files. On one EC2 host, timed until every write was durable, it took 10.7 s on the instance’s own EBS volume, 12.4 s on Archil, and 71.4 s on Amazon S3 Files. The animation plays the three runs side by side, 10 times faster than they ran.

## Two jobs, five file systems

The pattern holds for `git clone` and for other designs. On the same host, a clone of the Django repository took 15.5 s on local disk, 19.8 s on Archil, and 107.5 s on S3 Files. JuiceFS, which keeps its metadata in a database (here Redis on ElastiCache, in the same Availability Zone), took 161.6 s for the install and 247.4 s for the clone with its defaults. Its write-back mode stages writes on local disk and uploads them later. Timed until the upload finished, it took 53.8 s and 29.2 s.

**Two jobs, timed until durable**

Two developer jobs on one AWS c7gn.2xlarge in us-east-1, each timed until its writes were durable, medians of 3 runs. `npm ci` of a Next.js app (18,602 files): local EBS gp3 10.7 s, Archil 12.4 s, JuiceFS in write-back mode 53.8 s, Amazon S3 Files 71.4 s, JuiceFS with defaults 161.6 s. `git clone` of Django: local EBS gp3 15.5 s, Archil 19.8 s, JuiceFS in write-back mode 29.2 s, Amazon S3 Files 107.5 s, JuiceFS with defaults 247.4 s.

A separate benchmark against Amazon EFS shows the same shape. Cloning React (7,260 files) took 9.8 s on Archil and 81.5 s on EFS. `npm ci` of a create-next-app project took 7.6 s on Archil and 86.3 s on EFS. That benchmark stopped the clock when each command returned, rather than when its writes were durable. (For the hosts, versions, and commands behind both benchmarks, see [File systems on one host](https://docs.archil.com/reference/benchmarks/file-systems).)

## Where the time goes

Divide each install by its 18,602 files. On local disk, 10.7 s is 0.58 ms per file. That’s npm’s own work: reading its cache, unpacking, and making system calls that the kernel answers from memory. On S3 Files, 71.4 s is 3.84 ms per file, so each file spent about 3.3 ms waiting on the network. On Archil, each file spent about 0.09 ms.

A round trip to a server in the same region is short. Our published 4 KiB read that misses the client’s cache is one round trip, and it takes 0.50 ms at the median. So each file on S3 Files waited for the equivalent of six or seven round trips.

The waits come from how a network file system creates a file. A program makes four calls: `open` with `O_CREAT`, `write`, `fchmod`, and `close`. A local file system answers all four in memory. An NFS client has to send three of them to the server, as `CREATE`, `SETATTR`, and `WRITE` followed by `COMMIT`, and wait for each answer.

**One file, three ways**

The same four calls (open, write, chmod, close) on three kinds of storage. A local disk answers all four in memory. An NFS client waits for the server on three of them, for four round trips: CREATE, SETATTR, and WRITE plus COMMIT on close. A client that has checked out the directory answers all four in memory and sends the change later in one message with many other files.

The client can’t skip the question, even when no one else is using the directory. If two machines both create `hello.txt`, only the server sees both requests, so only the server can decide which one gets the name. The client has no way to know that nobody else is looking, so it asks every time.

A client can answer from memory only once it holds the right to decide: the server has handed it the directory, and nobody else can change that directory until the client hands it back. We call that a _checkout_. NFS and our docs call it a _delegation_. By default, an Archil client checks out the whole disk when it mounts it. It answers `open`, `write`, `chmod`, and `close` from memory, and it sends the changes to our servers later, many files to a message, and whenever the program calls `fsync()`. For how this idea grew out of AFS callbacks, leases, and NFSv4 delegations, see [Anatomy of a file create over a network](https://archil.com/guides/how-file-systems-work/anatomy-of-a-file-create).

## A test you can run

Run the job you care about on the same host, once on local disk and once on each storage you’re considering. Stop the clock when the writes are durable, not when the command returns. A design that stages writes on a local disk and uploads them later looks like local disk until you wait for the upload.

```bash
# Set MNT to each mount in turn, local disk first.
time (git clone -q file:///data/repo.git "$MNT/repo" && sync -f "$MNT/repo")
time (cd "$MNT/app" && npm ci --prefer-offline --ignore-scripts && sync -f .)
```

`sync -f` flushes the whole file system that holds the path. On Archil, run `archil sync <mountpoint>` instead. Keep the inputs on local disk, as in the preceding commands, so the network to GitHub or the npm registry isn’t part of the time. Then divide by the number of files. The following calculator starts from our install and takes your numbers too.

**From a timing to a cost per file**

The calculator divides a job’s time by the files it writes. For our `npm ci` of 18,602 files, local disk is 0.58 ms per file. Amazon S3 Files adds 3.26 ms of waiting per file, about 6.5 round trips at 0.5 ms each. JuiceFS with its defaults adds 8.11 ms, about 16 round trips. Archil adds 0.09 ms, about a fifth of one round trip.

## Where Archil is slower

Small files aren’t the only workload, and on several others we lose. In the same EFS benchmark, EFS was faster on the following three tests.

| Test                                         | Archil   | Amazon EFS |
| -------------------------------------------- | -------- | ---------- |
| Sequential write, 4 GiB, then `fsync()`      | 194 MB/s | 590 MB/s   |
| 4 KiB random writes for 30 s, then `fsync()` | 953 IOPS | 2,976 IOPS |
| SQLite, one insert of 1 million rows         | 11.0 s   | 1.7 s      |

We have more work to do on throughput.

A checkout makes one client fast by making the others ask first. In the default exclusive mode, other machines can mount the disk read-only, but they can’t write until the holder unmounts. To share writes, clients check out separate paths, or mount in conditional mode, where creates and renames run on the server as they do on NFS, at the speed of a remote file system. When many clients keep writing the same files, handing ownership back and forth can cost more than NFS’s round trips. Other clients see changes within about 30 seconds by default, or at once after `archil invalidate-cache`. For more information about sharing, see [Can many machines use it at once?](https://archil.com/guides/evaluating-storage/can-many-machines-use-it-at-once)

Distance still counts for reads. A checkout removes the waits on files a client writes, but a file the client has never read has to cross the network. A read that also misses our servers’ cache goes to the object store, typically 10 to 30 ms on S3. From a Modal sandbox about 40 ms from our servers, Archil read a 1 GiB file into a new sandbox at 331 MB/s, and a Modal Volume read it at 624 MB/s. Keep compute in the same region as the disk, warm the cache with [`archil prefetch`](https://docs.archil.com/reference/archil-cli#prefetch) before a job reads a tree it hasn’t seen, or run the command next to the data with [serverless execution](https://docs.archil.com/compute/serverless-execution).