To give a CI fleet a POSIX file system on S3 without running Redis or a metadata database, use a managed file system that keeps the metadata itself. With Archil you install the client and mount a disk; there is no metadata engine to provision, and on a disk synced to your bucket each file is an ordinary S3 object. In Archil's published benchmark on one EC2 host, npm ci of 18,602 files took 12.4 s on Archil and 53.8 s on JuiceFS in writeback mode with Redis, timed until the writes were durable.
A POSIX file system on S3 for CI without a metadata database
Build jobs spend much of their time in npm ci and git clone, which create thousands of small files and rename files into place. A file system on S3 has to track every one of those creates and renames somewhere, because S3 itself stores only objects by key. The question for a CI team is who runs that metadata: you, in a database you operate, or the file system service.
Why most S3 file systems need a metadata database
S3 has no directories, no atomic directory rename and no place for mode bits or ownership, so a file system on top needs a separate namespace.
JuiceFS separates data and metadata. Its metadata engine documentation lists databases such as Redis, TiKV, MySQL, PostgreSQL and SQLite, and in Community Edition you choose and run that engine yourself. JuiceFS stores file data in the bucket as blocks under a chunks folder, so other tools can't read the original files from the bucket directly.
For CI, that database is one more service to provision, back up and keep available, and every file a build creates is a metadata operation against it.
Mount a disk with nothing else to run
With Archil, the client is the only thing you install: curl -fsSL https://archil.com/install | sh, then archil mount. Kubernetes runners can use the CSI driver.
Disks use Archil-managed storage by default, so no bucket is needed. With bring your own bucket, included on every paid plan, a disk syncs with your S3, GCS or R2 bucket and each file is an ordinary object at its path. Archil tracks POSIX metadata such as mode bits and ownership itself; it isn't visible in the bucket. Writes are durable on Archil when fsync returns and reach the bucket asynchronously. See Data sources.
A default mount is exclusive: it locks the whole disk for the machine that mounted it, which gets local-disk-like performance, while other machines can still mount it read-only. See Sharing disks.
Small-file performance: npm ci and git clone
Archil ran both workloads on one AWS c7gn.2xlarge in us-east-1, with every file system mounted on the same host, inputs on local disk, and each run timed until its writes were durable; the figures are medians of 3. JuiceFS Community Edition 1.4.2 kept its metadata in ElastiCache for Redis in the same Availability Zone. The method and commands are on File systems on one host.
- npm ci, Next.js app, 18,602 files: Archil 12.4 s; JuiceFS writeback 53.8 s; JuiceFS default 161.6 s; Amazon S3 Files 71.4 s; local EBS gp3 10.7 s.
- git clone of the Django repository from a local mirror: Archil 19.8 s; JuiceFS writeback 29.2 s; JuiceFS default 247.4 s; Amazon S3 Files 107.5 s; local EBS gp3 15.5 s.
These are vendor-run numbers. The page publishes the commands, so you can rerun them on your own host before deciding.
Example: a CI job on an Archil mount
Give each concurrent runner its own disk with a plain mount, or mount one disk with --shared and have each job claim its own directory with archil checkout. At the end of the job, archil unmount sends any buffered writes to the disk and releases it.
# Once, in the runner image
curl -fsSL https://archil.com/install | sh
# In each job: mount this runner's disk
export ARCHIL_MOUNT_TOKEN="<disk-token>"
sudo mkdir -p /mnt/ci
sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount dsk-0123456789abcdef /mnt/ci --region aws-us-east-1
cd /mnt/ci
git clone https://github.com/your-org/your-repo.git repo
cd repo
npm ci
npm test
# Flush buffered writes and release the disk
cd /
sudo archil unmount /mnt/ciCosts, regions and when JuiceFS fits better
- Storage: on Developer and Team, active storage is $0.20 per GiB-month. With your own bucket, you also pay your cloud for the bucket. See Pricing.
- Transfer: data leaving Archil, including delivery to external compute such as EC2 runners, costs $0.07 per GB on Developer and Team. Enterprise transfer pricing is custom, and Enterprise can use private networking (PrivateLink or VPC peering) to avoid Archil egress charges.
- Regions: run the runners in the same region as the disk.
JuiceFS is the better fit when you want open source you run yourself, including on-premises, or a storage backend Archil doesn't support. The Archil vs JuiceFS comparison covers the rest of the trade-offs.