# Archil for CI and build platforms: mount the cache, skip the tarball

Source: https://archil.com/solutions/ci-builds

Put package caches, toolchains, and a repository mirror on one Archil disk that CI jobs mount read-only, with no save and restore step. One refresh job on main writes it, and your remote build cache keeps task outputs.

CI and build platforms

# Mount the cache. Skip the tarball.

Your remote build cache handles task outputs. Put package caches, toolchains, and a repository mirror on one Archil disk: jobs mount it read-only in 50 ms, and one refresh job on main writes it.

[Talk to an integration specialist](https://cal.com/team/archil/demo)[Read the docs](https://docs.archil.com/getting-started/introduction)

Each job downloads and extracts the whole archive, and uploads a new one when the key changes.

Mounts on your runners

-   _Linux_
-   _Kubernetes_
-   _Docker_
-   _Amazon Web Services_
-   _Google Cloud_

VMs, FUSE containers, and Kubernetes pods via the CSI driver; an IAM role on AWS, a token anywhere. [Mounting on Linux](https://docs.archil.com/mounting/linux).

## Everything your build cache doesn't hold.

A job that needs 200 MB of a 20 GB cache archive still downloads and unpacks all 20 GB.

### Archives restored whole

actions/cache downloads and extracts the whole archive before the first step.

### Per-host NVMe copies

A job lands warm on a host with a copy, and cold anywhere else.

### Shared file systems sized for everything

FSx for Lustre starts at 1.2 TiB, and EFS bills every byte you keep.

### Golden images restore cold

A restored volume pulls each block from S3 on its first read.

## Stop tarring up your cache.

The job's first step becomes a mount.

### Warm on a runner that just booted

Jobs that open the same toolchain at once share one S3 fetch; after that, any runner in the region reads it from Archil's SSD cache.

### No save and restore step

Each job mounts the disk in 50 ms at the median.

### Read what the job opens

A job reads only what it opens, and archil prefetch warms a directory it knows it needs.

## Your remote cache holds outputs. The disk holds the rest.

Keep task outputs in your remote cache, and put dependencies, layers, toolchains, and the repository mirror on the disk. Most package managers can read a cache they can't write.

shared-caches.sh

Every job, read-only

```
# Every job: the shared cache, read-only
sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount \
    dsk-XXXXXXXXXXXX /mnt/cache \
    --region aws-us-east-1 --read-only
 
# Gradle: a read-only dependency cache; misses go
# to the build's own. Outputs: your remote cache
export GRADLE_RO_DEP_CACHE=/mnt/cache/gradle-ro
./gradlew build --build-cache
 
# Go: modules from the disk, then the network
GOMODS=file:///mnt/cache/gomod/cache/download
export GOPROXY="$GOMODS,https://proxy.golang.org"
 
# BuildKit: layers from the disk. (The GitHub Actions
# backend is rate limited and skips cache mounts.)
docker buildx build . \
    --cache-from type=local,src=/mnt/cache/buildx
```

### Outputs in one place, inputs in another

Bazel, Turborepo, Nx, and Gradle [key outputs by their inputs](https://bazel.build/remote/caching) and expect dependencies on the runner.

[Gradle read-only cache](https://docs.gradle.org/current/userguide/dependency_caching.html)[GOPROXY protocol](https://go.dev/ref/mod#goproxy-protocol)[BuildKit local cache](https://docs.docker.com/build/cache/backends/local/)

## Main writes the cache. Everything else reads it.

Only main's refresh job writes the cache, so a pull request can't change what your release builds from. Every other job mounts it read-only.

[Sharing disks](https://docs.archil.com/concepts/sharing-disks)

refresh.sh

On main, the only writer

```
# Your orchestrator runs this when the lockfile
# changes on main. It's the only job that writes.
sudo archil mount dsk-XXXXXXXXXXXX /mnt/cache \
    --region aws-us-east-1 --conditional
 
# Fetch into the mirror; never let git collect it
git -C /mnt/cache/repo.git -c gc.auto=0 \
    -c maintenance.auto=false fetch origin
 
# Add main's packages to the shared npm cache
src="$(mktemp -d)/src"
git clone -q --reference /mnt/cache/repo.git "$REPO_URL" "$src"
cd "$src" && npm ci --ignore-scripts --cache /mnt/cache/npm
 
# Make the writes durable, then record this cache
sudo archil sync /mnt/cache
archil checkpoints create /mnt/cache "cache-$(date +%Y%m%d%H%M)"
sudo archil unmount /mnt/cache
```

job.sh

Every other job, read-only

```
# Every other job, on any runner
export ARCHIL_MOUNT_TOKEN="$JOB_TOKEN"   # read-only token
sudo --preserve-env=ARCHIL_MOUNT_TOKEN archil mount \
    dsk-XXXXXXXXXXXX /mnt/cache \
    --region aws-us-east-1 --read-only
 
# Clone onto the runner, borrowing objects from the mirror
git clone --reference /mnt/cache/repo.git "$REPO_URL" src
cd src && git checkout --force "$SHA"
 
# Install from the shared cache; if this PR changed
# the lockfile, install from the registry as today
npm ci --offline --cache /mnt/cache/npm || npm ci
 
npx turbo run build test   # outputs: your remote cache
sudo archil unmount /mnt/cache
```

### A checkpoint per refresh, a branch to roll back

Each refresh ends with an immutable checkpoint in 13 ms; if one breaks builds, mount a branch of the last good one.

Checkpoints and branches (managed disks, AWS) can't be deleted yet; keep one per refresh and per rollback. [Branches and checkpoints](https://docs.archil.com/concepts/branches-and-checkpoints).

## Put your cache volumes on Archil.

Back each customer's cache with its own disk, not a copy per host: stored once, read from any host.

-   A disk per customer is one API call: 25,000 per region on Team, unlimited on Enterprise.
-   [Mount tokens](https://docs.archil.com/concepts/authorization) are scoped to one disk; Archil stores only a hash.
-   [BYOC](https://docs.archil.com/reference/byoc) runs Archil, managed by Archil, in your cloud, your customers', or on bare metal; disk data never leaves.
-   Data touched in the last hour bills at [$0.20 per GiB-month](https://docs.archil.com/administration/billing); the rest of a managed disk, about $0.027.

dispatch.ts

TypeScript SDK

```
// In your orchestrator, before each job. The API
// key stays here; the runner gets only a token.
import { Archil } from "disk"
 
const archil = new Archil({
  apiKey: process.env.ARCHIL_API_KEY,
  region: "aws-us-east-1",
})
const cache = await archil.getDisk(customer.cacheDiskId)
 
// One mount, gone in 2 hours: outlast your longest
// job, or the runner's reads and writes fail.
// Only the refresh job on main writes; forks read.
const { token } = await cache.addUser({
  type: "token",
  nickname: `job-${job.id}`,
  ttl: "2h",
  oneUse: true,
  readOnly: job.kind !== "refresh",
})
 
// The runner mounts with ARCHIL_MOUNT_TOKEN=token.
// Terraform's archil_disk_user takes the same options.
```

## Time before the first step.

Your cache restore time and job count, against a 50 ms mount.

Your pipeline

Cache restore step, per your CI logs

20 s

5 s5 min

Jobs per pipeline that restore it

10

1100

Pipelines a day

100

102,500

### Save and restore

5.6 hof runner time a day before first steps

Each job waits

20 s

Jobs a day

1,000

### One mounted cache

50 sof runner time a day before first steps

Each job mounts, median

50 ms

Jobs a day

1,000

A model, not a benchmark: it counts only the wait before the first step. Saving a [cache key](https://docs.github.com/en/actions/reference/workflows-and-actions/dependency-caching) and refreshing the disk happen only when the lockfile changes, so neither counts.

## Five ways to cache a build.

If your jobs always land on a warm NVMe cache, keep it. Archil holds what a remote build cache doesn't.

Five ways to cache a build, compared
| Property | Save and restore (actions/cache) | Per-host cache volumes | Remote build cache (Bazel, Turborepo, Nx) | Shared file system (EFS, FSx for Lustre) | Archil |
| --- | --- | --- | --- | --- | --- |
| Before the first step | Download, extract everything | Attach a nearby copy | Nothing; outputs fetched on hits | Mount | Mount, 50 ms median |
| Reads, once warm | Local disk | Local NVMe: fastest | Local disk | NFS or Lustre | Memory, then shared SSD |
| Many jobs at once | A copy each | Private copies; last commit wins | Safe: keyed by inputs | One live tree | Jobs read; refresh job writes |
| Retention | 7 days unused; 10 GB per repo | 7 to 14 days unused | Your server's policy | Until deleted | Until deleted; idle bills as archive |
| Rolling back a bad cache | Save a new key | The vendor's tools | Not needed | Your backups | Branch the last good checkpoint |

### Save and restore (actions/cache)

Before the first step

Download, extract everything

Reads, once warm

Local disk

Many jobs at once

A copy each

Retention

7 days unused; 10 GB per repo

Rolling back a bad cache

Save a new key

### Per-host cache volumes

Before the first step

Attach a nearby copy

Reads, once warm

Local NVMe: fastest

Many jobs at once

Private copies; last commit wins

Retention

7 to 14 days unused

Rolling back a bad cache

The vendor's tools

### Remote build cache (Bazel, Turborepo, Nx)

Before the first step

Nothing; outputs fetched on hits

Reads, once warm

Local disk

Many jobs at once

Safe: keyed by inputs

Retention

Your server's policy

Rolling back a bad cache

Not needed

### Shared file system (EFS, FSx for Lustre)

Before the first step

Mount

Reads, once warm

NFS or Lustre

Many jobs at once

One live tree

Retention

Until deleted

Rolling back a bad cache

Your backups

### Archil

Before the first step

Mount, 50 ms median

Reads, once warm

Memory, then shared SSD

Many jobs at once

Jobs read; refresh job writes

Retention

Until deleted; idle bills as archive

Rolling back a bad cache

Branch the last good checkpoint

[How performance is measured](https://docs.archil.com/details/performance)

## Bring your slowest pipeline.

[Talk to an integration specialist](https://cal.com/team/archil/demo)[Start building](https://console.archil.com)