Robotics and physical AI

Train wherever the GPUs are. Leave the data where it is.

Keep the corpus in one Cloudflare R2 bucket, which charges no egress fees. Wherever you find GPUs, create an Archil disk over that bucket close to them and mount it on every node. R2 sends only the ranges your jobs read, and nothing is copied ahead of time.

Talk to an integration specialist Read the docs

Each time the GPUs move, training waits while the whole corpus is copied next to them, and the next move starts the copy over.

Opens the files you already have

MCAP
ROS 2 bags
ROS 1 bags
Parquet
HDF5
Camera video
Calibration

Put the cache in an Archil region on AWS or Google Cloud, or, on Enterprise, run Archil with BYOC in your own Kubernetes cluster on any other cloud or bare metal. Regions · BYOC

Three ways to follow the GPUs.

Each works. Each slows the move down in a different place.

Copy to each cloud

Training waits for the whole copy, and every move starts it over.

Stream from the far bucket

Training starts at once, but every job pays the distance again, and each node's cache helps only that node.

A file system in one region

FSx for Lustre imports automatically only from an S3 bucket in its own region.

Where you train next.

Enter your corpus and your jobs, and see what has to cross the distance before and during training.

Corpus in the bucket100 TB

5 TB1,000 TB

Share each job reads10%

1%100%

Jobs reading it, together or within the hour4

132

Link from the bucket100 Gbit/s

10 Gbit/s400 Gbit/s

Copy to the new location

100 TBcrosses the distance

Wait for step 1: 2.2 h
Moved before step 1: 100 TB
During 4 jobs: 0

Stream from the bucket

40 TBcrosses the distance

Wait for step 1: none
Moved before step 1: nothing
During 4 jobs: 40 TB

A cache by the GPUs

10 TBcrosses the distance

Wait for step 1: none
Moved before step 1: nothing
During 4 jobs: 10 TB

A simplified model, not a benchmark: each job reads its share once, and jobs share Archil's cache only when they run together or within the hour it keeps data. Archil may fetch up to 10 MiB more than each read asks for.

Move the GPUs, not the data.

The corpus stays in R2 as ordinary objects. Only the cache moves.

R2 sends each range once

Nodes that ask mid-fetch wait on that fetch; later ones that hour read the cache next to them.

Read only what you open

An indexed file like MCAP reads a short window; R2 sends that range, plus at most 10 MiB.

Write results as files

Files written on any mount land in R2 as ordinary objects at matching keys.

A disk wherever the GPUs are

Use an Archil region on AWS or Google Cloud, or BYOC (Enterprise) in your own cluster anywhere else.

One bucket, a disk per location.

Create a disk over the R2 bucket wherever the GPUs land, and mount it on every node.

ML training with Archil

disks.tf

Terraform provider

# One R2 bucket. A disk wherever the GPUs are this month.
variable "r2_key_id" {}
variable "r2_secret" { sensitive = true }
 
provider "archil" {
  alias  = "aws"
  region = "aws-us-east-1"
}
 
provider "archil" {
  alias  = "gcp"
  region = "gcp-us-central1"
}
 
resource "archil_disk" "corpus_aws" {
  provider = archil.aws
  name     = "corpus"
 
  mount {
    type              = "r2"
    bucket_name       = "corpus"
    bucket_endpoint   = "https://<account>.r2.cloudflarestorage.com"
    access_key_id     = var.r2_key_id
    secret_access_key = var.r2_secret
  }
}
 
# Next month's GPUs: the same mount block, another region.
resource "archil_disk" "corpus_gcp" {
  provider = archil.gcp
  name     = "corpus"
 
  mount {
    type              = "r2"
    bucket_name       = "corpus"
    bucket_endpoint   = "https://<account>.r2.cloudflarestorage.com"
    access_key_id     = var.r2_key_id
    secret_access_key = var.r2_secret
  }
}

train-job.yaml

Indexed Job

# 64 workers in this location, all reading one disk
apiVersion: batch/v1
kind: Job
metadata:
  name: train
spec:
  completionMode: Indexed
  completions: 64
  parallelism: 64
  template:
    spec:
      restartPolicy: Never
      containers:
        - name: train
          image: "your-training-image"
          # Reads $JOB_COMPLETION_INDEX to pick its shard
          args: ["--data", "/data"]
          volumeMounts:
            - name: corpus
              mountPath: /data
              readOnly: true
      volumes:
        - name: corpus
          persistentVolumeClaim:
            claimName: corpus  # an RWX claim on this location's disk

Up to 5,000 nodes per disk with no per-disk read cap, and read-ahead hides the distance to R2 on sequential reads.
Archil bills reads out to your nodes on Developer and Team; Enterprise can avoid that with private networking.

scenario.py

mcap reader

from mcap.reader import make_reader
 
NS = 1_000_000_000
 
 
def window(path, offset_s, seconds=15,
           topics=("/camera/front", "/lidar/top")):
    """15 seconds of two topics from an hours-long log.
 
    The reader seeks with the file's chunk index, so the
    disk fetches those byte ranges from R2, not the file.
    """
    with open(path, "rb") as f:
        reader = make_reader(f)
        t0 = reader.get_summary().statistics.message_start_time
        start = t0 + offset_s * NS
        yield from reader.iter_messages(
            topics=list(topics),
            starting_at=start,
            ending_before=start + seconds * NS,
        )
 
 
log = "/data/robot-17/2026-10-14/drive-0003.mcap"
for schema, channel, message in window(log, offset_s=6130):
    ...

mount.sh

archil CLI

# A node outside Kubernetes, in the second location.
# Raise read-ahead past the 4 MiB default for long
# sequential files, such as video.
archil mount corpus /data --region gcp-us-central1 \
  --read-only --read-ahead 33554432
 
# New uploads straight to R2 show up within a few
# minutes in directories in use. Nothing to sync.

Caching docs

Pin what changes.

Raw logs in R2 don't change, so a list of their keys pins them. Labels and configs do, so keep those on an Archil-managed disk (AWS regions) and checkpoint it when a run starts.

How checkpoints pin a training set

How the options compare.

Copying and streaming both work. With Archil, training starts at once, and R2 sends each range once to jobs reading together.

Ways to get a corpus to GPUs in a new location, compared
Property	Copy to each cloud	Stream from the bucket	FSx for Lustre on S3	Archil over R2
Before the first step	Wait for the whole copy	Nothing	Create, link, load metadata	Create a disk, mount it
When the GPUs move	Copy everything again	Same bucket, farther away	Same-region auto import only	New disk, same bucket
Jobs reading the same data	Read the local copy	Each job reads again	Served by FSx once loaded	Once from R2, within the hour
Tools that open files	Full POSIX	Remote-aware readers	Full POSIX	POSIX, atomic directory renames
Where the GPUs can be	Wherever you copy to	Anywhere, far from the data	The file system's AWS region	Archil regions, or BYOC anywhere

Copy to each cloud

Before the first step: Wait for the whole copy
When the GPUs move: Copy everything again
Jobs reading the same data: Read the local copy
Tools that open files: Full POSIX
Where the GPUs can be: Wherever you copy to

Stream from the bucket

Before the first step: Nothing
When the GPUs move: Same bucket, farther away
Jobs reading the same data: Each job reads again
Tools that open files: Remote-aware readers
Where the GPUs can be: Anywhere, far from the data

FSx for Lustre on S3

Before the first step: Create, link, load metadata
When the GPUs move: Same-region auto import only
Jobs reading the same data: Served by FSx once loaded
Tools that open files: Full POSIX
Where the GPUs can be: The file system's AWS region

Archil over R2

Before the first step: Create a disk, mount it
When the GPUs move: New disk, same bucket
Jobs reading the same data: Once from R2, within the hour
Tools that open files: POSIX, atomic directory renames
Where the GPUs can be: Archil regions, or BYOC anywhere

How performance is measured ML training with Archil

Keep the data in R2. Train wherever the GPUs are.

Talk to an integration specialist Start building