Life sciences
One disk over your bucket, mounted on every node.
Archil turns your bucket into one disk that every node mounts, with a shared cache in front. Tasks open files in place, and the bucket serves each file once.
Each task copies the reference files into its own scratch, and the next task copies them again.
Mounts where your pipelines run
Slurm nodes and EKS or GKE pods mount one disk, on AWS, Google Cloud, or BYOC in your account. How BYOC works.
Why pipelines still copy.
Most options still copy shared files to every node, or need a disk provisioned to hold them.
Copies in, copies out
Nextflow on AWS Batch, Cromwell, Snakemake storage plugins, and HealthOmics copy inputs in before a task runs.
A cache per node, or per process
Mountpoint, Cloud Storage FUSE, and the FUSE layer sold for Nextflow cache per node, so each new node pulls the files again.
Capacity to size up front
FSx for Lustre needs capacity set before the run, in 1.2 or 2.4 TiB steps, in one Availability Zone.
Files that aren't quite files
Tools that need the crossed-out operations still need a scratch disk beside Mountpoint.
What a shared disk changes.
One disk over your bucket, at the same path on every node.
Many readers, one fetch
Tasks that open a file mid-fetch wait on that fetch; later ones, on any node, read the shared cache until it has sat an hour unread.
No stage-in, no stage-out
Outputs are durable when fsync() returns, and reach your bucket at the same key tens of seconds later.
Run where the capacity is
Nodes in any zone of the region mount the disk, so jobs run wherever instances are free.
The bucket stays the source of truth
Direct writes to the bucket appear within minutes; uploads through Archil's S3 API appear on return.
Tools that expect a shared file system.
Nextflow, Snakemake, and cryoSPARC expect a file system every node sees at one path. An Archil disk is one, over your bucket.
Nextflow · Snakemake
Genomics pipelines
The work directory gets its own Archil-managed disk, so intermediates stay out of your bucket.
cryoSPARC · RELION
Imaging
Every worker sees one absolute path; repeat reads within the hour come from the shared cache.
Python · R · bash
Research agents
Each tool call runs in a fresh microVM beside the data: the cohort read-only, the session read-write.
- 2 vCPU, 4 GiB, and up to 5 minutes per call; sandboxes run longer, with up to 32 vCPU and 64 GiB.
import archilfrom archil import ExecMountSpec # Exec allows 5 minutes per command; the client's default# request timeout is 30 seconds, so raise it.client = archil.Archil(timeout=330) # The cohort mounts read-only; each session has its own disk.COHORT = ExecMountSpec("dsk-XXXXXXXXXXXX", read_only=True) def bash(session_disk: str, command: str): """The agent's bash tool: one fresh microVM per call.""" res = client.exec( disks={"cohort": COHORT, "session": session_disk}, command=command, ) return res.stdout, res.stderr, res.exit_code # The report stays on the session disk for the next call,# and your app can read it over the S3 API.bash("dsk-YYYYYYYYYYYY", "python3 session/survival.py " "--expr cohort/expression/tpm.tsv " "--clinical cohort/clinical.tsv " "--out session/reports/survival.md")Two disks and a config change.
One disk over your bucket, a second for the work directory, both mounted on every node.
Connect your bucket# The disk over the bucket your data already# lives in. Nothing is copied or converted.resource "archil_disk" "genomics" { name = "genomics" mount { type = "s3" bucket_name = "your-genomics-bucket" }} # Nextflow's work directory gets its own# Archil-managed disk, so intermediates never# land in your bucket.resource "archil_disk" "work" { name = "nextflow-work"} # With no access keys, your bucket policy grants# Archil's role access, scoped to this one disk# by its session ID.output "archil_session_id" { value = archil_disk.genomics.mount[0].session_id}# At boot on every Slurm compute nodecurl -s https://archil.com/install | sh # On AWS, the node's IAM role is a disk user: no token.# --conditional lets every node write at once.sudo mkdir -p /mnt/archil /mnt/worksudo archil mount genomics /mnt/archil \ --region aws-us-east-1 --conditionalsudo archil mount nextflow-work /mnt/work \ --region aws-us-east-1 --conditional # close() sends a file's writes to the disk, and creates# and renames land before they return, so another node# that opens the file next reads them. Listings cache# for five minutes by default; refresh the work# directory's in seconds.archil set-cache-expiry /mnt/work --readdir-expiry 5// nf-core/sarek on Kubernetes, EKS included. Both// claims are ReadWriteMany volumes on the Archil CSI// driver. (Nextflow's AWS Batch executor stages// inputs through S3, so use k8s or Slurm.)process.executor = 'k8s' k8s { storageClaimName = 'nextflow-work' storageMountPath = '/mnt/work'}workDir = '/mnt/work' process.pod = [volumeClaim: 'genomics', mountPath: '/mnt/archil'] // Point at prebuilt index files, so the pipeline// reads them in place instead of building them.params { igenomes_ignore = true fasta = '/mnt/archil/references/GRCh38/GRCh38.fa' bwamem2 = '/mnt/archil/references/GRCh38/bwamem2/' input = '/mnt/archil/runs/2026-11/samplesheet.csv' outdir = '/mnt/archil/results/2026-11'}# Snakemake on Slurm, with the project on the disk.# The nodes all see the same paths: no storage plugin.cd /mnt/archil/projects/wgs-2026-11snakemake --executor slurm --jobs 200- No read cap per disk; up to 8 GiB/s per client.
- No local disk needed: the cache is RAM, 2 GiB by default.
Where the data lives.
Data stays in your bucket and region, and Archil can run inside your account.
Security and complianceYour bucket, your region, your account
Files stay ordinary objects in your bucket and region; with BYOC, no PHI leaves your account.
Access per disk
An IAM role or disk token opens one disk, and POSIX permissions apply on each mount.
HIPAA, SOC 2, and encryption
HIPAA workloads, with a BAA for Enterprise customers, and ongoing SOC 2 Type II audits.
How the options compare.
Staging is hard to beat on one big node. On a fleet, every task copies.
| Property | Stage in to task scratch | FUSE over the bucket | FSx for Lustre linked to S3 | Amazon EFS or S3 Files | Archil |
|---|---|---|---|---|---|
| Before a task computes | Copy inputs in | Nothing: reads on demand | Nothing: loads on first access | EFS: load first; S3 Files: nothing | Nothing: reads in place |
| Shared inputs, across nodes | Copied by every task | Cached per node | Loaded once, shared | Shared | One fetch, even concurrently |
| File semantics | Full POSIX, locally | Varies; Mountpoint can't modify files | Full POSIX | NFS; S3 Files lacks hard links | POSIX, renames, random writes |
| Shared work directory | No: S3 on AWS Batch | Only with a Nextflow FUSE layer | Yes | Yes | Yes, ReadWriteMany |
| Capacity | Sized per task | Local cache disk per node | Set up front, in TiB steps | Elastic | Elastic; no local disk |
| Replication | Bucket's; scratch is ephemeral | Bucket's; caches are ephemeral | One Availability Zone | Across zones | Across zones, then the bucket |
Stage in to task scratch
- Before a task computes
- Copy inputs in
- Shared inputs, across nodes
- Copied by every task
- File semantics
- Full POSIX, locally
- Shared work directory
- No: S3 on AWS Batch
- Capacity
- Sized per task
- Replication
- Bucket's; scratch is ephemeral
FUSE over the bucket
- Before a task computes
- Nothing: reads on demand
- Shared inputs, across nodes
- Cached per node
- File semantics
- Varies; Mountpoint can't modify files
- Shared work directory
- Only with a Nextflow FUSE layer
- Capacity
- Local cache disk per node
- Replication
- Bucket's; caches are ephemeral
FSx for Lustre linked to S3
- Before a task computes
- Nothing: loads on first access
- Shared inputs, across nodes
- Loaded once, shared
- File semantics
- Full POSIX
- Shared work directory
- Yes
- Capacity
- Set up front, in TiB steps
- Replication
- One Availability Zone
Amazon EFS or S3 Files
- Before a task computes
- EFS: load first; S3 Files: nothing
- Shared inputs, across nodes
- Shared
- File semantics
- NFS; S3 Files lacks hard links
- Shared work directory
- Yes
- Capacity
- Elastic
- Replication
- Across zones
Archil
- Before a task computes
- Nothing: reads in place
- Shared inputs, across nodes
- One fetch, even concurrently
- File semantics
- POSIX, renames, random writes
- Shared work directory
- Yes, ReadWriteMany
- Capacity
- Elastic; no local disk
- Replication
- Across zones, then the bucket