# Archil for life sciences: one disk over your bucket, mounted on every node

Source: https://archil.com/solutions/life-sciences

Mount your existing research bucket as one disk on every node, with one shared cache in front of it. Pipeline tasks open files in place instead of copying them, and many tasks reading the same file fetch it from your bucket once.

Life sciences

# One disk over your bucket, mounted on every node.

Archil turns your bucket into one disk that every node mounts, with a shared cache in front. Tasks open files in place, and the bucket serves each file once.

[Talk to an integration specialist](https://cal.com/team/archil/demo)[Read the docs](https://docs.archil.com/getting-started/introduction)

Each task copies the reference files into its own scratch, and the next task copies them again.

Mounts where your pipelines run

-   _Amazon Web Services_
-   _Google Cloud_
-   _Kubernetes_
-   _Linux_

Slurm nodes and EKS or GKE pods mount one disk, on AWS, Google Cloud, or BYOC in your account. [How BYOC works](https://docs.archil.com/reference/byoc).

## Why pipelines still copy.

Most options still copy shared files to every node, or need a disk provisioned to hold them.

### Copies in, copies out

Nextflow on AWS Batch, Cromwell, Snakemake storage plugins, and HealthOmics copy inputs in before a task runs.

### A cache per node, or per process

Mountpoint, Cloud Storage FUSE, and the FUSE layer sold for Nextflow cache per node, so each new node pulls the files again.

### Capacity to size up front

FSx for Lustre needs capacity set before the run, in 1.2 or 2.4 TiB steps, in one Availability Zone.

### Files that aren't quite files

Tools that need the crossed-out operations still need a scratch disk beside Mountpoint.

## What a shared disk changes.

One disk over your bucket, at the same path on every node.

### Many readers, one fetch

Tasks that open a file mid-fetch wait on that fetch; later ones, on any node, read the shared cache until it has sat an hour unread.

### No stage-in, no stage-out

Outputs are durable when fsync() returns, and reach your bucket at the same key tens of seconds later.

### Run where the capacity is

Nodes in any zone of the region mount the disk, so jobs run wherever instances are free.

### The bucket stays the source of truth

Direct writes to the bucket appear within minutes; uploads through Archil's S3 API appear on return.

## Tools that expect a shared file system.

Nextflow, Snakemake, and cryoSPARC expect a file system every node sees at one path. An Archil disk is one, over your bucket.

Nextflow · Snakemake

### Genomics pipelines

The work directory gets its own Archil-managed disk, so intermediates stay out of your bucket.

cryoSPARC · RELION

### Imaging

Every worker sees one absolute path; repeat reads within the hour come from the shared cache.

Python · R · bash

### Research agents

Each tool call runs in a fresh microVM beside the data: the cohort read-only, the session read-write.

-   2 vCPU, 4 GiB, and up to 5 minutes per call; sandboxes run longer, with up to 32 vCPU and 64 GiB.

[Serverless execution docs](https://docs.archil.com/compute/serverless-execution)

agent\_tool.py

Serverless execution

```
import archil
from archil import ExecMountSpec
 
# Exec allows 5 minutes per command; the client's default
# request timeout is 30 seconds, so raise it.
client = archil.Archil(timeout=330)
 
# The cohort mounts read-only; each session has its own disk.
COHORT = ExecMountSpec("dsk-XXXXXXXXXXXX", read_only=True)
 
 
def bash(session_disk: str, command: str):
    """The agent's bash tool: one fresh microVM per call."""
    res = client.exec(
        disks={"cohort": COHORT, "session": session_disk},
        command=command,
    )
    return res.stdout, res.stderr, res.exit_code
 
 
# The report stays on the session disk for the next call,
# and your app can read it over the S3 API.
bash("dsk-YYYYYYYYYYYY",
     "python3 session/survival.py "
     "--expr cohort/expression/tpm.tsv "
     "--clinical cohort/clinical.tsv "
     "--out session/reports/survival.md")
```

## Two disks and a config change.

One disk over your bucket, a second for the work directory, both mounted on every node.

[Connect your bucket](https://docs.archil.com/data-sources/s3-object-storage)

disk.tf

Terraform, over your bucket

```
# The disk over the bucket your data already
# lives in. Nothing is copied or converted.
resource "archil_disk" "genomics" {
  name = "genomics"
 
  mount {
    type        = "s3"
    bucket_name = "your-genomics-bucket"
  }
}
 
# Nextflow's work directory gets its own
# Archil-managed disk, so intermediates never
# land in your bucket.
resource "archil_disk" "work" {
  name = "nextflow-work"
}
 
# With no access keys, your bucket policy grants
# Archil's role access, scoped to this one disk
# by its session ID.
output "archil_session_id" {
  value = archil_disk.genomics.mount[0].session_id
}
```

mount.sh

Every compute node

```
# At boot on every Slurm compute node
curl -s https://archil.com/install | sh
 
# On AWS, the node's IAM role is a disk user: no token.
# --conditional lets every node write at once.
sudo mkdir -p /mnt/archil /mnt/work
sudo archil mount genomics /mnt/archil \
    --region aws-us-east-1 --conditional
sudo archil mount nextflow-work /mnt/work \
    --region aws-us-east-1 --conditional
 
# close() sends a file's writes to the disk, and creates
# and renames land before they return, so another node
# that opens the file next reads them. Listings cache
# for five minutes by default; refresh the work
# directory's in seconds.
archil set-cache-expiry /mnt/work --readdir-expiry 5
```

nextflow.config

[Nextflow on Kubernetes](https://docs.seqera.io/nextflow/kubernetes)

```
// nf-core/sarek on Kubernetes, EKS included. Both
// claims are ReadWriteMany volumes on the Archil CSI
// driver. (Nextflow's AWS Batch executor stages
// inputs through S3, so use k8s or Slurm.)
process.executor = 'k8s'
 
k8s {
    storageClaimName = 'nextflow-work'
    storageMountPath = '/mnt/work'
}
workDir = '/mnt/work'
 
process.pod = [volumeClaim: 'genomics', mountPath: '/mnt/archil']
 
// Point at prebuilt index files, so the pipeline
// reads them in place instead of building them.
params {
    igenomes_ignore = true
    fasta   = '/mnt/archil/references/GRCh38/GRCh38.fa'
    bwamem2 = '/mnt/archil/references/GRCh38/bwamem2/'
    input   = '/mnt/archil/runs/2026-11/samplesheet.csv'
    outdir  = '/mnt/archil/results/2026-11'
}
```

run.sh

[Snakemake CLI](https://snakemake.readthedocs.io/en/stable/executing/cli.html)

```
# Snakemake on Slurm, with the project on the disk.
# The nodes all see the same paths: no storage plugin.
cd /mnt/archil/projects/wgs-2026-11
snakemake --executor slurm --jobs 200
```

-   No read cap per disk; up to 8 GiB/s per client.
-   No local disk needed: the cache is RAM, 2 GiB by default.

## Where the data lives.

Data stays in your bucket and region, and Archil can run inside your account.

[Security and compliance](https://docs.archil.com/details/security)

### Your bucket, your region, your account

Files stay ordinary objects in your bucket and region; with BYOC, no PHI leaves your account.

### Access per disk

An IAM role or disk token opens one disk, and POSIX permissions apply on each mount.

### HIPAA, SOC 2, and encryption

HIPAA workloads, with a BAA for Enterprise customers, and ongoing SOC 2 Type II audits.

## How the options compare.

Staging is hard to beat on one big node. On a fleet, every task copies.

How staging into task scratch, FUSE over the bucket, FSx for Lustre linked to S3, Amazon EFS or S3 Files, and Archil compare for pipelines
| Property | Stage in to task scratch | FUSE over the bucket | FSx for Lustre linked to S3 | Amazon EFS or S3 Files | Archil |
| --- | --- | --- | --- | --- | --- |
| Before a task computes | Copy inputs in | Nothing: reads on demand | Nothing: loads on first access | EFS: load first; S3 Files: nothing | Nothing: reads in place |
| Shared inputs, across nodes | Copied by every task | Cached per node | Loaded once, shared | Shared | One fetch, even concurrently |
| File semantics | Full POSIX, locally | Varies; Mountpoint can't modify files | Full POSIX | NFS; S3 Files lacks hard links | POSIX, renames, random writes |
| Shared work directory | No: S3 on AWS Batch | Only with a Nextflow FUSE layer | Yes | Yes | Yes, ReadWriteMany |
| Capacity | Sized per task | Local cache disk per node | Set up front, in TiB steps | Elastic | Elastic; no local disk |
| Replication | Bucket's; scratch is ephemeral | Bucket's; caches are ephemeral | One Availability Zone | Across zones | Across zones, then the bucket |

### Stage in to task scratch

Before a task computes

Copy inputs in

Shared inputs, across nodes

Copied by every task

File semantics

Full POSIX, locally

Shared work directory

No: S3 on AWS Batch

Capacity

Sized per task

Replication

Bucket's; scratch is ephemeral

### FUSE over the bucket

Before a task computes

Nothing: reads on demand

Shared inputs, across nodes

Cached per node

File semantics

Varies; Mountpoint can't modify files

Shared work directory

Only with a Nextflow FUSE layer

Capacity

Local cache disk per node

Replication

Bucket's; caches are ephemeral

### FSx for Lustre linked to S3

Before a task computes

Nothing: loads on first access

Shared inputs, across nodes

Loaded once, shared

File semantics

Full POSIX

Shared work directory

Yes

Capacity

Set up front, in TiB steps

Replication

One Availability Zone

### Amazon EFS or S3 Files

Before a task computes

EFS: load first; S3 Files: nothing

Shared inputs, across nodes

Shared

File semantics

NFS; S3 Files lacks hard links

Shared work directory

Yes

Capacity

Elastic

Replication

Across zones

### Archil

Before a task computes

Nothing: reads in place

Shared inputs, across nodes

One fetch, even concurrently

File semantics

POSIX, renames, random writes

Shared work directory

Yes, ReadWriteMany

Capacity

Elastic; no local disk

Replication

Across zones, then the bucket

[How performance is measured](https://docs.archil.com/details/performance)[Data-intensive work beyond pipelines](https://archil.com/use-cases/data-intensive)

## Point a disk at your bucket.

[Talk to an integration specialist](https://cal.com/team/archil/demo)[Start building](https://console.archil.com)