Observability and analytics

One cache for your whole query fleet.

Archil mounts one shared SSD cache over the bucket you already have into every ingest and query pod. Any pod can read the last few minutes without finding the ingest host, and new pods start warm when agent traffic spikes.

Talk to an integration specialist Read the deep dive

Each new pod starts with an empty cache. Its misses go to the bucket, 10 to 30 ms each.

What changes when the cache is shared.

The cache stops being a property of each node and becomes a disk every pod mounts. Three things follow.

Read recent history from any pod

Ingest writes each new segment to the shared disk as it arrives, instead of batching it on its own disk first. A query over the last five minutes reads it there, with no routing to the ingest host that holds it and no merging its answer with the bucket's.

Before: query → find the ingest host → merge with the bucket
After: query → the disk

Scale with bursty agent traffic

Agents query in bursts. When one lands, new pods mount the disk and read from a cache that's already warm, so the fleet grows as fast as you can schedule pods, not as fast as you can refill NVMe. When the burst passes, scale down without throwing a cache away.

Before: new pod → empty NVMe → misses at 10–30 ms
After: new pod → mount → warm

No data to migrate

Point an Archil disk at the bucket that already holds your segments. The objects there become files, new files go back as ordinary objects, and your bucket stays the source of truth. There's nothing to copy into a new store, and every other tool keeps reading the bucket.

Before: copy petabytes into a new store
After: point a disk at your bucket

Why the cache lives on the node.

Your segments and index files sit in object storage, and every read there pays a request charge and 10 to 30 ms to the first byte. So the query fleet copies hot files onto local NVMe, and the cache becomes the fleet's problem.

The cache picks your machines

Every query node needs enough local SSD to hold its share of hot data, so instance types follow the cache, not the query load.

New nodes start cold

Scale out for an incident or a burst of agent queries, and the new nodes have empty caches. Their queries go to the bucket until they warm up.

Containers resist local disks

Kubernetes can expose a node's disk as a volume, but multi-tenant platforms avoid it for fairness and scheduling.

Fresh data is stuck on one host

Ingest batches recent data on its local disk before uploading, so a query for the last few minutes has to find that host.

One disk, every pod.

Point an Archil disk at the bucket that already holds your segments, and mount it into every query and ingest pod with the CSI driver. The engine keeps opening, reading, and mmap-ing files. It never sees the cache.

CSI driver docs

segments.yaml

PersistentVolume and claim

# One Archil disk over your bucket
apiVersion: v1
kind: PersistentVolume
metadata:
  name: segments
spec:
  capacity:
    storage: 1000Gi  # Archil disks are elastic
  accessModes:
    - ReadWriteMany
  persistentVolumeReclaimPolicy: Retain
  csi:
    driver: csi.archil.com
    volumeHandle: "dsk-XXXXXXXXXXXX"
    volumeAttributes:
      region: "aws-us-east-1"
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: segments
spec:
  accessModes:
    - ReadWriteMany
  resources:
    requests:
      storage: 1000Gi
  volumeName: segments

query.yaml

Deployment

# Every query pod mounts the same disk
apiVersion: apps/v1
kind: Deployment
metadata:
  name: query
spec:
  # replicas, selector, and labels omitted
  template:
    spec:
      containers:
        - name: query
          image: "your-query-engine"
          volumeMounts:
            - name: segments
              mountPath: /data
      volumes:
        - name: segments
          persistentVolumeClaim:
            claimName: segments

Stateless query pods

Every pod mounts the same ReadWriteMany volume, so autoscaling adds pods, not empty caches.

Scale down freely

Removing a pod removes no cached data. The cache belongs to the disk, so the next pod picks it up.

Any S3-compatible bucket

Amazon S3, Google Cloud Storage, Cloudflare R2, or another S3-compatible store.

Unlimited read throughput

No cap on reads per disk. The highest one-minute average seen on one disk in production is 5.9 GB/s.

For the hottest paths, say what you'll read.

POSIX resolves a deep path one directory at a time, and the kernel splits a large read into small serial ones. libarchil lets an engine that knows its next reads skip both, calling the Archil client directly.

Available today by invite for teams running large-scale analytics.

Ask for an invite Read the article

“Many analytics applications find it easier to know what they're about to read than to edit the underlying query engine itself.”

Hunter Leath, Beating NVMe by extending POSIX

libarchil

Invite only

// Look up deep paths in parallel
preOpen(filePaths []string) error
 
// Declare reads before you make them
preRead(reads []struct {
  filePath string
  offset   int
  size     int
}) error

open() on a deep path

Illustrative

LookupKernel, one at a timepreOpen, in parallel

logs/service=api/date=2026-11-05/hour=14/part-0042.parquet

Why shared beats per-node.

“As your query fleet grows, and your data set grows, the probability of the query that needs that data being routed to the correct host is very, very low.”

Hunter Leath, Beating NVMe by extending POSIX

Query nodes16

164

With a cache on every node, one node holds this query's data. The query lands on any of the 16.

Cache on every node

6%land on their data (1 in 16)

Average read: 9.4–28 ms

The rest go to the bucket at 10 to 30 ms.

One shared cache

faster on average

100%find it in the cache

Average read: 1 ms+

Every node reads the same SSD cache, at least 1 ms across the network.

A simplified model, not a benchmark. It assumes the data was read before and sits in one node's cache, queries land on nodes at random, and nothing is evicted. Local NVMe reads take about 0.1 ms (tens to hundreds of microseconds), the shared cache at least 1 ms, and a bucket read 10 to 30 ms, typical for Amazon S3. With preRead, the slowest reads match local NVMe, since both are waiting on object storage. Try the full latency calculator.

How the options compare.

Local NVMe still serves the fastest single read. That stops being the common case once the fleet grows.

How a local NVMe cache, network block volumes, a cache you build, and Archil compare
Property	Local NVMe cache	Network block volumes	Build your own	Archil
Fastest read	Best: never leaves the box	A network round trip	A network round trip	At least 1 ms over the network
Shared by every pod	No	No, attached to set machines	Yes	Yes
A new pod starts	Cold	Cold, on a volume of its own	Warm, once it's built	Warm
Hot bytes cached again	On every node that reads them	On every volume	No	No
What you run	SSD machine shapes	Volume counts that track pods	A distributed cache, and the team for it	A CSI mount

Local NVMe cache

Fastest read: Best: never leaves the box
Shared by every pod: No
A new pod starts: Cold
Hot bytes cached again: On every node that reads them
What you run: SSD machine shapes

Network block volumes

Fastest read: A network round trip
Shared by every pod: No, attached to set machines
A new pod starts: Cold, on a volume of its own
Hot bytes cached again: On every volume
What you run: Volume counts that track pods

Build your own

Fastest read: A network round trip
Shared by every pod: Yes
A new pod starts: Warm, once it's built
Hot bytes cached again: No
What you run: A distributed cache, and the team for it

Archil

Fastest read: At least 1 ms over the network
Shared by every pod: Yes
A new pod starts: Warm
Hot bytes cached again: No
What you run: A CSI mount

“Using a shared SSD cache turns out to be cheaper than local NVMe because you don't have to recache the same bytes over and over again.”

Hunter Leath, Beating NVMe by extending POSIX

Data-intensive work beyond analytics

Bring your query fleet.

Talk to an integration specialist Start building