# Archil for observability and analytics: one cache for your query fleet Source: https://archil.com/solutions/observability A shared SSD cache over the bucket you already have: read recent data from any pod, scale with bursty agent traffic, and migrate nothing. Observability and analytics # One cache for your whole query fleet. Archil mounts one shared SSD cache over the bucket you already have into every ingest and query pod. Any pod can read the last few minutes without finding the ingest host, and new pods start warm when agent traffic spikes. [Talk to an integration specialist](https://cal.com/team/archil/demo)[Read the deep dive](https://archil.com/post/beating-nvme-by-extending-posix) Each new pod starts with an empty cache. Its misses go to the bucket, 10 to 30 ms each. ## What changes when the cache is shared. The cache stops being a property of each node and becomes a disk every pod mounts. Three things follow. 01 ### Read recent history from any pod Ingest writes each new segment to the shared disk as it arrives, instead of batching it on its own disk first. A query over the last five minutes reads it there, with no routing to the ingest host that holds it and no merging its answer with the bucket's. Before query → find the ingest host → merge with the bucket After query → the disk 02 ### Scale with bursty agent traffic Agents query in bursts. When one lands, new pods mount the disk and read from a cache that's already warm, so the fleet grows as fast as you can schedule pods, not as fast as you can refill NVMe. When the burst passes, scale down without throwing a cache away. Before new pod → empty NVMe → misses at 10–30 ms After new pod → mount → warm 03 ### No data to migrate Point an Archil disk at the bucket that already holds your segments. The objects there become files, new files go back as ordinary objects, and your bucket stays the source of truth. There's nothing to copy into a new store, and every other tool keeps reading the bucket. Before copy petabytes into a new store After point a disk at your bucket ## Why the cache lives on the node. Your segments and index files sit in object storage, and every read there pays a request charge and 10 to 30 ms to the first byte. So the query fleet copies hot files onto local NVMe, and the cache becomes the fleet's problem. ### The cache picks your machines Every query node needs enough local SSD to hold its share of hot data, so instance types follow the cache, not the query load. ### New nodes start cold Scale out for an incident or a burst of agent queries, and the new nodes have empty caches. Their queries go to the bucket until they warm up. ### Containers resist local disks Kubernetes can expose a node's disk as a volume, but multi-tenant platforms avoid it for fairness and scheduling. ### Fresh data is stuck on one host Ingest batches recent data on its local disk before uploading, so a query for the last few minutes has to find that host. ## One disk, every pod. Point an Archil disk at the bucket that already holds your segments, and mount it into every query and ingest pod with the CSI driver. The engine keeps opening, reading, and mmap-ing files. It never sees the cache. [CSI driver docs](https://docs.archil.com/reference/csi-driver) segments.yaml PersistentVolume and claim ``` # One Archil disk over your bucket apiVersion: v1 kind: PersistentVolume metadata: name: segments spec: capacity: storage: 1000Gi # Archil disks are elastic accessModes: - ReadWriteMany persistentVolumeReclaimPolicy: Retain csi: driver: csi.archil.com volumeHandle: "dsk-XXXXXXXXXXXX" volumeAttributes: region: "aws-us-east-1" --- apiVersion: v1 kind: PersistentVolumeClaim metadata: name: segments spec: accessModes: - ReadWriteMany resources: requests: storage: 1000Gi volumeName: segments ``` query.yaml Deployment ``` # Every query pod mounts the same disk apiVersion: apps/v1 kind: Deployment metadata: name: query spec: # replicas, selector, and labels omitted template: spec: containers: - name: query image: "your-query-engine" volumeMounts: - name: segments mountPath: /data volumes: - name: segments persistentVolumeClaim: claimName: segments ``` ### Stateless query pods Every pod mounts the same ReadWriteMany volume, so autoscaling adds pods, not empty caches. ### Scale down freely Removing a pod removes no cached data. The cache belongs to the disk, so the next pod picks it up. ### Any S3-compatible bucket Amazon S3, Google Cloud Storage, Cloudflare R2, or another S3-compatible store. ### Unlimited read throughput No cap on reads per disk. The highest one-minute average seen on one disk in production is 5.9 GB/s. ## For the hottest paths, say what you'll read. POSIX resolves a deep path one directory at a time, and the kernel splits a large read into small serial ones. libarchil lets an engine that knows its next reads skip both, calling the Archil client directly. Available today by invite for teams running large-scale analytics. [Ask for an invite](https://cal.com/team/archil/demo)[Read the article](https://archil.com/post/beating-nvme-by-extending-posix) > “Many analytics applications find it easier to know what they're about to read than to edit the underlying query engine itself.” Hunter Leath, [Beating NVMe by extending POSIX](https://archil.com/post/beating-nvme-by-extending-posix) libarchil Invite only ``` // Look up deep paths in parallel preOpen(filePaths []string) error   // Declare reads before you make them preRead(reads []struct { filePath string offset int size int }) error ``` open() on a deep path Illustrative LookupKernel, one at a timepreOpen, in parallel logs/service=api/date=2026-11-05/hour=14/part-0042.parquet ## Why shared beats per-node. > “As your query fleet grows, and your data set grows, the probability of the query that needs that data being routed to the correct host is very, very low.” Hunter Leath, [Beating NVMe by extending POSIX](https://archil.com/post/beating-nvme-by-extending-posix) Query nodes 16 164 With a cache on every node, one node holds this query's data. The query lands on any of the 16. ### Cache on every node 6%land on their data (1 in 16) Average read 9.4–28 ms The rest go to the bucket at 10 to 30 ms. ### One shared cache faster on average 100%find it in the cache Average read 1 ms+ Every node reads the same SSD cache, at least 1 ms across the network. A simplified model, not a benchmark. It assumes the data was read before and sits in one node's cache, queries land on nodes at random, and nothing is evicted. Local NVMe reads take about 0.1 ms (tens to hundreds of microseconds), the shared cache at least 1 ms, and a bucket read 10 to 30 ms, typical for Amazon S3. With preRead, the slowest reads match local NVMe, since both are waiting on object storage. [Try the full latency calculator](https://archil-latency-calculator.vercel.app/). ## How the options compare. Local NVMe still serves the fastest single read. That stops being the common case once the fleet grows. How a local NVMe cache, network block volumes, a cache you build, and Archil compare | Property | Local NVMe cache | Network block volumes | Build your own | Archil | | --- | --- | --- | --- | --- | | Fastest read | Best: never leaves the box | A network round trip | A network round trip | At least 1 ms over the network | | Shared by every pod | No | No, attached to set machines | Yes | Yes | | A new pod starts | Cold | Cold, on a volume of its own | Warm, once it's built | Warm | | Hot bytes cached again | On every node that reads them | On every volume | No | No | | What you run | SSD machine shapes | Volume counts that track pods | A distributed cache, and the team for it | A CSI mount | ### Local NVMe cache Fastest read Best: never leaves the box Shared by every pod No A new pod starts Cold Hot bytes cached again On every node that reads them What you run SSD machine shapes ### Network block volumes Fastest read A network round trip Shared by every pod No, attached to set machines A new pod starts Cold, on a volume of its own Hot bytes cached again On every volume What you run Volume counts that track pods ### Build your own Fastest read A network round trip Shared by every pod Yes A new pod starts Warm, once it's built Hot bytes cached again No What you run A distributed cache, and the team for it ### Archil Fastest read At least 1 ms over the network Shared by every pod Yes A new pod starts Warm Hot bytes cached again No What you run A CSI mount > “Using a shared SSD cache turns out to be cheaper than local NVMe because you don't have to recache the same bytes over and over again.” Hunter Leath, [Beating NVMe by extending POSIX](https://archil.com/post/beating-nvme-by-extending-posix) [Data-intensive work beyond analytics](https://archil.com/use-cases/data-intensive) ## Bring your query fleet. [Talk to an integration specialist](https://cal.com/team/archil/demo)[Start building](https://console.archil.com)