Back to articles

Persistent storage for AI agent sandboxes that survives restarts

What survives an AI agent sandbox restart, and how durable disks, microVM isolation, egress rules and fast branching keep an agent’s work across sessions.

9 min read

Persistent storage for AI agent sandboxes is a layered design decision. What survives depends on whether it is a file, filesystem state, or process memory. A “persistent” sandbox must let an agent resume its work after compute disappears. That requires durable storage with a life of its own, plus explicit rules for recovery, isolation, and shared access.

Why "persistent sandbox" means three different things, and why conflating them breaks agent design

An agent installs a dependency, puts its results in an in-memory buffer, and pauses. On restart, the dependency survives but the results vanish. The package is home, but the homework is gone. The broken assumption is that both shared one survival contract. Files on durable storage survive process death, sandbox stop, and host failure only if their writes reached the durable layer.

Filesystem state has a different contract from individual saved outputs. It has the installed packages, directories, and environment variables written to disk. A disk snapshot can preserve that environment on pause. Tearing down an ephemeral filesystem without snapshotting loses it.

Process memory is a third, separate layer of state. With ordinary disk persistence, running processes, in-memory variables, and open sockets are lost on restart. A memory snapshot can keep execution state, but the application still needs logic to handle dropped external connections. “Resume” therefore needs a precise object attached to it.

What actually survives a sandbox restart under each common architecture

A persistent mount saves what is written inside it. Files outside the mount remain ephemeral, and pausing does not save RAM or running processes in this model. Snapshot-on-stop instead preserves the filesystem image. Control Plane uses this approach, while Vercel Sandbox’s platform overview says sandboxes are persistent by default since May 2026. Snapshots can expire after inactivity unless configured otherwise.

Archil distinguishes committed files, disk snapshots on pause, and process memory. Its persistent disk remains intact when a sandbox stops, and a fresh Firecracker microVM boots from that disk on restart. Pausing instead snapshots memory alongside the disk, so a resumed sandbox keeps its running processes. Choosing between stop and pause trades resume speed against preservation cost.

Blaxel preserves memory and filesystem state during standby. Standby billing covers snapshot storage rather than compute, and resumption takes under 25 ms, though dropped connections still need repair.

A tiered block store over NBD has another durability boundary. Writes first reach an in-cluster tier, then move asynchronously to object storage such as GCS or S3. The guest sees an ordinary filesystem, but a crash before write-back completes creates a durability gap. A successful local write is not automatically a durable one.

Why durable storage must be separate from compute for agents to resume useful work

An agent sandbox provides compute. It starts, runs code, and can disappear during a worker failure, scale-down, or infrastructure event. If its state lives only in an ephemeral container layer or an unbacked in-VM disk, that state shares the sandbox’s lifespan. A separately addressable disk breaks that dependency. Storage can remain even when no compute instance is running.

That separation makes recovery on another host practical. With Archil, each Firecracker microVM mounts an independently durable disk, so installed packages and intermediate files outlive the compute instance. The agent can get its working environment back while treating its old process as gone. Otherwise, long-running agents repeatedly clone repositories, reinstall dependencies, and regenerate artifacts. Each new session becomes a subscription to doing yesterday’s work again.

The isolation decision for microVMs, gVisor, and containers

Durable state is useful only when the code that uses it is safely isolated. Standard containers share the host kernel, so a kernel vulnerability can expose the host or other tenants. They are not sufficient for unreviewed agent-generated code. gVisor interposes a user-space kernel, keeping most system calls away from the host kernel. Northflank, for example, uses gVisor for GPU sandboxes and microVMs for CPU sandboxes.

MicroVMs give each workload its own guest kernel, with KVM as the shared hypervisor layer. Archil uses Firecracker and prevents sandboxes from reaching each other, private networks, or cloud metadata. Hardware isolation is strong, but it is not unconditional. Firecracker lacks GPU passthrough, so GPU workloads require gVisor or a full VM. Cold-start costs still matter.

Firecracker published its first escape-class CVEs in 2026. CVE-2026-5747 is an out-of-bounds write in virtio PCI transport, rated CVSS v4 8.7 high, requiring a custom guest kernel or specific snapshot configurations for host code execution. CVE-2026-1386 allows arbitrary host file overwrite through a symlink in the jailer, is rated CVSS v4 6.0 medium, and is easier to weaponize for specific files. These vulnerabilities warrant tracking upstream disclosures while retaining microVM isolation as an option. The practical split remains gVisor for routine execution and GPU work, and microVMs where kernel-level separation is non-negotiable.

Egress control and credential handling as extensions of the isolation boundary

Kernel isolation does not contain unrestricted outbound traffic. An agent with plaintext secrets and arbitrary network access can exfiltrate data without escaping its VM. To limit that risk, the recommended model allows only outbound access that has been approved, instead of waiting for trouble before blocking destinations. Archil’s Free plan uses a platform-managed allowlist for package registries, Git hosts, container registries, OS package mirrors, and model provider APIs, which users cannot change. That is distinct from a customer-controlled default-deny firewall.

On Developer plans and above, teams can set custom domain and CIDR rules, including a default that denies access; a sandbox created without a policy has unrestricted egress. The updateNetwork operation replaces the policy of a sandbox that is running, while getNetwork returns that policy. New rules affect only new connections. Guest iptables or nftables rules cannot override the hypervisor-enforced egress boundary.

Credentials belong outside that boundary as well. On Developer plans and above, an egress proxy adds API keys to outgoing HTTP requests while keeping raw credentials out of the reach of sandbox code. The resulting pattern is to restrict egress during arbitrary execution, allow specific output destinations when needed, and inject credentials at the network boundary. A secret in an environment variable is still a secret handed to the guest.

Giving agents access to large datasets without copying them into the sandbox

Large-dataset access calls for mounting data. Copying petabyte-scale S3, GCS, or R2 data into a sandbox is infeasible in practice. With a lazy POSIX mount, agents can use tools they know, such as ls, cat, grep, and mv, without a copy or ETL step. Models already have a lot of training on bash and file tools. The filesystem avoids adding bespoke tools merely to reach the data.

Archil supports managed disks or mounts of existing S3, GCS, R2, Azure Blob, and S3-compatible buckets. A configured bucket remains the source of truth, with writes copied there in the background. Multi-AZ durability requires fsync or archil sync in place of plain sync. Its S3-compatible API excludes S3 versioning and Object Lock. Amazon S3 Files is another way to share files through POSIX, and its close-to-open consistency lets other agents see writes after close.

On Developer and Team, disks synchronizing to your own bucket do not support checkpoints and branching. Teams on those plans that need both should keep the source-of-truth disk separate from the working disk.

Branching and checkpointing agent workspaces to make parallel attempts and rollbacks routine

A forkable workspace changes how an agent can explore. A harness can checkpoint the starting state, attempt a solution, backtrack after failure, and branch again without destructive writes to the original. That provides an execution substrate for tree-search strategies, including those that use random sampling to explore possible paths. Multi-day rollouts and parallel attempts can keep the starting point they share.

On Archil disks, a checkpoint takes about 15 ms and a branch about 35 ms, regardless of disk size. Each fork runs commands independently from the fork point, and disk branching has been generally available in all AWS regions since June 2026. Branching works on Archil-managed disks on every plan; bring-your-own-bucket disks support it on Enterprise. Fly.io Sprites offers another checkpoint-and-restore option, with 100 GB persistent NVMe per sandbox.

Shared and per-session workspaces for agents that run across multiple machines or conversations

Persistent context need not belong to one conversation or host. Archil disks mount on multiple machines at once and expose an S3-compatible API, so agents, pipelines, and monitoring tools can access the same files without copying them. S3 Files likewise provides shared access across EC2, Lambda, EKS, ECS on Fargate, and Bedrock AgentCore Runtime.

Archil has no distributed file locking, and flock and fcntl operate per client. Concurrent writers across machines must coordinate through read-only or conditional mounts and archil checkout. New files can also take up to five minutes, by default, to appear in an already-running reader’s directory listings unless it runs archil invalidate-cache or shortens the cache with archil set-cache-expiry. That delay affects real-time coordination designs. The recommended division puts task queues, locks, and assignment tables in a database, while files, artifacts, and intermediate outputs stay on the shared disk.

Per-tenant storage isolation for multi-tenant agent products at scale

The recommended pattern gives each tenant a scoped disk and its own credential, so execution isolation is only one part of the protection. Archil separates account-level control-plane API keys from disk tokens that mount one specific disk. Tokens are created automatically with a disk and shown once. A compromised disk token exposes only that disk.

This design also avoids allocating identical capacity to unequal tenants. Archil bills on active data rather than provisioned capacity, so a tenant using 1 MB pays for 1 MB. Bring-your-own-bucket storage also sets a boundary on where tenant data lives, letting it stay in its own cloud account and region. Cache nodes run in the same location as compute, and data moves across regions only when configured to do so. Revoking access leaves data in the customer’s bucket, making the storage service an access layer.

Deciding which persistence model fits a given agent workload

A team choosing a persistence model should ask three questions. What state must survive, how will machines and sessions access it, and which failures must recovery cover. Files and installed packages need a persistent volume or filesystem snapshot. Large external datasets call for lazy bucket mounting, with a separate working disk if branching is needed. Untrusted code calls for microVM isolation and upstream vulnerability tracking, while GPU requirements can favor gVisor.

Workflow requirements then narrow the implementation choices. Parallel exploration needs fast branching rather than full copies, and shared writes need explicit coordination. Long sessions require checking plan limits, since some platforms cap sessions at 24 hours. Northflank has no fixed session time limit, offers persistent volumes up to 64 TB with multi-read-write support, and separates CPU microVMs from GPU gVisor sandboxes. Archil combines separate persistent disks, Firecracker isolation, branching in tens of milliseconds, and bucket mounting across the supported clouds.

Control Plane combines Kata Containers microVMs with filesystem snapshot-and-restore, configurable inactivity expiry, and Universal Cloud Identity for short-lived AWS, GCP, and Azure credentials. Blaxel offers memory-plus-filesystem standby with zero compute cost and resumption in under 25 ms. Sprites provides roughly 300 ms checkpoint creation, roughly 1 second restore, and stops charging for idle compute. Design the environment so agents can resume useful work, rather than bolting persistence on after the work has already disappeared.

Hunter Leath

Founder & CEO