Engineers use "shared memory" loosely in multi-agent conversations. That looseness is an issue, because the phrase usually smuggles in one of four genuinely different things at once.
In-context memory lives inside the active context window. It's fast, volatile, private to one agent, and gone when the session ends. External retrieval covers your vector stores and databases which is great for semantic search but not designed for live collaboration. Episodic memory tracks past interactions across sessions. This is useful for a single agent picking up where it left off, but not built for concurrent writers. And then there's persistent shared workspace: a mutable, addressable store any agent can read or write at any time.
Most purpose-built agent memory tools target the second category, with use cases for retrieval on query and semantic search over past data. That's genuinely useful. However, retrieval is also architecturally different from coordination, and most teams get it wrong by conflating the two.
Helping one agent remember is not the same problem as letting two agents safely build the same evolving plan without stepping on each other. Retrieval is a library. Coordination is a conversation. These require different tools, and that is what this article covers: specifically, how to structure persistent shared workspaces, which POSIX primitives make concurrent agent coordination safe, and how to pick the right storage architecture for the access patterns agents actually produce.
Why Standard Cloud Storage Falls Short — and What the Filesystem Fixes
Object storage is the default for cloud AI workloads. Its flat namespace and high throughput suit high-scale workloads well, but it does not suit concurrent coordination well — understanding exactly why leads directly to what the filesystem can provide instead.
The core problem is that basic object storage APIs provide weak concurrency guarantees by default. Two agents can read the same object, both decide to modify it, and the last writer wins with no conflict signal — a classic read-modify-write race with no protection at the storage layer. The second agent's work disappears silently — no error, no warning, no indication anything went wrong. It's worth noting that some object stores do offer conditional writes or object versioning, which can mitigate some of these issues, but these features require explicit design choices and add application-level complexity. Without careful coordination logic layered on top, duplicate effort and lost updates are not edge cases. They are expected behavior under any meaningful degree of parallelism.
The access pattern mismatch compounds the problem. Agent coordination workloads are almost entirely small-file operations — the kind of metadata-heavy, latency-sensitive requests that object stores handle poorly. Bandwidth is not the bottleneck; metadata latency is. Using object storage directly as a coordination substrate can force teams to rebuild in application code the invariants that a proper POSIX-compliant filesystem provides at the layer below. The standard way to discover this is watching something break in production.
What a filesystem provides that is difficult to replicate otherwise is a shared, addressable namespace with no broker in the middle. Agent A writes a plan to plans/v1.md. Agent B reads it, revises it to plans/v2.md. Agent C audits both. There's no handoff protocol, no message queue, and no central orchestrator routing information between them.
Late-joining agents can orient themselves by walking the directory. They can read the files and get the full picture without anyone briefing them. Failures stay localized. An agent that crashes mid-task leaves its partial work exactly where it stopped and another agent picks it up without ceremony. Files are not inherently auditable — meaningful auditability requires discipline around naming, versioning, and attribution — but a filesystem makes those practices straightforward to enforce.
The CORAL multi-agent framework (arxiv 2604.01658) is a useful concrete example here. It implements shared persistent memory as a structured filesystem inside a .coral/public/ directory, with subdirectories for attempts/, notes/, skills/, and heartbeat/. That public layer is symlinked into each agent's worktree. Each agent gets a local worktree view, so they share a common truth. The symlink is the coordination primitive. Build something more complicated and the architecture eventually converges on exactly this.
One pattern worth considering in production: object storage as the source of truth, with a POSIX filesystem layer mounted over it as the interface agents actually touch. Reads hit a cache. Writes flush asynchronously back to the bucket. Agents see POSIX. The bucket stays authoritative. Each layer does what it was built to do. That said, it's important to be clear-eyed here: a filesystem layer over object storage does not automatically provide full POSIX semantics. Implementations vary significantly in how faithfully they support things like atomic rename, locking, or consistent directory reads — so the guarantees you can rely on depend heavily on the specific solution you deploy. But that layered architecture, when implemented carefully, only holds if the filesystem itself provides the right guarantees, which brings us to the specific primitives that make coordination safe.
The POSIX Primitives That Make the Model Work
A generic shared folder isn't enough on its own. The coordination properties that make the filesystem model work depend on specific guarantees. POSIX defines them, and each one closes a failure mode that unmediated object storage leaves open — though it's worth emphasizing that not every filesystem implementation delivers all of these reliably, especially in distributed or cloud-mounted contexts.
Atomic rename (rename(2)) is the foundational one. An agent writes its result to a temporary file, then renames it to the canonical path. Readers see either the old version or the complete new version — there is no window in which a partial write is visible. This prevents agents from reading corrupted intermediate state. However, atomic rename does not eliminate read-modify-write races: if two agents each read the current file, independently compute an update, and then both rename their result into place, the last rename wins and silently discards the other's changes. Atomic rename ensures writes are all-or-nothing; it does not provide compare-and-swap semantics. Closing the read-modify-write race requires explicit coordination — either file locking, a compare-and-swap mechanism at the application layer, or a design that avoids concurrent modification of the same file entirely. Some object stores offer conditional writes (also called optimistic concurrency control) to address this, but this requires explicit application-level design rather than being guaranteed by the storage primitive itself.
File locking (flock/fcntl) lets an agent claim exclusive access to a task file before working on it, preventing two agents from racing on the same work item. The filesystem becomes the queue — no external job scheduler required and no separate locking service to maintain. However, a critical caveat: advisory locks like flock only coordinate processes that explicitly participate in the locking protocol. Agents running as separate clients, potentially on different hosts, cannot rely on these locks to coordinate with each other unless the underlying distributed filesystem implements and enforces them correctly across client boundaries. Many do not. Verify that your specific implementation supports distributed lock semantics before depending on them for cross-agent coordination.
Symlinks let you update current/ to point to a new directory of results without breaking any agent holding a handle to the old path. CORAL's worktree design exploits this directly. However, symlink replacement is only atomic if it is implemented via rename(2) on the symlink itself — simply calling symlink() to overwrite an existing symlink is not atomic on all platforms and can fail or produce inconsistent results. When done correctly with rename, updates propagate without notifications. The directory tree becomes a signaling layer without any signaling infrastructure.
Hard links, memory mapping (mmap), fsync, and sparse files each enable a class of access pattern that agents or the code they run actually depend on: shared data with no copy, memory-mapped model weights, durable checkpoints, efficient large-file representations. What matters most is sufficiently complete POSIX semantics so that existing code runs as expected. None of these primitives is a nice-to-have; each closes a gap that direct object storage access leaves open and that application code would otherwise have to paper over.
This is why the object storage → POSIX layer architecture matters when it's implemented well. You're not just adding a cache in front of a bucket. You're adding a contract. Agents and the tools they call can rely on rename atomicity, lock semantics, and consistent directory reads, but only if the implementation actually provides them. The question then becomes whether the implementation is fast enough to support the access patterns agents actually produce — because getting the semantics right and getting the performance right are two different problems.
The Performance Reality That Makes or Breaks This Model
The coordination model only works if the underlying filesystem is fast enough. This is where otherwise good architectural thinking falls apart, usually because training and inference get most of the attention and teams treat coordination as an afterthought until the slowness becomes noticeable.
Training storage needs high sustained throughput for dataset reads and burst write capacity for checkpoints. Inference needs low latency for model loading and key-value (KV) cache access. Agent coordination is a third profile: dominated by metadata throughput and small-file latency. The bottleneck is not bandwidth. This matters because the instinct when something is slow is to add capacity, which does nothing if the constraint is latency on small operations.
Training teams learned this lesson with data loading: GPUs sit idle waiting on input pipelines, and more storage capacity doesn't help, because the bottleneck is the access pattern. The same mistake applies to coordination workloads. Adding more object storage to a latency-bound problem is like widening a highway to fix a broken traffic light. You're solving for the wrong constraint.
A low-latency cache in front of object storage directly addresses the coordination profile. The practical architecture: hot working files in a fast cache, cold datasets and completed artifacts in object storage, with a filesystem layer presenting a single namespace across both. Agents see one tree. The system decides where each byte lives. The semantics are correct because of careful POSIX implementation, and the speed is right because the hot path never touches the bucket. Together, they make the model viable at the pace agents actually work.
What Persistent, Shared Agent Context Actually Changes About How You Build
The dominant model today: stateless agents that hold no context between sessions. Each run reconstructs context by reloading from a database or requerying a vector store. This is expensive, lossy, and architecturally fragile for multi-session tasks. It's also the default because it's the path of least resistance, not because it's a good idea.
Persistent shared context, backed by a properly cached POSIX filesystem, changes the design assumptions in concrete ways. An agent can pick up mid-task where another left off, because the partial work is sitting exactly where it was left. Parallel agents divide work without a scheduler, each locking a task file, working on it, and writing results back, with the filesystem handling coordination — provided locking is implemented correctly across all participating clients. A new agent joining a running system can inspect the full workspace history without any prior briefing, as long as the team has enforced consistent naming and versioning conventions. And the agent can discover context as it works rather than having it front-loaded into the prompt — the agent navigates to what it needs rather than being handed everything upfront.
A significant portion of the "data readiness" problem enterprises report has less to do with data quality and more to do with access architecture. Data that exists in buckets but cannot be reached through a coherent, fast, POSIX-compliant interface is effectively unavailable to agents. The data is there. The access model is broken. Fixing it rarely requires migrating data; it often requires putting the right filesystem layer in front of what already exists — provided teams choose and configure that layer with care.
The shift in practice: build agent systems around a persistent shared workspace first, then attach compute. Not the other way around. This means teams stop provisioning storage as an afterthought and start designing it as the coordination layer. The filesystem doesn't replace databases or vector stores. Structured data with a clean schema still belongs in a database. Semantic retrieval still benefits from vector indexing. But evolving plans, intermediate results, and coordination state belong in the filesystem, where the access model matches how agents actually work.
Where Archil fits
Archil is built on this layered model. It mounts an existing Amazon S3, Google Cloud Storage, Cloudflare R2 or Azure Blob bucket, or Archil-managed storage, as a POSIX filesystem. Data in its durable, multi-Availability-Zone cache is served in under a millisecond, a write is durable once fsync returns, and changes synchronize back to your bucket as ordinary objects any S3 tool can read.
Of the primitives above, it supports atomic renames (including directory renames), symbolic links, hard links, and flock and fcntl locks. Those locks behave exactly as on a local disk for every process on one client, but they are not shared between clients. Agents on different machines coordinate writes through Archil's checkout and checkin ownership model, or mount in conditional mode, where several clients can write and the application coordinates them. Checkpoints (about 15 ms) and branches (about 35 ms), regardless of disk size, let parallel agents start from the same state and keep the result that wins.