# 3. Many agents, one machine

Source: https://archil.com/guides/building-agents/many-agents-one-machine · Part of How to build agents · Facts checked 2026-11-05

**The short answer:** Run the harness as ordinary multi-tenant server code. One process holds every open conversation, each command runs on per-call compute that sees only its tenant’s workspace, and each conversation’s history lives with that tenant’s files, so any process can pick it up. This chapter builds it with pi as the harness, and Archil’s serverless execution, S3 API, and disks underneath.

Serve many agents by running the harness as ordinary multi-tenant server code, the design that [Two places to run an agent](https://archil.com/guides/building-agents/two-places-to-run-an-agent) argues for. Follow one message through it. A user types “the build is red, fix it,” and it reaches a harness process that already holds hundreds of other conversations. The process finds this conversation’s pi session, or loads it from the user’s disk, and calls `prompt()`. The model answers with `bash: npm test`, which runs in a microVM with only this user’s workspace mounted, and the output comes back to the same process. The process never ran the test, and it never held a machine for this user.

This chapter builds that process with pi as the harness and Archil for the tool calls and storage. The part that needs the most care is how each of pi’s tools reaches the right tenant’s files. After that come history, isolation, load, and where to run the host.

## One process, many sessions

[pi](https://github.com/badlogic/pi-mono) is an open-source coding agent whose SDK, `@earendil-works/pi-coding-agent`, runs the whole agent inside your own Node process. `createAgentSession()` returns an `AgentSession` that owns one conversation: its model, its tools, its queued messages, and its history. Nothing ties a session to the process’s working directory or to a machine, so one process can hold a session for every open conversation. (From pi’s [SDK guide](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/sdk.md) and the `createAgentSession` options in [`sdk.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/src/core/sdk.ts).)

Here’s a session, opened for one conversation of one tenant:

```typescript
import * as archil from "disk";
import { getModel } from "@earendil-works/pi-ai/compat";
import { createAgentSession, ModelRuntime, SettingsManager } from "@earendil-works/pi-coding-agent";

// One model and one provider key for the whole process.
const modelRuntime = await ModelRuntime.create({ authPath: "/srv/harness/auth.json", modelsPath: "/srv/harness/models.json" });
await modelRuntime.setRuntimeApiKey("anthropic", process.env.ANTHROPIC_API_KEY!);
const model = getModel("anthropic", "claude-sonnet-4-5")!;

export async function openSession(diskId: string, threadId: string) {
  const disk = await archil.getDisk(diskId);
  // Each tenant's files appear at a path of their own; the next section explains why.
  const cwd = `/mnt/archil/${diskId}`;
  const history = await loadHistory(disk, threadId, cwd);

  const { session } = await createAgentSession({
    cwd,
    model,
    modelRuntime,
    settingsManager: SettingsManager.inMemory(),
    sessionManager: history.manager,
    tools: ["read", "bash", "edit", "write"],
    customTools: diskTools(disk, cwd),
  });
  saveHistory(session, history, disk, threadId);
  return session;
}
```

The `tools` list keeps pi’s four default tools and nothing else. `customTools` replaces each of them with a version that works on the tenant’s disk, because a custom tool with a built-in’s name takes its place. An in-memory `SettingsManager` keeps one tenant’s settings out of the files every session would otherwise share. A message that arrives while the session is mid-run goes through `session.steer()` or `session.followUp()`, which pi requires instead of a second `prompt()`.

**One process, every conversation**

A harness host runs one Node process with a pi session for each open conversation: tenants a, b, c, and d. Tenant a’s session runs `npm test` and tenant c’s runs `rg TODO src`; each starts a microVM that mounts only that tenant’s disk, returns its output to the session, and is gone. Tenant b’s session reads `src/app.ts`, tenant d’s edits `README.md`, and tenant a’s appends to its history; each of those goes to the tenant’s disk through Archil’s S3 API, with no microVM. Each disk holds a `workspace/` directory and a `sessions/` directory. Between calls, no microVM runs.

## Bash on serverless execution

pi’s bash tool doesn’t spawn processes itself. It calls a `BashOperations` object with `exec(command, cwd, options)`, and the SDK lets you pass your own. Ours sends each command to Archil’s serverless execution:

```typescript
import * as archil from "disk";
import type { BashOperations } from "@earendil-works/pi-coding-agent";

function bashOps(diskId: string): BashOperations {
  return {
    async exec(command, cwd, { onData }) {
      const { stdout, stderr, exitCode } = await limit(() =>
        archil.exec({
          // Mounted at /mnt/archil/<diskId>: the workspace only, writable without a checkout.
          disks: { [diskId]: { disk: diskId, subdirectory: "workspace", conditional: true } },
          command: `cd ${cwd} && ${command}`,
        }),
      );
      onData(Buffer.from(stdout + stderr));
      return { exitCode };
    },
  };
}
```

Mounting the disk under its own ID gives every tenant’s files a different path, which the file tools rely on. `subdirectory: "workspace"` keeps the conversation history, stored beside it, out of the command’s reach. `conditional: true` lets the command write without a checkout, as [Serverless tool calls](https://archil.com/guides/building-agents/serverless-tool-calls) recommends for a bash tool. `limit()` is the back-pressure wrapper from later in this chapter.

Some of pi’s local behavior doesn’t carry over. The output arrives in one piece when the command finishes, instead of streaming. A call takes no timeout or abort signal, so a call the user cancels keeps running until it finishes or reaches the 5-minute cap. Nothing but the disk survives between calls, including the environment, so `exposeSessionEnvironment: false` turns off the `PI_*` variables pi would otherwise try to pass. The other limits are in [Serverless tool calls](https://archil.com/guides/building-agents/serverless-tool-calls#limits). (From [Serverless execution](https://docs.archil.com/compute/serverless-execution#limits) and the `exec` signatures in the [`disk` package](https://www.npmjs.com/package/disk). pi’s `BashOperations` and `exposeSessionEnvironment` are in [`bash.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/src/core/tools/bash.ts).)

**A serverless tool call**

The agent’s harness sends `npm test` to the disk. A microVM starts with the disk mounted, runs the command, returns its output, and shuts down. The call bills $0.00005. Until the next call, nothing runs and no compute bills, and the files stay on the disk.

## File tools in the harness

pi’s file tools are different: `read`, `write`, and `edit` run inside the harness process and expect a local path. Here the files aren’t local, so each tool needs another way to reach the tenant’s storage. There are three, and Archil supports each.

**Run every file operation as a tool call.** `read` becomes `cat` in a microVM. Each read then pays for a microVM start and counts as a call. A read is capped at the output limit, and a write has to fit its whole contents into the command.

**Use an object API from the harness.** Every Archil disk is also an S3-compatible bucket, and the `disk` package calls it with `getObject`, `putObject`, and `headObject`. A read is one HTTPS request, with no microVM. The S3 view is strongly read-after-write consistent with committed writes, and a tool call commits its writes before it returns. So the harness reads what `npm test` wrote, and the next tool call reads what `edit` saved. Our own agent tools for the AI SDK, Mastra, and LangChain work this way. (From the [S3 API](https://docs.archil.com/protocols/s3-api) and [TypeScript SDK](https://docs.archil.com/sdks/typescript#reading-and-writing-objects) docs. The `read_file` and `write_file` tools in our [SDK repository](https://github.com/archil-data/archil-sdk) call `getObject` and `putObject`, and their bash tool calls `exec`.)

**Mount every tenant’s storage on the harness host.** pi’s tools then work unchanged, but every tenant’s files sit in one directory tree on one machine. pi’s `read` accepts absolute paths and `~`, so a model that asks for `../tenant-b/.env` reaches another tenant, and only your own path check prevents it. A mount also serves from its own cache, which can lag a tool call’s writes until it refreshes.

We use Archil’s S3 API. It keeps the harness off any per-tenant mount, and each session holds a `Disk` object that can only address its own tenant’s bucket.

**Three ways to read one file**

Three ways for pi’s read tool, running in the harness, to reach `src/app.ts` on tenant a’s disk. Through exec, a microVM starts, runs `cat src/app.ts`, and returns stdout, so every read costs a microVM and a billed call. Over the S3 API, a `GetObject` request for the key `workspace/src/app.ts` returns the bytes from tenant a’s disk. With every tenant’s disk mounted on the host, the read reaches `/mnt/tenant-a`, and a read of `../tenant-b/.env` reaches another tenant’s files, which only the harness’s path check would stop.

pi’s `ReadOperations`, `WriteOperations`, and `EditOperations` interfaces take absolute paths, so the only work is mapping a path to a key and refusing anything outside the workspace:

```typescript
import { isAbsolute, relative, sep } from "node:path";
import type { Disk } from "disk";
import {
  createBashToolDefinition, createEditToolDefinition, createReadToolDefinition, createWriteToolDefinition,
  type ReadOperations, type ToolDefinition,
} from "@earendil-works/pi-coding-agent";

function diskTools(disk: Disk, cwd: string): ToolDefinition[] {
  // /mnt/archil/<diskId>/src/app.ts -> workspace/src/app.ts; nothing outside cwd.
  const key = (path: string) => {
    const rel = relative(cwd, path);
    if (!rel || isAbsolute(rel) || rel.split(sep)[0] === "..") throw new Error(`${path} is outside the workspace`);
    return `workspace/${rel}`;
  };
  const readFile = async (path: string) => Buffer.from(await disk.getObject(key(path)));
  const writeFile = async (path: string, content: string) => {
    await disk.putObject(key(path), content);
  };
  const access = async (path: string) => {
    if (!(await disk.objectExists(key(path)))) throw new Error(`No such file: ${path}`);
  };
  const read: ReadOperations = { readFile, access };

  return [
    createBashToolDefinition(cwd, { operations: bashOps(disk.id), exposeSessionEnvironment: false }),
    createReadToolDefinition(cwd, { operations: read }),
    createEditToolDefinition(cwd, { operations: { readFile, writeFile, access } }),
    // PutObject creates parent directories, so mkdir has nothing to do.
    createWriteToolDefinition(cwd, { operations: { writeFile, mkdir: async () => {} } }),
  ];
}
```

A few details follow from this design:

- **Parallel calls.** pi runs the tool calls in one model response concurrently. Its `edit` and `write` queue per absolute path, process-wide, so a separate `cwd` per tenant keeps two tenants’ `src/app.ts` from waiting on each other. A bash call isn’t in that queue, so it can race an edit to the same file, as on a laptop.
- **Checkouts.** If a command checks out a path, a `PutObject` to that path returns `409 Conflict` while the command writes it. pi reports a thrown error to the model as a failed tool call.
- **Search.** pi’s `grep` always runs `rg` on the local machine, so we leave it out, along with `find` and `ls`. The model runs `rg` through bash, or you can wrap [`disk.grep`](https://docs.archil.com/compute/search-files) as a custom tool for large trees.
- **Large and binary files.** Without a `createReadStream`, pi’s `read` downloads the whole file before it picks lines. Without a `detectImageMimeType`, it treats every file as text. (From pi’s [extensions guide](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/extensions.md#remote-execution), the operations interfaces in [`read.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/src/core/tools/read.ts), [`edit.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/src/core/tools/edit.ts), and [`write.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/src/core/tools/write.ts), and the [S3 API’s limitations](https://docs.archil.com/protocols/s3-api#limitations).)

## History on the tenant’s disk

A conversation’s history belongs with the tenant’s files, not on whichever host ran it. By default, pi’s `SessionManager` appends a conversation’s entries to a JSONL file on the local machine, which ties the conversation to one host. We keep the file on the tenant’s disk instead, under `sessions/` and outside every tool call’s mount, so the model’s code can’t read or rewrite its own history.

`SessionManager.inMemory()` accepts entries held outside the file system, and the disk’s `appendObject` adds bytes to the end of a file. Load the history when a conversation opens, and append new entries after each turn:

```typescript
import { parseSessionEntries, SessionManager, type AgentSession } from "@earendil-works/pi-coding-agent";

const historyKey = (threadId: string) => `sessions/${threadId}.jsonl`;

async function loadHistory(disk: Disk, threadId: string, cwd: string) {
  const key = historyKey(threadId);
  const saved = (await disk.objectExists(key))
    ? parseSessionEntries(new TextDecoder().decode(await disk.getObject(key)))
    : undefined;
  return { manager: SessionManager.inMemory(cwd, undefined, saved), written: saved?.length ?? 0 };
}

function saveHistory(session: AgentSession, history: { manager: SessionManager; written: number }, disk: Disk, threadId: string) {
  let saving = Promise.resolve();
  session.subscribe((event) => {
    if (event.type !== "turn_end" && event.type !== "agent_settled") return;
    saving = saving.then(async () => {
      const entries = [history.manager.getHeader()!, ...history.manager.getEntries()];
      const lines = entries.slice(history.written).map((e) => `${JSON.stringify(e)}\n`);
      if (lines.length) await disk.appendObject(historyKey(threadId), lines.join(""));
      history.written = entries.length;
    });
  });
}
```

pi appends a turn’s entries before it ends the turn, so each `turn_end` saves a finished turn. Each append can carry at most 1 MiB, so split a larger batch. The SDK doesn’t retry an append after a transient error, because a retry could write the bytes twice, so check the file’s size before you append again. (From pi’s [SDK guide](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/sdk.md#session-storage) and [`session-manager.ts`](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/src/core/session-manager.ts), and our [S3 API](https://docs.archil.com/protocols/s3-api#appending) and [TypeScript SDK](https://docs.archil.com/sdks/typescript#reading-and-writing-objects) docs.)

With the history on the disk, a process can exit, and the next message can go to any other process, which reads the same file and carries on. The disk doesn’t decide which process owns a conversation, though. Route each conversation to one process at a time, with sticky routing or a lease in your own database.

**A conversation changes hands**

Harness process A runs thread 42 for tenant a and appends each finished turn to `sessions/thread-42.jsonl` on tenant a’s disk: turns 1, 2, and 3. Then process A exits. The user’s next message is routed to harness process B, which loads the 3 turns from the disk and appends turn 4.

## Isolation and credentials

The design has three trust zones, and each holds only what it needs:

- **The harness is trusted.** It holds the model provider’s key, the Archil API key, and every tenant’s session. An API key can read, delete, and run sandboxes on every disk in its organization and region, so it never leaves the harness.
- **Each tool call is untrusted.** The exec request carries only the command and its mounts. The microVM sees one tenant’s `workspace/` directory, and nothing else from Archil: not that tenant’s history, and not any other tenant’s disk.
- **Each tenant’s data is on its own disk.** One disk per tenant makes the disk the unit of isolation and of deletion. Plans cap disks per region at 1,000 on Developer and 25,000 on Team, so with a disk per tenant, those are your tenant caps too. Enterprise has no cap. (From [Authorization](https://docs.archil.com/concepts/authorization#api-key-authorization), the [S3 API’s authentication](https://docs.archil.com/protocols/s3-api#authentication), and our [limits](https://docs.archil.com/reference/limits#disks-per-plan).)

Inside the harness, the boundary between tenants is your code: one `Disk` object per session, and the `key()` check. To have the S3 API enforce it too, give each session disk-scoped credentials from `disk.createS3Credentials({ nickname, ttl })` and use a standard S3 client. They return `AccessDenied` on any other disk. `exec` has no per-disk credential, so the API key stays in the process either way.

A tool call that needs a secret, such as a private registry token, is the hard case. `exec` takes no environment, so the secret travels in the command or sits on the disk, where the model’s code can read it. That’s no worse than a sandbox, but give tool calls only the credentials they need.

**What each part can reach**

Your harness, trusted code, holds the model API key, the Archil API key, and every tenant’s sessions. It sends a microVM only the command text. The microVM, which runs the model’s untrusted command, mounts tenant a’s `workspace/` directory. Tenant a’s `sessions/` directory isn’t mounted, and neither is tenant b’s disk, so attempts to reach them fail. The keys never leave the harness process.

## How much one host can take

Each session spends almost all of its time awaiting the model or the user, and one event loop interleaves those waits. The work per session is small: streaming the response, truncating output, and applying edits. The tool calls run in microVMs, not on the host. The host has to absorb three kinds of pressure:

- **Tool calls in flight.** An account can run hundreds of exec calls at once by default, and thousands on request. A limiter in the process keeps a burst from turning into errors.
- **The model provider’s rate limits.** pi retries failed model requests, and its settings set how often.
- **Open sessions.** Each one holds its history in memory, so cap how many one process opens and route the rest elsewhere.

Here’s the limiter `bashOps` uses. A finishing call hands its slot directly to the next waiting one:

```typescript
const MAX_IN_FLIGHT = 100; // under your account's exec limit
let inFlight = 0;
const waiting: (() => void)[] = [];

async function limit<T>(run: () => Promise<T>): Promise<T> {
  if (inFlight < MAX_IN_FLIGHT) inFlight++;
  else await new Promise<void>((go) => waiting.push(go));
  try {
    return await run();
  } finally {
    const next = waiting.shift();
    if (next) next();
    else inFlight--;
  }
}
```

How many calls are in flight depends on how much of each session is tool calls. The next figure simulates one host from the loops measured in [Two places to run an agent](https://archil.com/guides/building-agents/two-places-to-run-an-agent). It treats sessions as independent, and real traffic bunches up, as when a team starts work at nine, so set the limiter from the peaks you observe. (The account limit is from [Serverless execution](https://docs.archil.com/compute/serverless-execution#limits), and the retry settings from pi’s [settings reference](https://github.com/badlogic/pi-mono/blob/main/packages/coding-agent/docs/settings.md).)

**Many sessions, one host**

An explorable host running from 10 to 1,000 pi sessions, shown as dots that light up while a session is in a tool call, with a square for each microVM in flight and a chart of vCPU over the last moments. Each call counts as the 2 vCPU a serverless call gets, and the comparison is one 2 vCPU sandbox per agent, kept up for the whole session. The share of time in tool calls comes from 384 of our coding-agent loops with a 15-minute auto-stop: 12% in the median loop and 35% across all loops. The harness host’s size is the reader’s choice: 4, 8, or 16 vCPU. At 500 sessions, 12%, and an 8 vCPU host, one sandbox per agent reserves 1,000 vCPU; the host and microVMs use 128 vCPU on average and 174 at the busiest moment in 1,000, which is 17% of the sandboxes. At 35%, they use 358 on average and 424 at the busiest moment, 42%. At 1,000 sessions and 12%, sandboxes reserve 2,000 vCPU, and the host and microVMs use 248 on average and 314 at the busiest moment. At 10 sessions, the host and microVMs reach 18 vCPU at the busiest moment, 90% of the sandboxes’ 20, and the host is most of that.

## Where the harness runs

The harness is a long-running Node service with no files of its own, so any machine that reaches your execution and storage APIs can run it. With Archil, two homes fit.

**One Archil persistent sandbox.** A sandbox can have up to 32 vCPU and 64 GiB of memory, it runs in the same region as your disks, and transfer within Archil is free, so the harness’s S3 reads cost nothing to move. Set `idleTtlSeconds: 0`, or the sandbox pauses after 30 idle minutes, and expose the port your API server calls. A session still ends after 24 hours: the sandbox pauses with its processes kept, and the next request on the exposed port resumes it. Maintenance can pause it too. Requests in flight when it pauses can fail, and those conversations resume from their disks. Run it on the Developer plan or above. There, a sandbox created without a network policy has unrestricted egress, and if you set a deny-by-default policy, allow your model provider and Archil’s API. On the Free plan, every sandbox uses a managed allowlist, and you can’t add rules of your own.

**Your own servers or containers.** The harness deploys like the rest of your backend, with health checks, rolling deploys, and more than one replica. The cost is transfer: data leaving Archil, which includes the harness’s S3 reads, costs $0.07 per GB. (From [Persistent sandboxes](https://docs.archil.com/compute/sandboxes/introduction#setting-a-time-to-live), the [sandbox firewall](https://docs.archil.com/compute/sandboxes/firewall), our [limits](https://docs.archil.com/reference/limits#sandboxes), and [Pricing & Billing](https://docs.archil.com/administration/billing#data-transfer).)

Start on one persistent sandbox. It’s the shortest path to every conversation on one machine, and the disks make the daily pause a short interruption. Move to your own platform when you need a second host or deploys without a pause. Nothing else in this chapter changes, because every conversation’s state is on its tenant’s disk.

**A day on one persistent sandbox**

Thirty hours of a harness on one persistent sandbox with its idle timeout off. It runs until its 24-hour session limit, then pauses with its processes kept. A request on its exposed port at about 25.5 hours resumes it, starting a new session. The tenant disks hold the workspaces and conversation histories the whole time.

Most teams start from an agent that already lives in a sandbox. [Migrating from an in-sandbox agent](https://archil.com/guides/building-agents/migrating-from-an-in-sandbox-agent) moves that agent onto this design one step at a time.