How fast is a serverless tool call for an AI agent?

A serverless tool call on Archil adds tens of milliseconds to a command. In our measurement from a laptop to a disk in AWS us-west-2, a trivial command (true) came back in 82 ms at the median and 95 ms at p90 end to end, internet round trip included, and Archil reported a median of 37 ms from the request arriving to the command starting, including booting a microVM and mounting the disk. For anything an agent actually runs, the command itself takes longer than the overhead.

How fast is a serverless tool call for an AI agent?

A serverless tool call runs one command in a fresh microVM with the agent's disk mounted, returns the output and shuts the microVM down. The worry is obvious: if every ls has to start a machine, does the agent crawl?

It depends on how fast the machine starts. Archil returns the breakdown with every call, so you don't have to guess. Each disk.exec() result includes:

• timing.queueMs: time spent queueing, scheduling, booting a microVM and mounting the file system before the command started.
• timing.executeMs: time the command itself ran.
• timing.totalMs: the server's end-to-end time for the request.

What we measured

One disk in AWS us-west-2 holding a clone of the Express repository with its dependencies installed. The client was a laptop on the public internet, about 40 ms from the region. Each row is 50 calls.

Two cases are slower than the table:

• The first call against a newly created disk took 331 ms at the median across five new disks. The second call on each took 72 ms.
• The first call after a pause. Five times, we left the disk untouched for 11 to 20 minutes and then made a call. Those calls took between 0.25 s and 1.1 s end to end. On the server, queueMs was 74 to 219 ms instead of the usual 35 to 40 ms. The rest of the extra time was mostly on the client side, which had to open a new connection after the old one closed. The next call was back to normal in three of the five rounds.

Read the table as overhead plus work. About 40 ms of each end-to-end figure is the round trip from the laptop, which a server in the same region doesn't pay. The queueMs column is the cost of starting a fresh microVM with the disk mounted, around 35 to 40 ms when calls arrive at a normal pace. Everything above that is the command.

Where the time goes in a real tool call

For an agent, the overhead that matters is the part it pays on every call, compared with what the call does:

• Trivial commands (ls, cat, pwd): the microVM start and the round trip are most of the time, and the total is still under a tenth of a second.
• Commands that read many files (rg, find, a type-check): each microVM starts with an empty client cache. The first rg over the repository in a fresh microVM took about 0.4 s; run again in the same microVM it took about 0.12 s. Agents that search large trees should use disk.grep, which fans the search out across many microVMs.
• Builds and tests: the command dominates. The test suite took 4.7 s at the median, of which 36 ms was starting the microVM.

Compare that with the model. In 30 days of our own engineering team's coding-agent transcripts, a model turn took 2.7 s at the median and 9.2 s at p90, so adding tens of milliseconds to a tool call barely changes how long an agent takes to finish a task.

Measure it on your own account

The overhead depends on your region, your client's distance from it and what your tools do, so measure it where your agent runs. This script runs a command 50 times and prints percentiles for the end-to-end time and for queueMs:

import * as archil from "disk";
 
const disk = await archil.getDisk(process.env.ARCHIL_DISK_ID!);
const command = process.argv[2] ?? "true";
const wall: number[] = [];
const queue: number[] = [];
 
for (let i = 0; i < 50; i++) {
  const start = performance.now();
  const { timing } = await disk.exec(command);
  wall.push(performance.now() - start);
  queue.push(timing.queueMs);
}
 
const pct = (xs: number[], p: number) => [...xs].sort((a, b) => a - b)[Math.floor(p * (xs.length - 1))];
console.log(`end to end p50 ${pct(wall, 0.5).toFixed(0)} ms, p90 ${pct(wall, 0.9).toFixed(0)} ms`);
console.log(`queueMs p50 ${pct(queue, 0.5)} ms, p90 ${pct(queue, 0.9)} ms`);

Save it as measure.mts, set ARCHIL_API_KEY, ARCHIL_REGION and ARCHIL_DISK_ID, and run it with a command, for example npx tsx measure.mts "ls -la". Each call counts as one serverless tool call, so 50 calls cost $0.0025 at $0.05 per 1,000.

When per-call startup is the wrong trade

Fast startup makes serverless tool calls practical for most of an agent's tools, but not for all of them:

• Long-running processes. A dev server or a file watcher has to keep running between calls; each call's microVM exits when its command does.
• State in memory. A Python kernel with a loaded dataset has to be rebuilt on every call.
• Long commands. Each call can run for up to 5 minutes, after which it returns 504 with Exec timed out.
• Many small file reads per call. A fresh cache on every call costs more for tools that read thousands of files than for tools that read a few.

For those, run the step in an Archil persistent sandbox with the same disk mounted, and keep serverless calls for the rest.