How long until a plori agent replies? Wake latency on v349
On production release v349, each agent run gets its own Firecracker VM. The run takes the one pre-booted VM in the warm pool (a pool hit), or it boots a new VM (on demand). We measured the time from sending a message to the first reply text on both paths, against production, on 2026-10-14. With a pool hit the median was 2,826 ms (n = 252). With a new VM the median was 5,532 ms (n = 34). Each of these runs started at the same moment as another run.
Production release v349 · serial probes 2026-10-14 05:41:39–12:13:35 UTC, 253 runs kept · concurrent pairs 12:53:43–14:11:50 UTC, 78 runs kept · Revised 2026-10-14
Two paths to a VM.
§01When a run starts, the control plane claims a VM for it. If the warm pool has a ready VM, the run takes it: a pool hit. If not, the run boots a VM from the root file system image: on demand. During this measurement the warm pool size was one VM, on one VM host.
| Run type (serial probes) | runs | pool hit | on demand |
|---|---|---|---|
| First visit (a new session) | 127 | 127 | 0 |
| Follow-up, 5 s after the previous run | 84 | 83 | 1 |
| Follow-up, 60 s after the previous run | 42 | 42 | 0 |
| All | 253 | 252 | 1 |
Serial probes took the pool VM in 99.6% of runs (252 of 253 kept). This rate comes from one-at-a-time probing with a pool of one VM. It is not a rate for user traffic. Between two probe runs, the pool almost always had a new ready VM. A cold boot happened once under serial probing (first reply text 3,455 ms, VM boot 1,602 ms). That is one observation, not a distribution.
For that reason the cold-boot numbers on this page come from a second experiment. It starts two first visits at the same moment, so that one of them finds the pool empty. In 34 of the 39 kept pairs, one run took the pool VM and the other booted a new VM. In the other 5 pairs both runs were pool hits (see the section on the replacement VM).
Pool hit, serial probes.
§02| Step | p50 | p90 | p99 | max | n |
|---|---|---|---|---|---|
| Run accepted → VM dispatchedserver timestamps: claim the VM, prepare the agent's Workspace files, mount them in the VM | 504 ms | 748 ms | 1,208 ms | 2,094 ms | 252 |
| Send → first executor eventthe first event from the agent process inside the VM | 948 ms | 1,275 ms | 1,914 ms | 3,427 ms | 252 |
| Send → first reply textcontains one full model round, explained below | 2,826 ms | 4,328 ms | 7,530 ms | 9,644 ms | 252 |
| Send → run finishedthe run reports that it is complete | 2,838 ms | 4,336 ms | 7,537 ms | 9,653 ms | 252 |
| First visit, end to endnew session + thread + WebSocket connect + attach + send → first reply text, summed per run | 3,444 ms | 4,531 ms | 5,659 ms | 5,921 ms | 127 |
The 252 pool hits are 127 first visits and 125 follow-up runs on the same agent (83 after 5 s, 42 after 60 s). The end-to-end row covers the first visits only: it adds the time to create the session and the thread and to open and attach the connection.
Cold boot (on demand), concurrent pairs.
§03These runs come from 40 rounds of paired first visits. Two new anonymous sessions sent their first message at the same moment: the two sends were 6 ms apart at the median and 184 ms at most. One run took the pool VM and the other booted a new VM. Each of these runs started at the same moment as another run on the same VM host, and the numbers include that.
| Step | p50 | p90 | max | n |
|---|---|---|---|---|
| Run accepted → VM dispatchedserver timestamps: claim the VM, prepare the agent's Workspace files, mount them in the VM | 2,983 ms | 3,417 ms | 4,349 ms | 34 |
| Send → first executor eventthe first event from the agent process inside the VM | 3,712 ms | 4,296 ms | 5,191 ms | 34 |
| Send → first reply textcontains one full model round, explained below | 5,532 ms | 6,611 ms | 7,482 ms | 34 |
| Send → run finishedthe run reports that it is complete | 5,538 ms | 6,626 ms | 7,508 ms | 34 |
| VM boot (vm_ready)the boot phase of the pool claim, from the control-plane log | 2,002 ms | 2,801 ms | 3,001 ms | 34 |
| First visit, end to endnew session + thread + WebSocket connect + attach + send → first reply text, summed per run | 6,178 ms | 7,266 ms | 7,986 ms | 34 |
No p99 is given: n = 34, and p99 needs at least 100 samples.
Starting two runs at once also slowed the run in each pair that took the ready pool VM. The table puts its medians next to the serial first visits that took the pool VM:
| Step, p50 | Serial first visit, pool hit (n = 127) | Pair, run that took the ready pool VM (n = 39) |
|---|---|---|
| Run accepted → VM dispatched | 473 ms | 687 ms |
| Send → first executor event | 895 ms | 1,331 ms |
| Send → first reply text | 2,761 ms | 3,011 ms |
VM boot in the pairs had a median of 2,002 ms. The one serial cold boot took 1,602 ms, and the fleet-wide on-demand claims since the rollout (see the fleet-wide context section) had a median of 1,801 ms. The on-demand numbers above are for a cold boot that starts at the same moment as another run.
Pool hit that waited for the replacement VM.
§04In 5 pairs both runs were recorded as pool hits. The first run took the ready VM, and that claim starts a replacement VM at once. The second run did not boot its own VM: it waited for the replacement and took it when it was ready. The control-plane log records these claims as hits with no boot time, but the claims took 2,369 to 2,956 ms. In the data, the run that waited is the run of a hit-and-hit pair with the longer pool claim. With n = 5, only the median and the maximum are given.
| Step | p50 | max | n |
|---|---|---|---|
| Run accepted → VM dispatched | 2,586 ms | 3,177 ms | 5 |
| Send → first executor event | 3,032 ms | 3,914 ms | 5 |
| Send → first reply text | 5,765 ms | 6,553 ms | 5 |
| Send → run finished | 5,782 ms | 6,580 ms | 5 |
Context: every cold boot on the fleet, claim only.
§05The control-plane log records every pool claim on production: real users, our own test traffic and these benchmark runs together. From 05:06 UTC, after the v349 rollout, to the last log line at 12:42 UTC on 2026-10-14, 17 claims booted a new VM. These numbers stop at the pool claim. They contain no model round and no reply text, so they are not a user wait time.
| Fleet-wide on-demand claims | p50 | p90 | max | n |
|---|---|---|---|---|
| Pool claim, total | 2,171 ms | 2,518 ms | 2,922 ms | 17 |
| VM boot (vm_ready) | 1,801 ms | 2,002 ms | 2,602 ms | 17 |
What “first reply text” means.
§06It is not a streamed first token. The agent holds its reply text for a whole model round, runs a check on it, and then sends it. So every first-reply time on this page contains one full model round. The model for each run comes from a configured mix. The time to the first executor event is similar for every model. The time to the first reply text is not:
| Model (pool hit, serial) | n | first executor event p50 | first reply text p50 | first reply text p90 | first reply text max |
|---|---|---|---|---|---|
| anthropic/claude-haiku-5.5 | 87 | 914 ms | 2,301 ms | 2,869 ms | 9,106 ms |
| anthropic/claude-sonnet-5.5 | 16 | 978 ms | 2,370 ms | 3,024 ms | 4,511 ms |
| openai/gpt-6-luna | 106 | 890 ms | 3,456 ms | 4,914 ms | 9,644 ms |
| openai/gpt-6.1-sol | 43 | 1,046 ms | 3,058 ms | 3,545 ms | 5,008 ms |
| Model (on demand, pairs) | n | first reply text p50 | first reply text max |
|---|---|---|---|
| anthropic/claude-haiku-5.5 | 12 | 4,849 ms | 6,632 ms |
| anthropic/claude-sonnet-5.5 | 6 | 6,028 ms | 7,482 ms |
| openai/gpt-6-luna | 13 | 5,532 ms | 7,178 ms |
| openai/gpt-6.1-sol | 3 | – | 6,634 ms |
A dash means the group has too few runs for that percentile: p50 needs at least 5 runs and p90 at least 10.
Exclusions.
§07- Serial probes: 130 rounds, 260 runs attempted, 258 completed, 253 kept.
- Refused follow-ups: 2 of 87 follow-ups sent 5 s after a first visit were refused. The save of the first visit's files had failed and kept retrying. The follow-up waited about 25 s, and the service refused it because the Workspace was still saving. This is a save bug that we are investigating. The two first visits whose save failed stay in the data: their timings end before the save starts.
- Other test traffic: we exclude a run when another test-class account claimed a pool VM in a set window. The window starts 12 s before the run was accepted and ends when its VM was dispatched. With a pool of one VM, that claim decides the outcome of the run. An account that was already deleted when we read it back counts as test traffic. 5 runs were excluded, 2 of them only because of such deleted accounts. The windows held 5 test-class claims, 3 claims by deleted accounts and 0 real-user claims. Without this rule the serial data has 256 pool hits and 2 cold boots.
- Pairs: 40 rounds, 80 runs, 78 kept. We excluded one pair (2 runs). Claims by other test-class accounts came just before it, and both of its runs booted a new VM. The claim by the other run of the same pair is part of the measurement and never excludes a run. With the excluded pair added back, the on-demand group has n = 36 and a first reply text p50 of 5,560 ms and p90 of 6,633 ms.
Methodology.
§08This is a controlled probe against production, not user traffic. The probe host is one host on the public internet that reaches plori through Cloudflare. We measured the network path once after the run. A TCP connect to the Cloudflare edge took 3–4 ms. The time to the first byte of GET /healthz was about 45 ms.
- Accounts: each first visit uses a new anonymous trial session, marked as test traffic.
- Message: every run sends Reply with exactly OK and nothing else. All 258 completed serial runs and all 80 pair runs replied OK.
- Serial probes: 130 rounds, nominally 180 s apart, one run at a time. Each round sends a first visit and then one follow-up run on the same agent and thread. The follow-up starts 5 s, 5 s or 60 s after the previous run ended, in that repeating order. On the server, the median gaps were 5,322 ms and 60,290 ms. Eight rounds were 151–273 s apart instead of 180 s. This changed the spacing, not any per-run timing.
- Pairs: 40 rounds, 120 s apart. Each round creates two sessions and starts both first visits at one barrier.
- Client timings (first executor event, first reply text, run finished) use the probe's monotonic clock, from its send of the run.create message over WebSocket. The end-to-end first-visit time is the per-run sum of the client timings for session, thread, connect, attach and first reply text.
- Server timing (run accepted → VM dispatched) uses timestamps from the control-plane database.
- Pool outcome, claim time and VM boot time come from the control-plane log line for the pool claim of each run. Each run is classified by its own log line, not by when it was sent.
- Percentiles use linear interpolation between closest ranks. p50 needs n ≥ 5, p90 n ≥ 10 and p99 n ≥ 100. Otherwise the cell shows a dash.
Limitations.
§09- One region (sea) and one vantage. Other routes have different network latency.
- One one-line prompt with a one-word reply. Other prompts and runs that call tools were not measured.
- A warm pool of one VM on one VM host, on the production fleet shared with real users. This is not a load test, and it does not show many users that arrive at the same time.
- No p99 for the cold-boot path (n = 34). Only the median and maximum for the runs that waited for the replacement VM (n = 5).
- Anonymous first visits and follow-ups after 5 s or 60 s only. Registered accounts and returns after a longer pause were not measured.
- The share of prompt tokens served from the model provider's cache was not measured.
- All numbers are from 2026-10-14 only.
- No cross-vendor comparison: this page measures plori against itself, under a stated method, and claims nothing about other platforms.
The previous version of this page.
§10Until 2026-10-14, this page reported a July 2026 measurement of the retired design. In that design, agents ran in Kubernetes warm pods, and the probe timed the first streamed token. That is a different event on a different system, so its numbers are not comparable with this page. Its data stays available: 2026-07-18/data.csv · 2026-07-18/data.json.
Reproducibility and data.
§11The scripts for this measurement are in scripts/bench/ of plori's private source repository:
- coldstart-collect.sh: the schedule.
- coldstart-sample.py: one probe run, with the uat-wsprobe WebSocket client.
- coldstart-harvest.py: server timings and log lines.
- coldstart-fleet-claims.py: the fleet-wide claims.
- coldstart-analyze.py: exclusions and percentiles.
The public repository github.com/plori-ai/coldstart-bench holds the July 2026 harness, not these scripts.
Raw data, one row per run, with the pool outcome, every timing, the resolved model and the exclusion reason: serial probes data.csv · data.json; pairs pairs.csv · pairs.json. The same files stay at /benchmarks/agent-wake-latency/2026-10-14/. They are the analyzer's per-run files without the run id and VM host columns. Except for the fleet-wide context section, every percentile on this page can be computed again from them. Keep the rows where excluded is false, then group by claim_outcome, arm or model as each table does. The script that writes these files and the page data from the analyzer output is scripts/bench/coldstart-publish-v349.py in our private repository. The per-claim fleet log data in the fleet-wide context section is not published. The data is published under CC BY 4.0. Reuse the data with attribution to plori.
plori engineering. The page changes only when new data is collected.
Related: idle compute billing , Workspace save time and what a task costs.