How to test AI agent sandbox persistence
· by Plori Engineering
An AI agent sandbox persistence test must cross the lifecycle boundary that the product promises. For durable files, that means reading the same bytes from an independent path after shutdown. A replacement test reads them again from a new sandbox attached to the same storage.
We learned this distinction during a controlled plori recovery test on September 7, 2026. The agent wrote a file and read it back successfully. The final saved-state update failed during normal shutdown. A separate reader selected an older valid snapshot and returned 404 for the new file. That result did not prove permanent data loss. It showed that the final state had not reached that reader.
What does “persistent” mean for an agent sandbox?
Start by naming the scope. Several different behaviors are often called persistence:
| Scope | Test boundary |
|---|---|
| Same process | Write and read before the process exits |
| Same sandbox | Stop and restart the agent process without replacing its environment |
| Same thread | Suspend and resume one conversation or execution context |
| Different thread | Start a new thread that should share stored state |
| New sandbox | Destroy the original environment, attach the same storage elsewhere, and read the file |
| External delivery | Retrieve the final file through the user-facing download or files path |
These scopes are not interchangeable. LangChain’s Deep Agents backend documentation, for example, says its default state backend persists within one thread. Its store backend supports state shared across threads. Test the scope that the product promises.
Also separate checkpoints from files. A checkpoint may include virtual files or filesystem state, depending on the backend. Storage-backed bytes have a different lifecycle. Your assertion should say which one you tested.
Why is write, flush, and readback insufficient?
A local read can return bytes from the running filesystem, cache, or current metadata view. Another reader may still select older state after shutdown.
Even a successful fsync has a limited meaning. It asks the current filesystem to make the write durable according to that filesystem’s contract. It does not prove that a remote checkpoint, replica, export, or sync-back step completed. If your architecture publishes a recovery point during shutdown, test that publication separately.
Public issue reports show why the lifecycle boundary matters, without establishing one universal cause. In an August 2025 E2B issue, a user reported that the first pause and resume retained changes, while later cycles did not. In an August 2026 Hermes Agent issue, a contributor traced stale Windows host files to two sync-back integration failures. The project merged a September 2026 correction with regression tests. One report concerned repeated resumes. The other concerned copying remote edits to a host. Similar symptoms did not imply the same mechanism.
How should the minimum persistence test work?
Use a marker whose content and hash are unique to the run. Record each observation with a timestamp and the identity of the environment that produced it.
- Create a new file in the documented persistent location.
- Flush the file through the filesystem API.
- Record the file’s byte count and hash from a read in the running environment.
- Request a normal stop immediately after the write completes.
- Read the file through an independent files or replica path.
- Destroy the original environment.
- Start a new sandbox with the same persistent storage.
- Compare the restored file’s bytes or cryptographic hash with the recorded result.
The independent reader should have no access to the writer’s memory or local caches. Record the new environment identity to prove that the remount replaced the sandbox.
Our corrected controlled test followed this shape. We wrote a fresh marker and requested a normal stop immediately. The final saved-state update completed during shutdown. An independent reader then returned HTTP 200 with the same hash. A new gVisor environment later mounted the storage and observed identical bytes.
That result has strict limits. The test covered one normal stop and one new-environment remount. It did not test abrupt host loss before the environment saved its final state. It also did not establish a general availability or zero-loss guarantee.
What belongs in a broader test matrix?
Repeat the minimum test at every boundary your product promises. Test several sleep and resume cycles, because a first cycle can pass while later cycles fail. Test access across turns and threads when the documented scope includes them.
Add failure timing deliberately. Stop immediately after a write, during sustained writes, and after an idle interval. Test graceful termination separately from forced process exit, sandbox loss, and host loss. Each case exercises different coordination and storage behavior.
Finally, test the path users depend on. If a completed task should deliver a file, retrieve that file through the user-facing path. A healthy checkpoint record does not prove that the file is downloadable. A successful remount does not prove that an external replica is current. Check both paths when the product promises both.