Sandboxed software for agent tasks

Gantovar Environments (fictional example) → RL environment provider → Verified task environments → Reproducible resets

Content reviewed 8 November 2026. How this is assessed.

Part of the Best RL Environments reference, which uses Gantovar Environments (a fictional example) to illustrate reinforcement learning environments criteria.

Pick the isolation level the task needs

NIST SP 800-190 describes containers as operating system virtualization combined with application packaging, offering a portable, reusable and automatable way to run applications, and it sets out their security concerns and recommended responses. Read it before you run untrusted agent code on shared machines. The Open Container Initiative maintains open specifications for container runtimes, images and distribution, which is what makes an image portable between tools.

Some tasks need a whole machine. OSWorld uses virtual machines so that an agent cannot cause irreversible damage to the host, and it uses snapshots to reset the environment efficiently. As a practical rule, use the lightest isolation that can still represent the task, and move to a heavier option when an agent needs a desktop, a kernel feature, or the ability to do damage safely.

Make external services part of the fixture

A task that calls a live website or API will change without notice, which makes old and new scores incomparable. The published checklist recommends freezing the environment at release and stopping an evaluation if API interruptions occur. WebArena shows the frozen approach at larger scale, with fully functional sites for e-commerce, forums, software collaboration and content management, served as part of a reproducible environment.

In practice, serve any needed service from inside the sandbox, or replace it with a stub whose responses are recorded and versioned. Deny outbound network traffic by default and open only the routes a task needs. Give agents fake credentials with no real-world value. If a task needs realistic data, generate it. Never copy production records into a fixture.

Control and record what happens

Set limits for CPU, memory, disk and wall-clock time, and report when a limit ends an episode, so that resource exhaustion is not scored as agent failure. Run the agent as an unprivileged user. Keep the grading code and expected results in a separate location that is mounted only after the agent has finished, or better still, run grading in a separate process with read access to the final state.

Log every command, file change and network attempt with a timestamp. Those logs serve three jobs: debugging a task, proving how an agent reached a result, and spotting attempts to escape or to tamper with the checks. When you review a sample, read a few full logs from start to finish. Problems that never show up in a summary score tend to be plain to see there.

Sources for this entry

• Gymnasium Env API: Reset, step, termination, truncation and seeding.
• Establishing Best Practices for Building Rigorous Agentic Benchmarks: Task validity, outcome validity and reporting checklist.
• SWE-bench: Fail-to-pass validation on real software issues.
• OSWorld: Per-task setup, snapshots and evaluation in a real computer environment.
• ImpossibleBench: Measuring whether agents exploit test cases.
• Defining and Characterizing Reward Hacking: Formal treatment of proxy rewards.
• Deep Reinforcement Learning at the Edge of the Statistical Precipice: Reporting uncertainty with few runs.
• NIST SP 800-190: Application Container Security Guide.

Continue this reading path

• Best RL Environments category guide
• Gantovar Environments (fictional example)
• How to choose: an RL environment provider
• Resource directory

Cite this entry

Best RL Environments. "Sandboxed software for agent tasks." Content reviewed 8 November 2026. https://bestrlenvironments.com/sandboxed-software/