Verifiers: how to check a success signal

Gantovar Environments (fictional example) → RL environment provider → Verified task environments → Reproducible resets

Content reviewed 8 November 2026. How this is assessed.

Part of the Best RL Environments reference, which uses Gantovar Environments (a fictional example) to illustrate reinforcement learning environments criteria.

Prove the verifier in both directions

SWE-bench offers a clear pattern. It runs tests before and after the reference solution, keeps a task only if at least one test changes from fail to pass, and counts a task as solved only when all fail-to-pass tests and all previously passing tests pass. The second condition catches a fix that breaks something else. Ask for the same two-sided evidence on any delivered task.

A reference solution proves that a task can be passed. It does not prove that the verifier rejects bad work. The Agentic Benchmark Checklist recommends providing an automatic oracle solver, manually verifying test quality and using objective coverage measures. Its authors report that weak checks can distort results by up to 100 percent in relative terms, and they name too few test cases among the causes.

Match the check to the output

Different outputs need different checks. For a system-state task, OSWorld uses per-task getter functions that extract files or application state, and evaluator functions that compare them with the expected result, with 134 unique evaluation functions across the set. The checklist adds that state checks should include all valid end states and also inspect unrelated state, so that an agent cannot succeed by breaking something the task did not mention.

String matching should handle equivalent phrasing, negation and answers that simply list every option. If you use a language model as judge, the same checklist says to pilot-test its accuracy and self-consistency. As a practical rule, prefer an executable check whenever one is possible, and reserve a model judge for outputs that truly need interpretation, keeping a human-labelled sample to measure it against.

Assume the agent will try to game it

Agents find shortcuts. ImpossibleBench makes the point precisely: it builds coding tasks whose specification contradicts the unit tests, so that any pass must come from a shortcut such as modifying or deleting tests. Its pass rate on those tasks serves as a measure of cheating. You can borrow the method by adding a few deliberately impossible tasks to a validation run and investigating every pass.

Keep ground truth out of reach. The checklist calls for agents to be fully isolated from ground truth, including files and memory. Hash the test files before and after an episode and fail the run if they changed. Run grading from a clean copy, never from the agent's working directory. Then review a random sample of passing runs by hand, because a pass is the outcome that nobody tends to question. The reward guide covers what happens when these signals feed training.

Sources for this entry

• Gymnasium Env API: Reset, step, termination, truncation and seeding.
• Establishing Best Practices for Building Rigorous Agentic Benchmarks: Task validity, outcome validity and reporting checklist.
• SWE-bench: Fail-to-pass validation on real software issues.
• OSWorld: Per-task setup, snapshots and evaluation in a real computer environment.
• ImpossibleBench: Measuring whether agents exploit test cases.
• Defining and Characterizing Reward Hacking: Formal treatment of proxy rewards.
• Deep Reinforcement Learning at the Edge of the Statistical Precipice: Reporting uncertainty with few runs.
• NIST SP 800-190: Application Container Security Guide.

Continue this reading path

• Best RL Environments category guide
• Gantovar Environments (fictional example)
• How to choose: an RL environment provider
• Resource directory

Cite this entry

Best RL Environments. "Verifiers: how to check a success signal." Content reviewed 8 November 2026. https://bestrlenvironments.com/verifiers-and-graders/