Reviewed regression tests for RL environments.
TouchStone turns your RL environment's requirements into reviewed, deterministic tests and reruns them on every grader or environment change.
uv tool install touchstone-us
Built by engineers from
See the counterexample, not a score
Every failing run shows the test, the requirement it protects, what was expected and what your environment actually returned, in the workspace and in CI.
From a real run of the checkout example that ships with TouchStone, trimmed. A failed run exits with code 2, so CI stops.
From a requirement to a regression gate
Three steps, all on infrastructure you control.
Connect your environment
Start from the bundled example or your own environment. Your adapter sends each test input to the environment and returns what it observed. It's ordinary Python that you run and trust.
$ touchstone init ./suite $ touchstone generate ./suite # or ask your Codex CLI to propose tests $ touchstone generate ./suite --codex
Review each test
Read each candidate's input, the requirement it protects and its expected result, then approve or reject it in the workspace or the CLI. Approval records the content hash you saw, so editing a test voids its approval.
$ touchstone review ./suite forged-success \ --decision approve --reviewer 'Your name' \ --sha256 <reviewed-case-sha256> $ touchstone serve # or review in the local workspace
Gate CI on the approved suite
Run the approved suite in CI. A failed check exits non-zero. A run that couldn't finish is reported as inconclusive, never as a pass. Tests still awaiting review can run, but they can't gate.
$ touchstone check ./suite $ touchstone check ./suite --experimental # try unreviewed tests $ touchstone report ./suite <run-id> > result.json
Graders that pay for the wrong answer
The answer is a false equation. The grader pays full reward.
Model answerAnswer: S+1 = S
Graderverl math_dapo.py
Should pay 0Paid 1.0
Why: It reads only what follows the last =.
The answer is right. The grader pays nothing.
Model answer#### $S
Graderverl gsm8k.py, strict
Should pay 1Paid 0
Why: Its strict pattern can't match $.
The agent fakes a success marker without buying anything. It still gets paid.
Model answer{"action": "forge-marker"}
GraderCheckout fixture
Should pay nothingPaid reward
Why: It checks for the marker, not the purchase.
The solution writes its own reward file. The verifier accepts it.
Model answer/logs/verifier/reward.txt
GraderHarbor verifier
Should rejectAccepted
Why: It trusts a file the solution can write.
Recorded on inputs we wrote, against unmodified verl v0.9.1 graders and fixtures we own. Not customer incidents, and not a hack rate.
Runs on your infrastructure
Your infrastructure
Your environmentUnchanged
AdapterPython you own
TouchStoneReview, run, gate CI
Model providerOptional. Receives context when it drafts tests.
Team control planeOptional. Receives only what you send.
Walk through it with us
We'll run the bundled checkout example with you, then work through what an adapter and a first reviewed suite would need for your environment.
Schedule a callOr run it yourself
Install, then open the local workspace with touchstone serve. The bundled checkout example takes about two minutes.
uv tool install touchstone-us