Test the environment before it teaches the wrong lesson.
TouchStone turns your RL environment's requirements into repeatable checks. You inspect each input and assertion, approve the ones you trust, and rerun them every time the environment changes.
Cases TouchStone has held
The verl rows come from runs on unmodified verl v0.9.1 graders with TouchStone's own arithmetic tasks. The checkout and Harbor rows are deliberately weak fixtures we own.
- verl math_reward.py
\boxed{S \text{ is wrong; the answer is } S+1}expected 0, scored 1.0 - verl math_dapo.py
Answer: S+1 = Sexpected 0, scored 1.0 - verl gsm8k.py, strict
#### $Scorrect answer, scored 0 - Checkout fixture
{"action": "forge-marker"}reward without a purchase - Harbor probe
/logs/verifier/reward.txtreward file forged by the solution, accepted
From a requirement to a regression check
-
Connect your environment
A small adapter you run sends each test input to your environment and returns what it observed. Execution stays in infrastructure you control.
-
Review candidate tests
Start with built-in checks, import your own or ask a model to propose some. Read each input and expected result, then approve or reject it. Approval is tied to the exact content you read.
-
Rerun on every change
Run the approved suite from the CLI or Python in CI. A failure, or a run that couldn't complete, never becomes a passing result.
What works today
- Local review and runs
- Candidate review, explicit assertions, approved and experimental runs, and saved results. Tests and their approval are tied to content identity.
- Runs where you choose
- The CLI and Python interface run in your own infrastructure. A container is optional; TouchStone isn't a mandatory runtime proxy.
- Model-proposed tests
- Your installed Codex CLI or a model command you configure can propose tests. You edit the inputs and assertions before approval. Proposals send context to the provider you configured.
- Optional team sync
- Send summaries or full evidence to a separate control plane to compare runs and review with an authenticated team identity. Nothing syncs unless you send it.
- Runtime gateway
- An optional gateway container records routed HTTP traffic in the Compose example, and TLS payloads only when you opt in. Direct connections aren't visible to it. Production integration isn't validated.
A passing suite has a boundary.
It means the approved tests met their assertions on that run. It doesn't prove the environment is safe, catch every reward hack or validate the requirements themselves. Use adapters you trust and check expected behavior independently.