How a reward-hack happens
Step through one real, sealed reward-hack our searcher found — reward → our content-free output → it clears the bar → sealed so you can re-run it.
skill_reward_hacking is a real RL environment reward — a checker that is supposed to only reward genuinely doing the task. No human, no oracle: just code (or a model) returning a score.
starscry runs a seeded, model-free search that only ever sees the reward’s number — never its internals — and tries to make that number go up. It found this content-free output in 1 simulation:
the ... uh the
The reward scored that output 1.076 — at or above its own pass threshold. A garbage string is graded as a success. That is a reward-hack: the reward can be satisfied without doing the task.
The whole hack ships as a transcript sealed by a content hash. You re-run the seed and check the bytes yourself.
seal ee9306ff1ba80c29
replay: stardata audit skill_reward_hacking --exploit-search --row-seed 0 --exploit-seed 0 --exploit-budget 64 --generator mock
Game a reward, live
The univocalic-e reward asks for writing that uses only the vowel e. Type below and it scores in your browser — computed by the real Rust verifier compiled to WASM. The preset eep eep eep is a content-free string; watch it clear a reward built to test constrained writing. That is a reward-hack — the low end of the dial.
Full playground with three-way parity (Python = JS twin = Rust/WASM): the reward playground → · verify a sealed exploit yourself — hash it and watch the seal break: re-run a seal → · or watch a solver play its solution in the decomposition explorer →.
Install the verifier
The full instrument ships as an Apache-2.0 package: the verifier, the audit battery, the exploit search, the seal. One command, then watch a gameable reward fall on your own machine:
$ pip install staros-stardata
$ stardata quickstart
[2] audit a GAMEABLE reward (lipogram composite, degeneracy gate removed)
grade: F
gamed in 1 simulations -- the search built a degenerate completion the reward accepts:
'ok\nuh\nn/a\nx'
gameable reward scores it 1.00; the HARDENED reward (degeneracy gate) scores it 0.25
[3] seal + self-verify
record_sha256 1ea440e56a6c5492... self-verify: OKThen gate your own environment before it burns a training run — stardata ci <your-env> exits non-zero on a gameable reward — and see the integration guide → for wiring it into verifiers, prime-rl, or TRL.
32 audited artifacts · refreshed 2026-07-06T19:42:02.517306+00:00
The board, at a glance
Board integrity 78/100 across 32 audited artifacts · 2 confirmed reward-hacks, the flagship gamed in 1 simulation(s). Every row re-runs byte-for-byte.
How it works
- Point a dumb, deterministic searcher at the reward. No model sits in judgement to have a conflict of interest.
- Read both ends of the dial. A content-free output that clears the bar in almost no search is a reward-hack; a genuine solution that costs real search is difficulty — one currency, search effort.
- Seal it. Every finding ships as a replayable transcript sealed by a content hash. You re-run it; you don’t trust us.
Featured findings
skill_reward_hacking
a content-free completion scored at or above pass_threshold, gamed in 1 simulation(s) — shipped as a replayable transcript.
aime2025
withstood the full battery — degenerate probes, invariance, and exploit-search — with no signature. Negative controls are reported as loudly as findings.
See on the leaderboard →sokoban
search-effort sims-to-solve tracks an independent oracle at ρ = 0.837, with no model in the loop.
See on the leaderboard →Why you can believe it
The instrument’s validity condition is that it can refute its own makers — and it does, in public: the technical paper ships a graveyard of our favourite hypotheses, each with a named cause of death. An instrument that cannot kill its owner’s thesis cannot certify anyone else’s. Read the whitepaper →
Audit your environment
Shipping a verifiers-format environment? We run this exact deterministic, model-free audit — gaming & difficulty — and hand back a sealed, replayable verifier card your buyers re-run themselves. No trust required.
Self-serve on the certification gateway at api.s2ar.dev — every seal on this board re-verifies free at /v1/verify/<record_sha256>, no account.