starlens · flagship demo

The simulation that grades the model.

Every compression demo grades itself with human labels. This one doesn't need any: the world below is a deterministic element simulation — seed 7, 300 fixed-timestep steps, byte-identical frames, and an exact matter census enforced by conservation laws:

water = 925 at every step measured   (exact conservation — the answer key)
stone = 15940 at every step
activity: 16865 → 6073 → 2545 → 1537   (the splash settling)

So "is there water in the basin?" has a machine-checkable answer, and anyone can regenerate the answer key from the seed. We asked two production vision models 10 such questions at five compression rungs, and recorded what two witnesses claimed at each rung: the shipped starlens witness, and naive RMSE — the industry default.

The ladder

rungstarlens witnessRMSE witnessclaude-sonnet-4.6gpt-4o-mini
x1.0 · reference9/9 (100%)7/7 (100%)
certified band ≈ x0.875isomorphic9/9 (100%)5/7 (71%)
x0.5refusedisomorphic (!)9/9 (100%)5/7 (71%)
x0.2refusedisomorphic (!)7/9 (78%)6/7 (86%)
x0.05refusedisomorphic (!)4/9 (44%)3/7 (43%)

Three readings. The certificate held — at the rung the witness certified (targeted at claude-sonnet-4-6), the target model kept 100% of its answers, and every deeper rung was refused, correctly. RMSE over-certifies all the way down — it calls a 32-pixel-wide crush "isomorphic" while both models sit under 45%. And certificates name their target for a reason: gpt-4o-mini, not the target, already drops inside claude's certified band — cross-model transfer is asymmetric.

What the model saw

full resolution
full resolution
certified x0.875
certified x0.875
x0.05 — RMSE calls this
x0.05 — RMSE calls this "isomorphic"

Why this matters for your agent

These frames are the same observations a screenshot-per-turn agent — or a reinforcement-learning policy — consumes. starlens prices every one of them with a sealed, re-verifiable certificate, and charges a share of the savings it proves. A certified refusal costs $0.

← starlens · the preprint · frames, questions, and the eval harness ship in the artifact (examples/sim_census) — regenerate everything from the seed.

s2ar.dev · every number replays from the sealed answer cache · built claude-sonnet-4-6-targeted, graded by conservation law