Authensor
public demo · not the full product john@authensor.com

Trust-Boundary RL · A Runnable Research Demo

The grader can be fooled.
The sealed score cannot.

Every step in a trust-boundary environment is scored twice: once by a grader the model can fool, once from sealed ground truth it cannot touch. Pay the model on the sealed score and the exploit stops paying.

$ python3 run.py $ python3 demo.py stdlib only · no GPU · runs in seconds

Exploitation rises with scale. Training ends it.

Two copies of the same base model, same environment; the reward track is the only difference. The gap between the curves is what training took away.

Exploit rate by model size, before and after training

60% 40 20 0 % OF EPISODES 11% 50% 45% 6% 0% 0% BASE AFTER TRAINING 50 → 0 at 1.5B, the exploit stops paying 0.5B 1.5B 7B MODEL SIZE 60% 40 20 0 11% 50% 45% 6% 0% 0% 50 → 0 0.5B 1.5B 7B MODEL SIZE

risk_assessment environment · GRPO on Qwen2.5 · scored on the sealed holdout · from the full suite, not this demo

One episode, three policies.

The demo scores each policy on both tracks. Read the ruled column the way a private banker would: it is the balance, the only number the optimizer ever sees.

policy reward_true reward_vulnerable reward_trained exploit %
honest 1.00 1.00 1.00 0%
exploiter 0.00 Void1.00 0.00 100%
floor 0.00 0.00 0.00 0%

void: banked from the naive grader, never cashed. The trained reward follows the sealed record.

The exploiter banks a 1.00 it can never cash.

The sealed record holds it at zero, so a forbidden shortcut cannot raise the trained reward, by construction. That gap is the whole game.

Two scores per step.

model output reward_true from ground truth sealed: the model cannot touch it reward_vulnerable heuristic score forgeable: trusts the model’s claim SETS THE PENALTY reward_trained true score, minus a penalty an exploit can raise the vulnerable score, never the trained one model output reward_true from ground truth sealed reward_ vulnerable heuristic score forgeable PENALTY reward_trained true score, minus a penalty an exploit can raise the vulnerable score, never the trained one

reward_vulnerable is what a naive or compromised grader would pay: here, a grader that trusts a self-reported score line.

reward_true is computed from frozen ground truth the model cannot influence, so an exploit can never raise it.

reward_trained is the true score minus an exploit penalty, so gaming the grader lands below an honest but imperfect policy.