Safety Boundary And Governance
Frozen judges, sandbox limits, kill switch, rollback, and records that keep autonomy measured.
A boundary that code enforces
Safety is a property of the boundary, not a hope about the worker. The worker may be creative. The boundary stays strict. Sandboxing limits writes to the working branch. Network access stays off by default. Secrets never enter prompts or traces.
The judge code, the gate code, and the loop entry point are marked immutable. Any candidate that touches those paths is rejected by rule. High risk patches wait for human approval. A kill switch halts the scheduler at once and leaves the last kept mainline intact.
Records that make trust possible
Every promotion links to tests, scores, costs, and reviewer notes. Every rejection links to reasons and residue. Benchmarks run on held out cases that the generator never sees during search. Judge calibration runs on a schedule with known good and known bad sets.
Cost and latency trends sit beside quality trends so trade choices stay visible. Lineage views show which old win enabled a later breakthrough. A new reader can follow any cycle from task to verdict without hunting through logs.
Named failure modes
Self confirming loops approve weak work because the judge shares worker bias. Model collapse narrows outputs after too much self training without fresh grounding. Diversity collapse fills the archive with near copies of one win.
Metric gaming raises a proxy while true quality stalls. Direction drift changes goals without human consent. The atlas answers each with a named guard, from separate judges to frozen paths to human checkpoints and rollback.
Failure mode and guard
| Failure mode | Early sign | Guard |
|---|---|---|
| Self confirming loop | Judge approves almost everything | Separate judge plus calibration sets |
| Model collapse | Outputs narrow across cycles | Fresh grounding plus diversity checks |
| Diversity collapse | Archive fills with near copies | Archive novelty scoring |
| Metric gaming | Proxy rises while transfer stalls | Held out transfer cases |
| Direction drift | Goals shift without review | Human checkpoints plus schedule logs |