SAFETY-04: record the coverage audit as a comment, with its invariant
The audit found a dead mechanism twice, both times a clamp standing in for the mechanism under test, so the technique has to survive this ticket. Not as a script: the only cheap way to automate it is to key on source lines, that goes stale silently, and a green audit that has quietly stopped testing anything is this ticket's own failure mode one level up. Automating it properly would mean decomposing compute() to make its statements separately addressable, which is a refactor of the most safety-critical function in the repo for the benefit of test tooling. So it goes in as a comment block next to the checks it describes, carrying the commit it was measured at, the thirteen figures, and the invariant with the teeth in it: every mechanism must be noticed by at least two checks when it is deleted, and adding a mechanism means re-running the audit. A comment cannot go stale-green, because it never claims to be running. Also recorded: fixtures must sit clear of every rail they are not testing, which is the rule both misses violated; and the `python -B` / clear-pycache discipline, with the reason (CPython invalidates on source mtime-in-seconds plus size, so a same-second same-size rewrite reuses stale bytecode) and the reason it casts no doubt on the figures (the error is one-directional, so every number is a lower bound). Figures are the lead's independent reproduction. I re-measured the one that differed: the detector is 11 for `saturated_now = False` and 10 for the weaker `frozen = False` form, so the table names the form. test_control.py stays at 55 checks, all passing. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01Du77usMj8XNKNFZGmUiWDa
This commit is contained in:
co-authored by
Claude Opus 5
parent
389d9ecd6d
commit
680461c9bf
@@ -24,6 +24,63 @@ def check(name, cond):
|
||||
fails.append(name)
|
||||
|
||||
|
||||
# ---------------------------------------------------------------------------
|
||||
# COVERAGE AUDIT - measured, not executed. Read this before adding a mechanism.
|
||||
#
|
||||
# THE INVARIANT: every mechanism in compute() must be noticed by AT LEAST TWO
|
||||
# checks when it is deleted. If you add a mechanism to compute(), re-run the
|
||||
# audit and add it to the table. If a figure here drops, a check has started
|
||||
# passing for a reason other than the one it names.
|
||||
#
|
||||
# THE TECHNIQUE, because there is no script to run: replace one mechanism in
|
||||
# control.py with a no-op, run this file, count the failures, restore. That is
|
||||
# the converse of the usual mutation - not "does a wrong value fail?" but "does
|
||||
# anyone notice when the mechanism is GONE?". It is kept as a comment rather
|
||||
# than as tooling on purpose: the only cheap way to automate it is to key on
|
||||
# source lines, which goes stale silently, and a green audit that has quietly
|
||||
# stopped testing anything is precisely the failure this ticket exists to fix.
|
||||
# A comment cannot go stale-green, because it never claims to be running.
|
||||
#
|
||||
# Measured at 389d9ec. Numbers are the lead's independent reproduction.
|
||||
#
|
||||
# mechanism in compute() checks that fail when deleted
|
||||
# ------------------------------------------ -----------------------------
|
||||
# integrator freeze (AC 3) 2
|
||||
# integrator clamp (AC 1) 6
|
||||
# integrator bound follows max_w 4
|
||||
# output clamp 3
|
||||
# slew limit 4
|
||||
# output freeze 2
|
||||
# deadband 5
|
||||
# quantisation 2
|
||||
# saturation detector, `saturated_now = False` 11
|
||||
# saturation duration (AC 2), fires instantly 2
|
||||
# sat counter reset on a good cycle 6
|
||||
# target_grid_w bias 3
|
||||
# i_w=None seeding from prev_w 6
|
||||
#
|
||||
# The detector figure is for the `saturated_now = False` form specifically;
|
||||
# disabling it further down as `frozen = False` is a weaker mutation and gives
|
||||
# 10. Reproduce the same form or the number will not match.
|
||||
#
|
||||
# ⚠️ IT HAS FOUND A DEAD MECHANISM TWICE, BOTH THE SAME WAY: a clamp standing in
|
||||
# for the mechanism under test. Deleting the integrator freeze once failed
|
||||
# NOTHING, because the fixtures sat at max_w 2000 and the integrator bound
|
||||
# truncated a wound value back to exactly 2000 - the assertion passed on the
|
||||
# clamp. The output clamp was masked the same way by the integrator bound.
|
||||
# Hence: A FIXTURE MUST SIT CLEAR OF EVERY RAIL IT IS NOT TESTING. Where a test
|
||||
# names one mechanism, make that mechanism the binding one (see TCLAMP and TF).
|
||||
#
|
||||
# ⚠️ RUN MUTATIONS WITH `python -B` AND CLEAR app/__pycache__. CPython
|
||||
# invalidates a .pyc on (source mtime in whole seconds, source size), so a
|
||||
# same-second rewrite that also preserves the file size reuses stale bytecode
|
||||
# and the suite reports on code you are no longer running. It under-reported one
|
||||
# mutation here as 2 where the true figure is 6. The error is one-directional -
|
||||
# stale bytecode can only under-report - so every figure above is a lower bound
|
||||
# at worst, and the two zeros ever recorded were both confirmed by fixing them
|
||||
# and watching the count rise, which a caching artefact cannot do.
|
||||
# ---------------------------------------------------------------------------
|
||||
|
||||
print("control law")
|
||||
|
||||
# Deadband: inside meter noise, hold exactly - do not drift.
|
||||
|
||||
Reference in New Issue
Block a user