CI-02: grade the merge, and stop a cancelled run reading as a test failure
checks / unit (pull_request) Successful in 3m44s
checks / unit (pull_request) Successful in 3m44s
Two things a green from `checks / unit (pull_request)` was claiming without having observed. Both were measured on a deliberately conflicted throwaway pull request (#29, now closed and its branch deleted) rather than read off the workflow, because the Actions log endpoints 404 on this Gitea and a step's own conclusion is the only channel out of a job. CASE 1, AND THE TICKET'S MECHANISM IS WRONG IN A WAY THAT MAKES IT WORSE. The card says a conflicted pull request loses its merge ref, so that git fetch -q --depth 1 origin "$GITHUB_REF" \ || git fetch -q --depth 1 origin "$GITHUB_SHA" falls through to the second half and quietly grades the bare branch head. The symptom is real - PR #14 reported success while `mergeable: false` - but that is not how. Asking one question per step on the conflicted probe (runs 558, 559): GITHUB_REF == refs/pull/<n>/head P01 passed GITHUB_REF != refs/pull/<n>/merge P02 failed GITHUB_SHA == the pull request head sha P03 passed fetching GITHUB_REF succeeds P04 passed <- even conflicted ...and it lands on the bare head P05 passed refs/pull/<n>/merge does not exist P06 failed `git ls-remote origin 'refs/pull/*'` on this server advertises only `/head`, for open and closed pull requests alike. There are no merge refs here at all. The first fetch has therefore always succeeded and the fallback has never fired once; both halves name the same commit anyway. So the gate did not lose merged content on conflicted pull requests - it has never had merged content, on any pull request in this repository's history. Green has meant "this branch passed on its own", and with `block_on_outdated_branch` off on release/1.0 the base has usually moved underneath it. Reproduced end to end on one pull request, same conflict, gate swapped: run 567 the ORIGINAL gate -> success, "Successful in 3m38s" run 566 the gate below -> failure in 2s, merge step red, both test steps `skipped` The checkout now fetches the base tip and the pull request head, merges them, and refuses if the merge does not apply - as its own NAMED step, because /actions/runs/<id>/jobs carries a per-step conclusion and that is what lets a reader tell "nothing was tested" from "the tests spoke". `--depth 1` is gone: a merge needs a merge base, and a full fetch of this repository measured 6.8 MB and 1.1 s. The fallback survives only on the workflow_dispatch path, which has no pull request to merge and now announces which bare ref it took. CASE 2, AND `cancel-in-progress` IS NOT THE KNOB. Runs 531 and 533 on PR #14 were cancelled at 0s, each at the exact second the next run on that pull request was created: 531 created 17:08:19 `Waiting to run`, cancelled 17:10:44 = 532's creation 533 created 17:45:11 `Waiting to run`, cancelled 18:11:22 = 534's creation Both have `started_at` at the Unix epoch. A cancelled run posts `failure` on the required context, described `Canceled after 0s`; branch protection compares state, not description, so "nothing ran" and "your tests are broken" arrive as the same red wanting opposite responses. Three settings were measured, each with one run holding the single runner slot and two further pushes behind it: no concurrency block at all runs 561 and 562 both queued, both executed group, cancel-in-progress:false run 564 sat `Blocked by required conditions` and was cancelled at 0s the second 565 existed A shared group cancels a run that has not started regardless of that flag - GitHub documents it and Gitea 1.27 implements it. Having no group is the only configuration measured to leave a queued run alive, so the block is gone and the measurement is written above it. The cost is stated in the file: a developer who pushes four times occupies the shared runner four times. That is CI-01's own trade written the other way round - a delayed verdict, never a destroyed one - and a destroyed verdict is worse here because it is indistinguishable from a real failure. A sha-keyed group was considered and rejected: the only cancellations it still permits are same-sha ones, whose red lands on a commit that IS still the head. WHAT IS NOT ESTABLISHED, SAID PLAINLY. Nothing here makes a cancelled run distinguishable at the gate BY STATE; that is Gitea posting the status, not this workflow, and no edit to this file can change it. What ships removes the cause instead of relabelling the symptom. And a commit that never had a run carries an empty `/statuses/<sha>` list, while a queued one carries `pending / Waiting to run` within seconds and a concurrency-held one `pending / Blocked by required conditions` - those were already distinguishable at the API. `/actions/tasks`, which is what was polled when this was first reported, lists only tasks that started and cannot answer the question at all. test_ci_gate.py grades the gate by RUNNING it. It extracts the checkout and merge steps out of checks.yml and executes the shipped commands against two throwaway local repositories it builds - one whose pull request merges, one whose does not - and pins that the old one-liner exits 0 on the unmergeable one while the new steps refuse and say nothing was tested. Part 1 pins the two tokens that assemble `checks / unit (pull_request)`: renaming `name:` or the `unit:` job key would leave every open and future pull request waiting forever on a context that never arrives, with no error anywhere in that sequence, and admin override blocked. Seven mutations of checks.yml were each confirmed red, including both renames and a restored `--depth 1`. .gitea/ci/Dockerfile gains `git` for it; the fixtures are local paths and no network is touched. 17/17 green in the checks image. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -2,6 +2,71 @@
|
||||
|
||||
## Unreleased
|
||||
|
||||
**The gate has never once graded the content a pull request would land, and a
|
||||
run that never started reported as a test failure (CI-02).** Both halves were
|
||||
measured on a deliberately conflicted throwaway pull request rather than read
|
||||
off the workflow, because the Actions log endpoints 404 on this Gitea and a
|
||||
step's own conclusion is the only channel out of a job.
|
||||
|
||||
**Case 1, and the ticket's mechanism was wrong in a way that made it worse.**
|
||||
CI-02 was filed saying a conflicted pull request loses its merge ref, so that
|
||||
`git fetch $GITHUB_REF || git fetch $GITHUB_SHA` falls through and grades the
|
||||
bare branch head. The symptom was real — PR #14 reported `checks / unit ->
|
||||
success` while `mergeable: false` — but the mechanism is not. On this server
|
||||
`GITHUB_REF` **is** `refs/pull/<n>/head`, `GITHUB_SHA` is the same commit, and
|
||||
`git ls-remote origin 'refs/pull/*'` advertises no `/merge` refs at all, for
|
||||
open and closed pull requests alike. The first fetch has always succeeded; **the
|
||||
fallback has never fired once.** So the gate did not lose merged content on
|
||||
conflicted pull requests — it has never had merged content, on any pull request
|
||||
in this repository's history. Green has meant "this branch passed on its own",
|
||||
which on a base that has moved is not what lands, and `block_on_outdated_branch`
|
||||
is off on release/1.0 so the base usually has moved.
|
||||
|
||||
**The checkout now fetches the base tip and the pull request head, merges them,
|
||||
and refuses if the merge does not apply** — as its own named step, so
|
||||
`/actions/runs/<id>/jobs` distinguishes "nothing was tested" from "the tests
|
||||
spoke". Verified by producing it: with the fix on a conflicted pull request the
|
||||
required context went `failure` in 2 s with the two test steps `skipped`. The
|
||||
`--depth 1` is gone (a merge needs a merge base; a full fetch of this repository
|
||||
is 6.8 MB and 1.1 s), and the fallback survives only on the `workflow_dispatch`
|
||||
path, which has no pull request to merge and now says so in its output.
|
||||
|
||||
**Case 2: `concurrency:` is gone, and `cancel-in-progress` was never the knob.**
|
||||
Runs 531 and 533 on PR #14 were cancelled at 0 s each at the exact second the
|
||||
next run was created, having never started; a cancelled run posts `failure` on
|
||||
the required context, described `Canceled after 0s`, and branch protection
|
||||
compares state and not description — so "nothing ran" and "your tests are
|
||||
broken" arrive at the gate as the same red, wanting opposite responses. Three
|
||||
settings were measured with one run holding the single runner slot and two
|
||||
pushes behind it: with **no block at all** both queued runs survived and
|
||||
executed; with the group kept and **`cancel-in-progress: false`** the queued run
|
||||
was still cancelled at 0 s. A shared group cancels a run that has not started
|
||||
regardless of that flag. Having no group is the only configuration measured to
|
||||
leave a queued run alive, and it is what ships. The cost — a developer who
|
||||
pushes four times occupies the shared runner four times — is CI-01's own trade
|
||||
written the other way round: a delayed verdict, never a destroyed one.
|
||||
|
||||
**`test_ci_gate.py` grades the gate by running it, not by reading it.** It
|
||||
extracts the checkout and merge steps out of `checks.yml` and executes the
|
||||
shipped commands against two throwaway local repositories it builds — one whose
|
||||
pull request merges, one whose does not — and pins that the old one-liner exits
|
||||
0 on the unmergeable one while the new steps refuse. It also pins the two tokens
|
||||
that assemble `checks / unit (pull_request)`: renaming `name:` or the `unit:`
|
||||
job key would leave every open and future pull request waiting forever on a
|
||||
context that never arrives, with no error anywhere in that sequence. Seven
|
||||
mutations of `checks.yml` were each confirmed red. `.gitea/ci/Dockerfile` gains
|
||||
`git` for it.
|
||||
|
||||
**What was NOT established, stated so nobody reads more into this.** The
|
||||
original report that "no run was ever created for `0942228d`" was already
|
||||
corrected on the card — run 533 was created and cancelled — and this work
|
||||
confirms that from the runs list and the per-commit status history. A commit
|
||||
that never had a run carries an empty `/statuses/<sha>` list; a queued one
|
||||
carries `pending / Waiting to run` within seconds, and a concurrency-held one
|
||||
`pending / Blocked by required conditions`. Those are distinguishable at the
|
||||
API. `/actions/tasks`, which is what was polled originally, lists only tasks
|
||||
that started, and cannot answer the question at all.
|
||||
|
||||
**A setpoint device smaller than `max_w` silently disarmed the frozen-meter
|
||||
detector, and the status page reported it as armed (SAFETY-20).** SAFETY-18's
|
||||
conjunction needs `abs(commanded_w) >= 0.95 * max_w`, but the emitted command is
|
||||
|
||||
Reference in New Issue
Block a user