CI hardening: job timeouts and the docs gate - Test Plan
| Field | Value |
|---|---|
| Issues | #910 job timeouts, #907 docs build unreachable locally |
| Branch / PR | folded into #917 |
| Date | 2026-08-18 |
Summary
| # | Scenario | Result |
|---|---|---|
| 1 | Every ci.yml job has a timeout | [x] 15/15 |
| 2 | Timeout values come from measured norms | [x] measured |
| 3 | Docs gate is reachable locally | [x] --scope docs |
| 4 | Docs gate can actually FAIL | [x] proven two-sided |
| 5 | Opt-in locally | [x] verified; the CI half was already covered by the build-docs job |
| 6 | Actionlint validates the ci.yml edits | [x] clean, run by the PM session |
1-2. Job timeouts (#910)
All fifteen jobs lacked timeout-minutes, not just the Test job that hung. Every one inherited GitHub's six-hour default.
Values are derived from measured durations. The first measurement attempt was wrong and would have produced confident nonsense: sampling "recent ci.yml runs" returned a single job, which looked like a query bug and was not. Nearly every recent run is a draft PR, and ci.yml guards its jobs on draft == false, so they legitimately skip. That sample was measuring the skip path. Re-sampled from runs whose PR had been marked ready:
| Job | median (min) | max | timeout |
|---|---|---|---|
| Test | 4.0 | 4.3 | 20 |
| Firestore Rules Tests | 1.4 | 2.5 | 15 |
| Build (Docs) | 1.9 | 2.2 | 15 |
| Lint & Security | 1.6 | 1.7 | 15 |
| Test Services, Builds | 0.6-0.8 | 0.8 | 15 |
| everything else | 0.1 | 0.6 | 10 |
n is only 2 for most jobs, which is thin, so the values are deliberately generous. A timeout is a hang detector, not a performance budget: it must never fire on a legitimately slow run and must turn a six-hour exposure into minutes. Test's 20 catches the observed 33 and 46 minute hangs with roughly 5x headroom.
4. The docs gate can FAIL (the point of #907)
A wired scope that never reds is decoration, so this was proven two-sided against the real failure class: a bare angle-bracket token in markdown prose, which vitepress hands to the Vue template compiler as an unclosed element.
| Arm | Result |
|---|---|
| Control, ordinary prose | PASS |
| Treatment, bare angle-bracket token in prose | FAIL, Element is missing end tag, naming the probe file |
Two false negatives on the way there, both mine, both worth recording because each looked like the gate failing to work:
- The probe was named
.docs-gate-probe.md. Vitepress ignores dotfiles, so it never entered the build and the gate correctly stayed green. - The probe wrapped the token in backticks. A code span is exactly the FIX for this failure class, so the probe was planting the cured form. The gate was right both times; the test was wrong both times.
5. Opt-in locally. CORRECTION: the "always-on in CI" half was already true, and not because of this change
Both local behaviours verified by running the real thing rather than reading the filter:
| Condition | Docs Build present? | Checks |
|---|---|---|
Default local run (CI unset) | no | 34 |
CI=true | yes, passed in 83.9s | 35 |
But CI never invokes validate.js. It runs the individual scripts per job, and it already had a dedicated build-docs job running npm run docs:build (measured at 1.9-2.2 min). That job is what caught the 2026-08-17 failure in the first place.
So the honest description of this change:
- The gap #907 named was purely LOCAL.
validatedid not compile the docs, so the failure class was unreachable before pushing. That gap is now closed by--scope docs. - CI was never missing this check. The
optIn/CIbranch infilterChecksis correct and harmless, but in practice it does nothing, because nothing in CI callsvalidate.js. Claiming this change makes the docs build "always on in CI" overstates it.
Kept rather than removed, because a future CI job that does call validate should get the docs build automatically rather than silently skipping it. But it is a provision for later, not a thing this change turns on.
6. Actionlint: clean (verified by the PM session, not by this one)
lint.actionlint.mjs needs a Go toolchain to fetch its pinned binary, and this VM has none. Locally it warn-skips; under CI=true it fails with actionlint binary unavailable in CI, which is an environment limitation rather than a finding about the edits.
The PM session closed this by pulling the prebuilt v1.7.12 binary, the exact version the repo pins, and running it against both workflow files extracted from the ref rather than a working tree: ci.yml with all fifteen timeouts and assign-milestone.yml with the classifier. Clean, exit 0, zero findings.
Attributed rather than claimed: this session still cannot run actionlint, so the result is recorded as another session's verification with the tool and version named, which is what makes it checkable.
Checked locally in addition: the YAML parses, all 15 jobs carry a timeout, and the insertions sit at job level rather than inside a step (read in the diff, not assumed).
Method worth reusing
Measuring a CI job's duration requires runs where the job actually executed. Draft-PR runs skip nearly everything by design, so a sample of "recent runs" measures the guard rather than the work. Filter to runs from a PR that was marked ready, and report n so a thin sample is visible as thin.
And when a gate refuses to go red, suspect the probe before the gate. Both false negatives here were the test planting something the gate was right to ignore: a dotfile the builder skips, and the backticked form that is the documented fix.