fix(health): rules that fired in the wrong language, and files never scanned at all #5614
AI Triage
(no summary)
Priority: medium Risk area: mixed
Suggested labels: (no label suggestions) Suggested reviewers: (no reviewer suggestions)
Suggestions only — nothing has been applied. The PR author stays in control.
AI review unavailable
The platform's AI balance is exhausted, so AI generation is temporarily unavailable. Nothing was lost. You can queue this as a repair for the internal agent from the repository's Health page, or try again once the balance is restored. The PR is otherwise unchanged.
Merge blocked — not by this PR. CI is green on 89519e3, mergeable: true, and both the MCP merge tool and POST /api/v2/.../merge fail:
- MCP:
400 {"type":"invalid_request_error","message":"Your credit balance is too low to access the Anthropic API."} - REST:
500 {"error":"Internal server error","requestId":"mtj3mdxs-1ea23eda"}
So an Anthropic call somewhere in the merge chain is throwing rather than degrading, and the platform's own Anthropic credits are exhausted. It fails closed, which is the safe direction — nothing merged in a half-state.
Two things worth separating:
- Operational (owner's call): the Anthropic balance needs topping up. That's a billing decision, not something to work around, and I'm not bypassing the gate to land this PR.
- Code (a real defect): a merge should not depend on an AI summary.
generatePrRiskSummaryandaiReviewGateStateare both correctly guarded — the first try/catches to a deterministic fallback, the second is a pure DB read — so the unguarded call is a third one I haven't located from the API surface alone. This is the documented "merge-gate holes fail closed when the Anthropic bill lapses" case materialising for real.
Also worth noting: MCP surfaced the true cause while REST returned a bare 500. The generic message is defensible for not leaking internals, but a requestId with no operator-reachable error log is not much of a trade.
This PR is unchanged and ready; it lands as soon as the platform can merge.
Files changed
Side-by-side diff for fix/rule-scope-and-saturation → main.
Cross-repo impact
See what breaks downstream if this PR merges.
See full signal breakdown
- files changed: 5
- lines added/removed: 672 / 14
- distinct owners touched: 0
- schema migration touched: no
- locked / sensitive path touched: no
- adds new dependency: yes
- bumps major dependency: no
- tests added for new code: yes
- diff-minus-test ratio: 0.26
src/lib/pr-risk.ts computePrRiskScore.
Two opposite defects, found the same day by two independent engines. The fixes depend on each other, which is why they ship together.
False positives — a rule fired where its syntax cannot exist
Vapron opened all 15 new
highfindings. All 15 were false.waf-rs/src/owasp.rs:71—no-evalevalthere is a WAF signature string, not a call.provisioners.ts×9 —no-shell-injectionthis.exec()is a PostgresExec, not a shell. Identifiers go through correctquoteIdentquoting.neon-provisioning.ts,dns-replica.ts×6 —no-sql-injectionsanitiseIdoutput ([a-z0-9_]only) and module constants; values use?placeholders.Only the first is fixable without dataflow — and it's the same defect as the credential rule reading
CELITECH_TOKEN_URLand flagging a URL: match a token, infer a meaning the surrounding language does not support.Rules whose syntax is language-bound now declare
appliesTo. Credentials, AWS keys, private keys and security TODOs stay unconstrained — they can appear in any file.The other nine stay, and they're honest limits. One deserves quoting: a
CREATE ROLE ... PASSWORD '${password}'interpolation is safe becausegeneratePgPassword()returns hex, and the author's comment says "a hex alphabet has no quote to escape in the first place." A rule cannot see that and shouldn't pretend to — same shape aspct()being provably numeric.False negative — documentation and config files were never scanned
The eligible set was source extensions only. Verified before fixing:
So a live key pasted into a README was invisible to every rule — including the ones that matter most there.
GateTest found the identical
.mdgap in its own secrets module within hours, and only because it planted a real key as a positive control while building our five false positives as fixtures. Its module was silent on all five and it nearly reported that as a clean win; the control showed it was equally silent on a real key in the same file type. A favourable cross-engine comparison was concealing a false negative.Widening the file set is only safe because syntax-bound rules now declare where they apply.
Saturation disclosed
Criticals cost 25 each, uncapped — deliberately, so forty leaked secrets can't hide behind four. Measured consequence:
Any repo with 4+ criticals scores exactly 0 and the number stops moving, so
security.scoreis a three-state indicator wearing a 0–100 label. This bit me directly: I predicted a real repo's score would return to "roughly 80" after removing three false criticals. It stayed 0 — it could not have moved.The uncapped design is right and is unchanged.
scoreSaturatednow says when the number has stopped being a measurement, so a caller readsfindings.bySeverityinstead. Tested by tying the flag to the score actually being 0, so the two cannot drift.Our own gate caught my test fixture
The push-time secret scanner rejected the first push:
possible Private Key (PEM) in security-rule-scope.test.ts:69. Correct call. A scanner's fixtures are no more entitled to commit a key pattern than anyone else's code, and clicking past would train the repo to wave through exactly what the rule exists to catch. Assembled at runtime instead. (GateTest hit the identical rejection from GitHub push protection the same day and resolved it the same way.)Verification