CodeIssuesPull RequestsActionsSecurityInsights
✨ AI
More
Settings

fix(health): rules that fired in the wrong language, and files never scanned at all #5614

OpenXLccantynz wants to mergefix/rule-scope-and-saturationmain↑4opened 51m agoLive: 0 editing
ccantynzcommented 51m ago

Two opposite defects, found the same day by two independent engines. The fixes depend on each other, which is why they ship together.

False positives — a rule fired where its syntax cannot exist

Vapron opened all 15 new high findings. All 15 were false.

finding what it actually is
waf-rs/src/owasp.rs:71no-eval Rust. eval there is a WAF signature string, not a call.
provisioners.ts ×9 — no-shell-injection this.exec() is a PostgresExec, not a shell. Identifiers go through correct quoteIdent quoting.
neon-provisioning.ts, dns-replica.ts ×6 — no-sql-injection Interpolations are sanitiseId output ([a-z0-9_] only) and module constants; values use ? placeholders.

Only the first is fixable without dataflow — and it's the same defect as the credential rule reading CELITECH_TOKEN_URL and flagging a URL: match a token, infer a meaning the surrounding language does not support.

Rules whose syntax is language-bound now declare appliesTo. Credentials, AWS keys, private keys and security TODOs stay unconstrained — they can appear in any file.

The other nine stay, and they're honest limits. One deserves quoting: a CREATE ROLE ... PASSWORD '${password}' interpolation is safe because generatePgPassword() returns hex, and the author's comment says "a hex alphabet has no quote to escape in the first place." A rule cannot see that and shouldn't pretend to — same shape as pct() being provably numeric.

False negative — documentation and config files were never scanned

The eligible set was source extensions only. Verified before fixing:

README.md        scanned=False
docs/setup.md    scanned=False
infra/main.tf    scanned=False
config.toml      scanned=False
app.ini          scanned=False

So a live key pasted into a README was invisible to every rule — including the ones that matter most there.

GateTest found the identical .md gap in its own secrets module within hours, and only because it planted a real key as a positive control while building our five false positives as fixtures. Its module was silent on all five and it nearly reported that as a clean win; the control showed it was equally silent on a real key in the same file type. A favourable cross-engine comparison was concealing a false negative.

Widening the file set is only safe because syntax-bound rules now declare where they apply.

Saturation disclosed

Criticals cost 25 each, uncapped — deliberately, so forty leaked secrets can't hide behind four. Measured consequence:

6 → 0    4 → 0    2 → 0    1 → 15    0 → 40

Any repo with 4+ criticals scores exactly 0 and the number stops moving, so security.score is a three-state indicator wearing a 0–100 label. This bit me directly: I predicted a real repo's score would return to "roughly 80" after removing three false criticals. It stayed 0 — it could not have moved.

The uncapped design is right and is unchanged. scoreSaturated now says when the number has stopped being a measurement, so a caller reads findings.bySeverity instead. Tested by tying the flag to the score actually being 0, so the two cannot drift.

Our own gate caught my test fixture

The push-time secret scanner rejected the first push: possible Private Key (PEM) in security-rule-scope.test.ts:69. Correct call. A scanner's fixtures are no more entitled to commit a key pattern than anyone else's code, and clicking past would train the repo to wave through exactly what the rule exists to catch. Assembled at runtime instead. (GateTest hit the identical rejection from GitHub push protection the same day and resolved it the same way.)

Verification

  • 12 new scope tests asserting both directions — a rule must not fire in the wrong language, and language-agnostic rules must fire in docs/config.
  • 4 saturation tests, one tying the flag to the score being 0.
  • Full CI-safe suite: 5,375 pass, 11 fail — identical to the pre-change baseline.
ccantynzAI Reviewcommented 51m ago

AI Triage

(no summary)

Priority: medium Risk area: mixed

Suggested labels: (no label suggestions) Suggested reviewers: (no reviewer suggestions)

Suggestions only — nothing has been applied. The PR author stays in control.

gluecron[bot]🤖 botAI Reviewcommented 51m ago

AI review unavailable

The platform's AI balance is exhausted, so AI generation is temporarily unavailable. Nothing was lost. You can queue this as a repair for the internal agent from the repository's Health page, or try again once the balance is restored. The PR is otherwise unchanged.

ccantynzcommented 47m ago

Merge blocked — not by this PR. CI is green on 89519e3, mergeable: true, and both the MCP merge tool and POST /api/v2/.../merge fail:

  • MCP: 400 {"type":"invalid_request_error","message":"Your credit balance is too low to access the Anthropic API."}
  • REST: 500 {"error":"Internal server error","requestId":"mtj3mdxs-1ea23eda"}

So an Anthropic call somewhere in the merge chain is throwing rather than degrading, and the platform's own Anthropic credits are exhausted. It fails closed, which is the safe direction — nothing merged in a half-state.

Two things worth separating:

  1. Operational (owner's call): the Anthropic balance needs topping up. That's a billing decision, not something to work around, and I'm not bypassing the gate to land this PR.
  2. Code (a real defect): a merge should not depend on an AI summary. generatePrRiskSummary and aiReviewGateState are both correctly guarded — the first try/catches to a deterministic fallback, the second is a pure DB read — so the unguarded call is a third one I haven't located from the API surface alone. This is the documented "merge-gate holes fail closed when the Anthropic bill lapses" case materialising for real.

Also worth noting: MCP surfaced the true cause while REST returned a bare 500. The generic message is defensible for not leaking internals, but a requestId with no operator-reachable error log is not much of a trade.

This PR is unchanged and ready; it lands as soon as the platform can merge.

Files changed

Side-by-side diff for fix/rule-scope-and-saturation → main.

View diff →

Cross-repo impact

See what breaks downstream if this PR merges.

Analyze →
Risk score:ℹ MEDIUM (4/10)8f65286
Touches 5 file(s) with 672 added and 14 removed across 0 owner(s). Includes a new dependency.
See full signal breakdown
  • files changed: 5
  • lines added/removed: 672 / 14
  • distinct owners touched: 0
  • schema migration touched: no
  • locked / sensitive path touched: no
  • adds new dependency: yes
  • bumps major dependency: no
  • tests added for new code: yes
  • diff-minus-test ratio: 0.26
How is this calculated? The score is a transparent sum of weighted signals — see src/lib/pr-risk.ts computePrRiskScore.
c comment · e edit title · m merge · a approve · r request changes · ? shortcuts