fix(archaeology): rendered the provider's raw error, and called it high confidence #5488
ccantynzAI Reviewcommented 21d ago
AI Triage
(no summary)
Priority: medium Risk area: mixed
Suggested labels: (no label suggestions) Suggested reviewers: (no reviewer suggestions)
Suggestions only — nothing has been applied. The PR author stays in control.
gluecron[bot]🤖 botAI Reviewcommented 21d ago
AI review unavailable
The platform's AI balance is exhausted, so AI generation is temporarily unavailable. Nothing was lost. You can queue this as a repair for the internal agent from the repository's Health page, or try again once the balance is restored. The PR is otherwise unchanged.
Cross-repo impact
See what breaks downstream if this PR merges.
⮌ Merged
This pull request was merged into main.
c comment · e edit title · m merge · a approve · r request changes · ? shortcuts
Two defects that compound each other into something worse than either.
1. The provider's error text was rendered to the page
} catch (err) { explanation = `Unable to generate explanation: ${err instanceof Error ? err.message : String(err)}`; }explanationis displayed to whoever is viewing the repo — including anonymous visitors on a public one. Anthropic's SDK errors carry operator billing state verbatim:So an AI outage published our account status to strangers. Observed 2026-08-11.
The outer catch did the same thing, and that path can surface database and filesystem errors — which leak internal paths and schema just as readily.
2. It was labelled "High confidence"
const confidence = deriveConfidence(commits, prs, relatedIssues);deriveConfidencescores the git evidence — commit, PR and issue counts. It knows nothing about whether the AI step succeeded.So a file with rich history scored
higheven when the explanation had just failed. The page renders explanation and confidence side by side, which means a well-trafficked repo displayed:A confidence score attached to the thing that didn't happen.
The fix
console.errorkeeps the stack for the operator, where it's needed.git log, not Claude.lowwhen the explanation failed. A report whose central claim could not be generated is not high confidence, whatever the git log says.Also checked
Grepped the sibling AI surfaces —
nl-search.ts,semantic-search.ts,claude-semantic-search.ts— for the sameerr.message-into-output pattern. None found, so this is isolated rather than a house style.Verification
tsc --noEmit: 0 errors in the changed file.To confirm after deploy: with
ANTHROPIC_API_KEYunset or the account out of credit, open Archaeology on any file. Expect the generic message and "Low confidence" — no provider text, no billing state.Tier 0 item T0-10 from the fleet dead-surface sweep. Context:
REMEDIATION-PLAN.mdinccantynz/platform-contracts.🤖 Generated with Claude Code