CodeIssuesPull RequestsActionsSecurityInsights
✨ AI
More
Settings

fix(archaeology): rendered the provider's raw error, and called it high confidence #5488

Merged⚡ AI-generatedXSccantynz wants to mergefix/archaeology-stops-leaking-provider-errorsmainopened 21d ago
ccantynzcommented 21d ago

Two defects that compound each other into something worse than either.

1. The provider's error text was rendered to the page

} catch (err) {
  explanation = `Unable to generate explanation: ${err instanceof Error ? err.message : String(err)}`;
}

explanation is displayed to whoever is viewing the repo — including anonymous visitors on a public one. Anthropic's SDK errors carry operator billing state verbatim:

Your credit balance is too low to access the Anthropic API…

So an AI outage published our account status to strangers. Observed 2026-08-11.

The outer catch did the same thing, and that path can surface database and filesystem errors — which leak internal paths and schema just as readily.

2. It was labelled "High confidence"

const confidence = deriveConfidence(commits, prs, relatedIssues);

deriveConfidence scores the git evidence — commit, PR and issue counts. It knows nothing about whether the AI step succeeded.

So a file with rich history scored high even when the explanation had just failed. The page renders explanation and confidence side by side, which means a well-trafficked repo displayed:

High confidence — Unable to generate explanation: Your credit balance is too low…

A confidence score attached to the thing that didn't happen.

The fix

  • Real error → server log. console.error keeps the stack for the operator, where it's needed.
  • Page → a message that is true and useless to an attacker. It also tells the user the git findings below are unaffected, which is accurate — those come from git log, not Claude.
  • Confidence clamped to low when the explanation failed. A report whose central claim could not be generated is not high confidence, whatever the git log says.

Also checked

Grepped the sibling AI surfaces — nl-search.ts, semantic-search.ts, claude-semantic-search.ts — for the same err.message-into-output pattern. None found, so this is isolated rather than a house style.

Verification

tsc --noEmit: 0 errors in the changed file.

To confirm after deploy: with ANTHROPIC_API_KEY unset or the account out of credit, open Archaeology on any file. Expect the generic message and "Low confidence" — no provider text, no billing state.


Tier 0 item T0-10 from the fleet dead-surface sweep. Context: REMEDIATION-PLAN.md in ccantynz/platform-contracts.

🤖 Generated with Claude Code

ccantynzAI Reviewcommented 21d ago

AI Triage

(no summary)

Priority: medium Risk area: mixed

Suggested labels: (no label suggestions) Suggested reviewers: (no reviewer suggestions)

Suggestions only — nothing has been applied. The PR author stays in control.

gluecron[bot]🤖 botAI Reviewcommented 21d ago

AI review unavailable

The platform's AI balance is exhausted, so AI generation is temporarily unavailable. Nothing was lost. You can queue this as a repair for the internal agent from the repository's Health page, or try again once the balance is restored. The PR is otherwise unchanged.

Cross-repo impact

See what breaks downstream if this PR merges.

Analyze →
⮌ Merged

This pull request was merged into main.

c comment · e edit title · m merge · a approve · r request changes · ? shortcuts