Quota Exhaustion Verdict

Decide whether an automated review step ACTUALLY RAN, or whether its meter ran out and the absence was read as a pass.

THE SILENT FAILURE, OBSERVED IN A LIVE PUBLIC PULL REQUEST. On github.com/EffortlessMetrics/perl-lsp-swarm/pull/5020, two different vendors' quotas were exhausted on the SAME PR, minutes apart, as ordinary bot comments:

  chatgpt-codex-connector[bot]: "You have reached your Codex usage limits for code reviews. To continue using code reviews, add credits to your account."
  coderabbitai[bot]:            "Review limit reached."

The pull request still merged. CI stayed green. A human scrolling the thread sees bot comments and reasonably concludes review happened. What actually happened is that a review STEP WAS SKIPPED, and the only trace is prose in a comment — no exit code, no failed check, nothing a merge gate reads.

THIS IS NOT A VENDOR HONESTY PROBLEM, and the listing says so plainly. Both vendors disclose exhaustion clearly and in a stable, machine-readable form — CodeRabbit ships an HTML marker comment present in both its old and new wording; Codex's sentence was identical in every occurrence found. This package would not exist without that disclosure. The gap it closes is between "the tool said so in prose" and "the platform's pass/fail field reflects it", which is a merge-gate wiring problem.

WHAT IT DECIDES. Given the facts of a run — which review steps were expected, which produced a real verdict, which produced a quota notice — it distinguishes "reviewed and found nothing" from "never ran". Those two are identical to every check that looks only at exit codes, which is why the failure survives.

IT REFUSES RATHER THAN GUESSING, and refusals carry their numbers.

VERIFIED: 61 tests, measured by running the suite. Vendor signature strings are quoted verbatim from real observed comments with their fetch dates, and the listing records exactly how many corroborating occurrences were found rather than rounding up.

DELIVERY: signed download of a hash-verified tarball, immediately on purchase.

Interface

What you call, and what comes back. Types and signatures only — the implementation ships with the source.

  export function wouldBlockMerge( required: StepObservation["required"], checkRun: StepObservation["checkRun"], ): "yes" | "no" | "undecidable";
  export function matchVendorSignature(commentBody: string): VendorSignature | null;
  export function evaluatePipelineRun(observations: StepObservation[]): StepVerdict[];

  export type StepOutcome = "REVIEWED" | "QUOTA_EXHAUSTED" | "NEVER_RAN" | "UNKNOWN";