Library Verdicts

An agent choosing a dependency has two sources: download counts, and whatever its training data absorbed. Neither contains the thing that decides the choice -- what the library actually does at the edge, on this version, today. Training data holds the README. It does not hold the result of calling sanitize("CON.txt") and watching an empty string come back.

EVERY VERDICT HERE WAS GENERATED BY EXECUTION against the installed package, with the version and timestamp resolved at probe time and never declared by hand.

WHAT THE CORPUS FOUND. sanitize-filename 1.6.4 returns an empty string for CON.txt, and path.join(dir, "") is the directory itself. tldts 7.4.10 collapses victim.github.io to github.io, erasing the boundary the Public Suffix List private section exists to draw. psl 1.15.0 answers "1.1" for the IP literal 192.168.1.1 -- a value that looks like a two-label domain and passes a truthiness check. mime-types 3.0.2 answers application/pdf for a file whose bytes are a Windows executable, because it maps extension to type and never reads a byte. normalize-email 1.1.1 and cron-parser 5.7.0 were measured and found SOUND, and saying so is part of the product: four defective and two clean is a finding, not a sales pitch.

THE RULE THE CORPUS RESTS ON: a probe that did not run produces no verdict. A missing library, a changed export or a thrown import yields unavailable, never passed. A corpus that silently records an uninstalled package as sound is confidently wrong in the direction that gets a defective library adopted. unmeasured takes precedence over sound, and an empty probe set is unmeasured, because zero failures is not evidence of correctness.

ONE WRONG ANSWER IS A DEFECT, NOT A PERCENTAGE. The verdict is deliberately not a ratio: a library right 95 percent of the time that returns an empty string for CON.txt is not 95 percent safe, because the 5 percent is the case an attacker picks.

VERDICTS ARE VERSIONED so a fix can clear a name. "tldts is wrong about github.io" is not a fact about tldts, it is a fact about a version on a date, and a corpus that forgets that becomes a libel the moment the maintainer ships a fix.

THE HAZARD, STATED PLAINLY: a bad probe does not produce a missing verdict, it produces a confident libel of a correct library and it looks exactly like a real finding. This happened during construction -- an early cron-parser probe called next() twice, read the fire after the DST transition, and recorded a correct library as defective. Fixed, and documented in the source as a warning to anyone adding probes.

WHAT THIS DOES NOT CLAIM: six libraries is a seed corpus, not a database of npm. It is the method, the harness and the verdicts accumulated so far. The libraries were chosen because a real product depended on the answer, not to be representative. The harness ships so you can re-run every claim and add your own.

VERIFIED: 91 tests, 10/10 mutations caught.

DELIVERY: signed download of a hash-verified tarball, immediately on purchase. Permissive licence: unlimited products, unlimited clients, unlimited seats, no attribution, perpetual and irrevocable. One restriction, do not republish the source as source.

Interface

What you call, and what comes back. Types and signatures only — the implementation ships with the source.

  export function compareToCorpus( committedVerdicts: LibraryVerdict[], freshVerdicts: LibraryVerdict[], ):;
  export function summarise( library: string, version: string | null, measuredAt: string, cases: CaseResult[], ): LibraryVerdict;
  export function render(v: unknown): string;
  export async function runProbe(library: string, opts: RunOptions, measuredAt: string): Promise<LibraryVerdict>;
  export function summariseCorpus(verdicts: LibraryVerdict[], measuredAt: string): CorpusSummary;
  export function classifySpread(samples: number[]): "clumped" | "narrow" | "spread";

  export type CaseOutcome = "correct" | "incorrect" | "threw" | "unavailable";