Skip to content

compareReports()

PromptScript API


compareReports(baseline, candidate, options): EvalComparisonV1

Defined in: eval/src/compare.ts:231

Compare a candidate report with a baseline. Suites match by name, models by ref, cases by id. A case regressed when it passed in the baseline and does not pass in the candidate; only ids both reports have are compared. A suite or model one report lacks gets an entry that lists its cases under onlyInBaseline or onlyInCandidate, and never counts as a regression. regression is true when a case regressed or a pass rate dropped by more than the tolerance.

Parameters

baseline

EvalReportV1

The earlier report

candidate

EvalReportV1

The report under test

options

CompareOptions

Tolerance

Returns

EvalComparisonV1

The comparison, entries sorted by suite name, then model order