EvalComparisonV1
Defined in: core/src/types/eval.ts:791
Comparison of a candidate report against a baseline, version 1: the
JSON output of a standalone compare, or comparison inside a report.
Properties
$schema?
optional$schema?:"https://getpromptscript.dev/schema/eval-comparison/v1.json"
Defined in: core/src/types/eval.ts:797
Schema URL, the version tag of a standalone comparison; absent inside a report.
Example
"https://getpromptscript.dev/schema/eval-comparison/v1.json"entries
entries:
EvalComparisonEntryV1[]
Defined in: core/src/types/eval.ts:813
Entries sorted by suite name, then model order
regression
regression:
boolean
Defined in: core/src/types/eval.ts:811
Whether any case regressed or a pass rate dropped beyond the tolerance
tolerance
tolerance:
number
Defined in: core/src/types/eval.ts:809
Pass-rate drop that counts as a regression.
Minimum
0
Maximum
1
version?
optionalversion?:1
Defined in: core/src/types/eval.ts:803
Format version, 1 in standalone comparisons this CLI writes. Absent inside a report and in comparisons from older versions; read a missing value as 1.