EvalComparisonEntryV1
Defined in: core/src/types/eval.ts:764
Comparison of one suite and model entry between two reports.
Properties
costUsdDelta?
optionalcostUsdDelta?:number
Defined in: core/src/types/eval.ts:772
Candidate cost minus baseline cost in USD, when both reports carry a cost
datasetChanged
datasetChanged:
boolean
Defined in: core/src/types/eval.ts:784
Whether the dataset hashes differ
fixes
fixes:
string[]
Defined in: core/src/types/eval.ts:778
Cases that passed in the candidate and not in the baseline
latencyP95MsDelta
latencyP95MsDelta:
number
Defined in: core/src/types/eval.ts:774
Candidate p95 latency minus baseline p95 latency in milliseconds
model
model:
string
Defined in: core/src/types/eval.ts:768
Model reference
onlyInBaseline
onlyInBaseline:
string[]
Defined in: core/src/types/eval.ts:780
Case ids only the baseline has
onlyInCandidate
onlyInCandidate:
string[]
Defined in: core/src/types/eval.ts:782
Case ids only the candidate has
passRateDelta
passRateDelta:
number
Defined in: core/src/types/eval.ts:770
Candidate pass rate minus baseline pass rate
regressions
regressions:
string[]
Defined in: core/src/types/eval.ts:776
Cases that passed in the baseline and not in the candidate
suite
suite:
string
Defined in: core/src/types/eval.ts:766
Suite name