Skip to content

EvalPlanModel

PromptScript API


Defined in: eval/src/run.ts:1006

One model entry of a suite in a EvalPlan.

Properties

apiId

apiId: string

Defined in: eval/src/run.ts:1009


calls

calls: number

Defined in: eval/src/run.ts:1017

Calls a record or off run would make: misses, rows with one request counted once


cases

cases: number

Defined in: eval/src/run.ts:1011

Cases: rows of the dataset


hits

hits: number

Defined in: eval/src/run.ts:1013

Cases the cache answers


judgeCalls?

optional judgeCalls?: number

Defined in: eval/src/run.ts:1024

Judge calls a record or off run would make, 0 in replay. A case with no cached answer counts its judge runs at their estimate.


judgeHits?

optional judgeHits?: number

Defined in: eval/src/run.ts:1019

Judge calls the cache answers; only for a suite with judge assertions


misses

misses: number

Defined in: eval/src/run.ts:1015

Cases the cache does not answer


priceSource?

optional priceSource?: PriceSource

Defined in: eval/src/run.ts:1026

Price source, when the run is priced


provider

provider: string

Defined in: eval/src/run.ts:1008


ref

ref: string

Defined in: eval/src/run.ts:1007


worstCaseUsd?

optional worstCaseUsd?: number

Defined in: eval/src/run.ts:1028

Worst-case cost of its calls in USD, when the run is priced