Skip to content

EvalPlan

PromptScript API


Defined in: eval/src/run.ts:1040

What a run would do, without any call.

Properties

budgetUsd?

optional budgetUsd?: number

Defined in: eval/src/run.ts:1054

The --max-cost limit, when given


calls

calls: number

Defined in: eval/src/run.ts:1046


errors

errors: EvalFinding[]

Defined in: eval/src/run.ts:1056

Setup errors: the same a run would stop at


hits

hits: number

Defined in: eval/src/run.ts:1044


judgeCalls?

optional judgeCalls?: number

Defined in: eval/src/run.ts:1050

Judge calls a run would make; only when a suite has judge assertions


judgeHits?

optional judgeHits?: number

Defined in: eval/src/run.ts:1048

Judge calls the cache answers; only when a suite has judge assertions


misses

misses: number

Defined in: eval/src/run.ts:1045


mode

mode: EvalCacheMode

Defined in: eval/src/run.ts:1041


suites

suites: EvalPlanSuite[]

Defined in: eval/src/run.ts:1043

Empty when setup found errors


worstCaseUsd?

optional worstCaseUsd?: number

Defined in: eval/src/run.ts:1052

Worst case of every planned call in USD, when a budget applies