EvalPlan
Defined in: eval/src/run.ts:1040
What a run would do, without any call.
Properties
budgetUsd?
optionalbudgetUsd?:number
Defined in: eval/src/run.ts:1054
The --max-cost limit, when given
calls
calls:
number
Defined in: eval/src/run.ts:1046
errors
errors:
EvalFinding[]
Defined in: eval/src/run.ts:1056
Setup errors: the same a run would stop at
hits
hits:
number
Defined in: eval/src/run.ts:1044
judgeCalls?
optionaljudgeCalls?:number
Defined in: eval/src/run.ts:1050
Judge calls a run would make; only when a suite has judge assertions
judgeHits?
optionaljudgeHits?:number
Defined in: eval/src/run.ts:1048
Judge calls the cache answers; only when a suite has judge assertions
misses
misses:
number
Defined in: eval/src/run.ts:1045
mode
mode:
EvalCacheMode
Defined in: eval/src/run.ts:1041
suites
suites:
EvalPlanSuite[]
Defined in: eval/src/run.ts:1043
Empty when setup found errors
worstCaseUsd?
optionalworstCaseUsd?:number
Defined in: eval/src/run.ts:1052
Worst case of every planned call in USD, when a budget applies