EvalConfig
Defined in: core/src/types/config.ts:403
Eval settings (eval in promptscript.yaml): the providers eval suites
call and the limits of a run.
Properties
maxCases?
optionalmaxCases?:number
Defined in: core/src/types/config.ts:432
Largest number of dataset rows a suite runs.
As Type
integer
Minimum
1
Default
1000maxOutputTokens?
optionalmaxOutputTokens?:number
Defined in: core/src/types/config.ts:425
Output token cap of a model call when the prompt sets no maxTokens.
As Type
integer
Minimum
1
Default
1024providers?
optionalproviders?:Record<string,EvalProviderConfig>
Defined in: core/src/types/config.ts:411
Providers by name. The name matches the provider of the resolved
model profile, such as anthropic or openai.
Example
providers: anthropic: { kind: anthropic, apiKeyEnvVar: ANTHROPIC_API_KEY }reportDir?
optionalreportDir?:string
Defined in: core/src/types/config.ts:437
Directory for eval reports, relative to the project root.
Example
".promptscript/eval-reports"timeoutMs?
optionaltimeoutMs?:number
Defined in: core/src/types/config.ts:418
Time limit of one model call in milliseconds, response body included.
As Type
integer
Minimum
1
Default
60000