EvalLlmJudgeAssertion
Defined in: core/src/types/eval.ts:171
Passes when the majority of judge runs say pass. The rubric receives
the inputs response, case_inputs, expected, criteria and the
reserved boundary.
Extends
Properties
criteria?
optionalcriteria?:string
Defined in: core/src/types/eval.ts:179
Extra criteria passed to the rubric as its criteria input
field?
optionalfield?:string
Defined in: core/src/types/eval.ts:109
Output property the assertion reads; absent means the whole response text
Inherited from
id
id:
string
Defined in: core/src/types/eval.ts:107
Assertion id, unique in the suite.
Pattern
^[A-Za-z0-9][A-Za-z0-9_.-]{0,63}$
Example
"decision-matches"Inherited from
kind
kind:
"llm-judge"
Defined in: core/src/types/eval.ts:172
model?
optionalmodel?:string
Defined in: core/src/types/eval.ts:181
Model reference the judge runs on; absent means the suite judge, then the rubric’s own model
rubric
rubric:
string
Defined in: core/src/types/eval.ts:177
Prompt id of the rubric.
Example
"acme-support/judge-polite-reply"runs?
optionalruns?:EvalJudgeRuns
Defined in: core/src/types/eval.ts:183
Judge runs per case; absent means the suite judge setting