EvalSampleV1
Defined in: core/src/types/eval-import.ts:26
One production sample, version 1: a line of the JSONL file prs eval import
reads. The schema is closed: there is no field for a comment, an assignment
key, a user, a session or a trace.
Properties
contentHash?
optionalcontentHash?:string
Defined in: core/src/types/eval-import.ts:38
Content hash of the served prompt: sha256- plus 64 lowercase hex characters.
Pattern
^sha256-[0-9a-f]{64}$
feedback?
optionalfeedback?:EvalSampleFeedbackV1
Defined in: core/src/types/eval-import.ts:50
Feedback on the response
inputs
inputs:
Record<string,EvalInputValue>
Defined in: core/src/types/eval-import.ts:46
Prompt input values by input name; scalars only
output?
optionaloutput?:string
Defined in: core/src/types/eval-import.ts:48
Model response text
prompt
prompt:
string
Defined in: core/src/types/eval-import.ts:33
Prompt id the sample was served from.
Min Length
1
Max Length
200
Example
"acme-support/refund-reply"variant?
optionalvariant?:string
Defined in: core/src/types/eval-import.ts:44
Arm of an online experiment, or control.
Pattern
^(?:control|[a-z][a-z0-9-]{0,31})$
Example
"warm"