ImportSamplesOptions
Defined in: eval/src/import/import.ts:34
Options of importSamples.
Properties
allowedPatterns?
optionalallowedPatterns?: readonlystring[]
Defined in: eval/src/import/import.ts:64
validation.allowedPatterns of the project, for the scan
contractFor
contractFor: (
contentHash) =>ImportContract
Defined in: eval/src/import/import.ts:43
The contract a sample’s inputs must fit: the verified bundle of its content hash when
there is one, else the current compile. Called with undefined for a sample without a hash.
Parameters
contentHash
string | undefined
Returns
current
current:
ImportContract
Defined in: eval/src/import/import.ts:48
The contract of the current compile. A row is written only when its inputs fit this one
too, because prs eval run checks rows against it.
expectedFromOutput?
optionalexpectedFromOutput?:boolean
Defined in: eval/src/import/import.ts:62
Copy the scrubbed output as expected
file
file:
string
Defined in: eval/src/import/import.ts:36
Real path of the sample file
labels?
optionallabels?: readonlystring[]
Defined in: eval/src/import/import.ts:58
Keep only samples with one of these feedback labels
maxRows
maxRows:
number
Defined in: eval/src/import/import.ts:52
Largest number of rows kept
maxScore?
optionalmaxScore?:number
Defined in: eval/src/import/import.ts:56
Keep only samples with a feedback score at or below this
outFile
outFile:
string
Defined in: eval/src/import/import.ts:50
Dataset path relative to the project root, for the findings
prompt
prompt:
string
Defined in: eval/src/import/import.ts:38
Prompt id of the suite; samples of other prompts are left out
redactInputs?
optionalredactInputs?: readonlystring[]
Defined in: eval/src/import/import.ts:60
Names of string inputs replaced by [redacted]
seed
seed:
number
Defined in: eval/src/import/import.ts:54
Seed of the reservoir sampling