Skip to content

ImportSamplesOptions

PromptScript API


Defined in: eval/src/import/import.ts:34

Options of importSamples.

Properties

allowedPatterns?

optional allowedPatterns?: readonly string[]

Defined in: eval/src/import/import.ts:64

validation.allowedPatterns of the project, for the scan


contractFor

contractFor: (contentHash) => ImportContract

Defined in: eval/src/import/import.ts:43

The contract a sample’s inputs must fit: the verified bundle of its content hash when there is one, else the current compile. Called with undefined for a sample without a hash.

Parameters

contentHash

string | undefined

Returns

ImportContract


current

current: ImportContract

Defined in: eval/src/import/import.ts:48

The contract of the current compile. A row is written only when its inputs fit this one too, because prs eval run checks rows against it.


expectedFromOutput?

optional expectedFromOutput?: boolean

Defined in: eval/src/import/import.ts:62

Copy the scrubbed output as expected


file

file: string

Defined in: eval/src/import/import.ts:36

Real path of the sample file


labels?

optional labels?: readonly string[]

Defined in: eval/src/import/import.ts:58

Keep only samples with one of these feedback labels


maxRows

maxRows: number

Defined in: eval/src/import/import.ts:52

Largest number of rows kept


maxScore?

optional maxScore?: number

Defined in: eval/src/import/import.ts:56

Keep only samples with a feedback score at or below this


outFile

outFile: string

Defined in: eval/src/import/import.ts:50

Dataset path relative to the project root, for the findings


prompt

prompt: string

Defined in: eval/src/import/import.ts:38

Prompt id of the suite; samples of other prompts are left out


redactInputs?

optional redactInputs?: readonly string[]

Defined in: eval/src/import/import.ts:60

Names of string inputs replaced by [redacted]


seed

seed: number

Defined in: eval/src/import/import.ts:54

Seed of the reservoir sampling