Skip to content

importSamples()

PromptScript API


importSamples(options): Promise<ImportResult>

Defined in: eval/src/import/import.ts:413

Turn a JSONL file of production samples into dataset rows. Every step runs before anything is written, and the function writes nothing itself.

A line that breaks the closed sample schema is skipped and counted; its content is never kept. Inputs the prompt does not declare skip the sample, and so does a sample that fits the bundle of its hash but not the current prompt. Every string input goes through scrubPii, and the result is checked with detectPii: a value that still holds personal data stops the import with PS6714, so a raw value is never returned. A sample that hides a value with zero-width characters, fullwidth forms or dots is skipped, and so is one with a number input that could be a card number. Rows are deduplicated by their scrubbed inputs and sampled down with a seeded reservoir.

Parameters

options

ImportSamplesOptions

Returns

Promise<ImportResult>

Throws

EvalError PS6711 when the file cannot be read, PS6712 over the line limit, PS6714 when the scrub left personal data