Class: EvaluationLogger

TypeScript SDK reference

weave / EvaluationLogger

EvaluationLogger enables incremental logging of predictions and scores.

Unlike the traditional Evaluation class which requires upfront dataset and batch processing, EvaluationLogger allows you to log predictions as they happen, with flexible scoring.

Example

const ev = new EvaluationLogger({name: 'my-eval', dataset: 'my-dataset'});

for (const example of streamingData) {
  const output = await myModel.predict(example);
  const pred = await ev.logPrediction(example, output);

  if (shouldScore(output)) {
    await pred.logScore("accuracy", calculateAccuracy(output));
  }
  await pred.finish();
}

await ev.logSummary();

Table of contents

Constructors

• constructor

Methods

• logPrediction
• logSummary

Constructors

constructor

• new EvaluationLogger(options): EvaluationLogger

Parameters

Name	Type
options	EvaluationLoggerOptions

Returns

EvaluationLogger

Defined in

evaluationLogger.ts:502

Methods

logPrediction

▸ logPrediction(inputs, output): Promise<ScoreLogger>

Log a prediction with its input and output. Creates a predict_and_score call (with child predict call). Returns a ScoreLogger for adding scores.

Parameters

Name	Type
inputs	Record<string, any>
output	any

Returns

Promise<ScoreLogger>

Defined in

evaluationLogger.ts:579

---

logSummary

▸ logSummary(summary?): Promise<void>

Log a summary and finalize the evaluation. Creates a summarize call and finishes the evaluate call.

Parameters

Name	Type
summary?	Record<string, any>

Returns

Promise<void>

Defined in

evaluationLogger.ts:669