Skip to content

Add Human Evaluation from inside Evalite #92

@mattpocock

Description

@mattpocock

The idea is:

  • You would specify a special humanScorer() in the scores array passed to evalite()
  • In the UI, you could mark the result with a tick or cross
  • This would be saved to a local JSON file, keyed under the eval name, input, expected and output. (this means that if the input, expected or output changes, the human score would be invalidated)
  • The UI would then retrieve this JSON information where needed to display the scores.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions