Skip to content
Demonstrator · items marked Example are invented · what exists today
E2ER

Evaluations · ev-0002

qualitative-coding@0.2.0: stress test

Agreement with human coders stays above 0.75 up to 6,000 words per transcript.

Example with invented datatemplatestress

Record

block:        qualitative-coding@0.2.0
test:         stress
material:     Interview corpus, 240 transcripts, coded by two humans
model:        a named model and version
runs:         36 in total, 7 failed, 2 aborted
result:       Agreement with human coders stays above 0.75 up to 6,000 words per transcript.
breaking:     Agreement falls below 0.75 at about 7,000 words.
evaluator:    Demo contributor, contributor role "validation"
scope:        Valid for this corpus; other corpora need their own record.
date:         2026-12-02

The record counts failed and aborted runs, because reporting only successful runs overstates performance.

Cite this evaluation record
Persistent identifiers planned
@software{democontributor2026evaluationofqualitativec,
  title   = {Evaluation of qualitative-coding@0.2.0 (stress)},
  author  = {Demo contributor},
  year    = {2026},
  note    = {E2ER evaluation record. Persistent identifier planned},
  url     = {https://e2er.org/evaluations/ev-0002}
}
Report a failure

Versions of qualitative-coding

0.3.1in reviewSplits long transcripts before coding.ev-0001
0.2.0listedFirst listed version.ev-0002

Other records for this block

Add a record

Another corpus, model or test type needs its own record.

Evaluate this block