AI engineering · interactive portfolio work

LLM Annotation Review Workbench

A browser-only review queue for independent labels, disagreement triage, adjudication and agreement statistics.

Browser-only demonstration
Human review workflow
Turn disagreement into an auditable decision

A small synthetic queue demonstrates independent labels, disagreement review and adjudication. Edit the label pairs to see agreement update.

Review queue
CaseInput / outputReviewer AReviewer BDecision
Agreement on jointly reviewed cases
Overlap—
Exact agreement—
Cohen’s κ—
Open disputes—

Unweighted Cohen’s kappa is undefined if expected agreement is 1 or there are no jointly labeled cases.

Synthetic examples only—this front-end stores changes in your current browser session and does not call the repository’s API or persist reviewer decisions. The source project is a local educational SQLite demo; it does not implement authentication, privacy controls, or immutable adjudication history.

Companion demo for the source repository. The sample prompts and responses are synthetic.