Pular para o conteúdo
AMANi DUNiA research
PT

Criado com assistência de IA. Fontes de medição, relatórios originais e limitações são divulgados em cada estudo.

Test series · Decision LLMs

Expanded test set and blind review

Which test cases are unambiguous, and where do reviewers disagree?

Continuar lendo ↓
Interim findingsData snapshot:

6,814 blind judgments across 33 questions

Research question

Which test cases are unambiguous, and where do reviewers disagree?

Approach

Models review a larger synthetic test set without seeing the intended answers. Disputed cases are examined separately.

What is available so far

A documented GPT-6 run contains 6,814 judgments, including 1,359 marked uncertain. A later list records 489 disputed cases from the review collection.

Limits of the findings

These counts describe different analysis stages. Model majorities do not replace a human reference; a complete, cleaned overall evaluation is not available here.

Next step

Resolve disputed cases and identify a fixed, separate evaluation set.

Basis of this account
  • Blind-review report dated 4 October 2026
  • Disputed-case list dated 5 October 2026

This page summarises the documented state of the work. A full standalone presentation of the results has not yet been prepared.

The data snapshot refers to the underlying investigation, not the date of this overview.

← Decision LLMs

Cite and reuse

Own texts, questions and classifications: CC BY 4.0. Give attribution, link the license and indicate changes. Third-party source reports retain their own rights. CC BY 4.0

AMANi DUNiA. Expanded test set and blind review. Snapshot: . https://research.amanidunia.berlin/en/studien/testsatz-blindpruefung/

Version: 2026-10-05

BibTeX
@misc{research__en_studien_testsatz_blindpruefung_,
  author = {{AMANi DUNiA}},
  title = {Expanded test set and blind review},
  year = {2026},
  url = {https://research.amanidunia.berlin/en/studien/testsatz-blindpruefung/},
  note = {Snapshot 2026-10-05; CC BY 4.0},
  version = {2026-10-05}
}
Language and review status

Main text: EN. Review notes apply only to the content they name.

Existing review notes appear with the respective content.

See language coverage and data snapshot →