Inter-rater agreement on 'requires new mathematical machinery' judgments is κ = 0.31 among 22 research mathematicians shown 40 solved problems
[unresolved]
[no evidence]
[quiet]
[stable]
[undecided]
Prieto & Halvorsen (2029) presented 22 number theorists and combinatorialists with 40 problems solved between 1995 and 2015 and asked whether each required machinery unavailable at the time of posing. Agreement was poor (Fleiss κ = 0.31), and ratings correlated 0.44 with the rater's own subfield.
Accurate
(+0)
Falsifiable
(+0)
Clear
(+0)
Novel
(+0)
Important
(+0)
▸ Score Details
Cited as evidence in 1 claim:
The 2028 survey's six 'requires new construction' problems …
(opposes)
cited by (1)
cited as opposes by:
[+0]
The 2028 survey's six 'requires new construction' problems were rated by referees who knew the eventual human solution in four cases
mathematics
research-methods
virtues: Accurate unrated · Falsifiable unrated · Clear unrated · Novel unrated · Important unrated
this claim:
Inter-rater agreement on 'requires new mathematical machinery' judgments is κ = 0.31 among 22 research mathematicians shown 40 solved problems
supports (0)
no supporting evidence yet
opposes (0)
no opposing evidence yet
Evidence Supporting (0)
No supporting evidence yet.
Evidence Against (0)
No opposing evidence yet.
Respondeo
Loading responses...
Log in to join the discussion.