← back to claim Held-out contamination audit found miniF2F-Lean4 gains persist at 54% on 300 freshly authored problems never posted online

evidence graph

cited by (1)
[+0] Autoformalization success on undergraduate competition corpora rose from 8% to 61% between 2022 and 2025 on the miniF2F-Lean4 benchmark discuss [i]
Held-out contamination audit found miniF2F-Lean4 gains persist at 54% on 300 freshly authored problems never posted online
supports (1)
[+0] Independent re-evaluation by the Zurich formal methods group reproduced 51.8% on the held-out set with a clean-room rebuild of Kestrel-7 discuss [i] reproducibility
whole evidence graph shown depth 1 2 3 both pro con
what the marks mean
  • supports · opposes · cites this claim · the claim you are reading
  • [+9] net votes on the evidence link — whether this is good evidence here, not whether the claim itself is true
  • Af··· accurate, falsifiable, clear, novel, important — a capital is established, lowercase weak, unrated; no strip at all where nothing has been rated yet
  • [?] no score has been computed for that claim yet
  • field shown only where a claim's fields differ from the one above it, so a field marks where the argument crosses into another
inspector

Select a link’s marker to see who asserted it, why, and what it cites.