Autoformalization success on undergraduate competition corpora rose from 8% to 61% between 2022 and 2025 on the miniF2F-Lean4 benchmark

open — on 2 supporting to 0 opposing, weighted 0.0 · minimal · what this means

no opposing evidence — this claim has not been tested · no sources · falsifiability unrated

0 ◎ tactic_search_tom (15) · 2 months ago · machine-learning, formal-verification

Tracking the public miniF2F-Lean4 leaderboard, the best reported pass@64 rate climbed from 8.2% (Dec 2022) to 61.4% (Nov 2025, reported by the Kestrel-7 group of Adeyemi & Lindqvist). Gains came disproportionately from search over tactic sequences guided by a learned value function, plus training on 14M synthetic Lean proof states generated by mutation of Mathlib lemmas.

rate this claim
  • Accurate 0
  • Falsifiable 0
  • Clear 0
  • Novel 0
  • Important 0

evidence

cited by (1)
[+0] By 2035 a machine-generated proof of a previously open Annals-level conjecture will be formally verified in Lean and accepted without a human-written proof sketch discuss ai-forecasting mathematics
this claim
supports (2)
[+0] Proof length distribution on solved miniF2F problems is bounded above by 40 tactic steps, with a 96th-percentile of 22 discuss
[+0] Held-out contamination audit found miniF2F-Lean4 gains persist at 54% on 300 freshly authored problems never posted online discuss
[+0] Independent re-evaluation by the Zurich formal methods group reproduced 51.8% on the held-out set with a clean-room rebuild of Kestrel-7 discuss reproducibility
whole evidence graph shown depth 1 2 3 both pro con full
what the marks mean
  • supports · opposes · cites this claim · the claim you are reading
  • [+9] net votes on the evidence link — whether this is good evidence here, not whether the claim itself is true
  • Af··· accurate, falsifiable, clear, novel, important — a capital is established, lowercase weak, unrated; no strip at all where nothing has been rated yet
  • [?] no score has been computed for that claim yet
  • field shown only where a claim's fields differ from the one above it, so a field marks where the argument crosses into another

respondeo

Loading responses… open them

discussion

Log in to join the discussion.