CLEVER: A Curated Benchmark for Formally Verified Code Generation

May 23, 2026

TL;DR: We introduce CLEVER, a hand-curated benchmark for verified code generation in Lean. It requires full formal specs and proofs. No few-shot method solves all stages, making it a strong testbed for synthesis and formal reasoning.

https://openreview.net/forum?id=pqNFDA2TFm

CLEVER: A Curated Benchmark for Formally Verified Code Generation <body> <h1> Array ( [0] => xlever: a curated benchmark for formally verified code generation [1] => vlever: a curated benchmark for formally verified code generation [2] => flever: a curated benchmark for formally verified code generation [3] => dlever: a curated benchmark for formally verified code generation [4] => dlever: a curated benchmark for formally verified code generation [5] => ckever: a curated benchmark for formally verified code generation [6] => cpever: a curated benchmark for formally verified code generation [7] => coever: a curated benchmark for formally verified code generation [8] => clwver: a curated benchmark for formally verified code generation [9] => clsver: a curated benchmark for formally verified code generation [10] => cldver: a curated benchmark for formally verified code generation [11] => clrver: a curated benchmark for formally verified code generation [12] => cl4ver: a curated benchmark for formally verified code generation [13] => cl3ver: a curated benchmark for formally verified code generation [14] => clecer: a curated benchmark for formally verified code generation [15] => cleber: a curated benchmark for formally verified code generation [16] => cleger: a curated benchmark for formally verified code generation [17] => clefer: a curated benchmark for formally verified code generation [18] => clefer: a curated benchmark for formally verified code generation [19] => clevwr: a curated benchmark for formally verified code generation [20] => clevsr: a curated benchmark for formally verified code generation [21] => clevdr: a curated benchmark for formally verified code generation [22] => clevrr: a curated benchmark for formally verified code generation [23] => clev4r: a curated benchmark for formally verified code generation [24] => clev3r: a curated benchmark for formally verified code generation [25] => clevee: a curated benchmark for formally verified code generation [26] => cleved: a curated benchmark for formally verified code generation [27] => clevef: a curated benchmark for formally verified code generation [28] => clevet: a curated benchmark for formally verified code generation [29] => cleve5: a curated benchmark for formally verified code generation [30] => cleve4: a curated benchmark for formally verified code generation [31] => clever: z curated benchmark for formally verified code generation [32] => clever: s curated benchmark for formally verified code generation [33] => clever: w curated benchmark for formally verified code generation [34] => clever: q curated benchmark for formally verified code generation [35] => clever: a xurated benchmark for formally verified code generation [36] => clever: a vurated benchmark for formally verified code generation [37] => clever: a furated benchmark for formally verified code generation [38] => clever: a durated benchmark for formally verified code generation [39] => clever: a durated benchmark for formally verified code generation [40] => clever: a cyrated benchmark for formally verified code generation [41] => clever: a chrated benchmark for formally verified code generation [42] => clever: a cjrated benchmark for formally verified code generation [43] => clever: a cirated benchmark for formally verified code generation [44] => clever: a c8rated benchmark for formally verified code generation [45] => clever: a c7rated benchmark for formally verified code generation [46] => clever: a cueated benchmark for formally verified code generation [47] => clever: a cudated benchmark for formally verified code generation [48] => clever: a cufated benchmark for formally verified code generation [49] => clever: a cutated benchmark for formally verified code generation [50] => clever: a cu5ated benchmark for formally verified code generation [51] => clever: a cu4ated benchmark for formally verified code generation [52] => clever: a curzted benchmark for formally verified code generation [53] => clever: a cursted benchmark for formally verified code generation [54] => clever: a curwted benchmark for formally verified code generation [55] => clever: a curqted benchmark for formally verified code generation [56] => clever: a curared benchmark for formally verified code generation [57] => clever: a curafed benchmark for formally verified code generation [58] => clever: a curaged benchmark for formally verified code generation [59] => clever: a curayed benchmark for formally verified code generation [60] => clever: a cura6ed benchmark for formally verified code generation [61] => clever: a cura5ed benchmark for formally verified code generation [62] => clever: a curatwd benchmark for formally verified code generation [63] => clever: a curatsd benchmark for formally verified code generation [64] => clever: a curatdd benchmark for formally verified code generation [65] => clever: a curatrd benchmark for formally verified code generation [66] => clever: a curat4d benchmark for formally verified code generation [67] => clever: a curat3d benchmark for formally verified code generation [68] => clever: a curates benchmark for formally verified code generation [69] => clever: a curatex benchmark for formally verified code generation [70] => clever: a curatec benchmark for formally verified code generation [71] => clever: a curatef benchmark for formally verified code generation [72] => clever: a curater benchmark for formally verified code generation [73] => clever: a curatee benchmark for formally verified code generation [74] => clever: a curated venchmark for formally verified code generation [75] => clever: a curated nenchmark for formally verified code generation [76] => clever: a curated henchmark for formally verified code generation [77] => clever: a curated genchmark for formally verified code generation [78] => clever: a curated genchmark for formally verified code generation [79] => clever: a curated bwnchmark for formally verified code generation [80] => clever: a curated bsnchmark for formally verified code generation [81] => clever: a curated bdnchmark for formally verified code generation [82] => clever: a curated brnchmark for formally verified code generation [83] => clever: a curated b4nchmark for formally verified code generation [84] => clever: a curated b3nchmark for formally verified code generation [85] => clever: a curated bebchmark for formally verified code generation [86] => clever: a curated bemchmark for formally verified code generation [87] => clever: a curated bejchmark for formally verified code generation [88] => clever: a curated behchmark for formally verified code generation [89] => clever: a curated behchmark for formally verified code generation [90] => clever: a curated benxhmark for formally verified code generation [91] => clever: a curated benvhmark for formally verified code generation [92] => clever: a curated benfhmark for formally verified code generation [93] => clever: a curated bendhmark for formally verified code generation [94] => clever: a curated bendhmark for formally verified code generation [95] => clever: a curated bencgmark for formally verified code generation [96] => clever: a curated bencbmark for formally verified code generation [97] => clever: a curated bencnmark for formally verified code generation [98] => clever: a curated bencjmark for formally verified code generation [99] => clever: a curated bencumark for formally verified code generation [100] => clever: a curated bencymark for formally verified code generation [101] => clever: a curated benchnark for formally verified code generation [102] => clever: a curated benchkark for formally verified code generation [103] => clever: a curated benchjark for formally verified code generation [104] => clever: a curated benchjark for formally verified code generation [105] => clever: a curated benchmzrk for formally verified code generation [106] => clever: a curated benchmsrk for formally verified code generation [107] => clever: a curated benchmwrk for formally verified code generation [108] => clever: a curated benchmqrk for formally verified code generation [109] => clever: a curated benchmaek for formally verified code generation [110] => clever: a curated benchmadk for formally verified code generation [111] => clever: a curated benchmafk for formally verified code generation [112] => clever: a curated benchmatk for formally verified code generation [113] => clever: a curated benchma5k for formally verified code generation [114] => clever: a curated benchma4k for formally verified code generation [115] => clever: a curated benchmarj for formally verified code generation [116] => clever: a curated benchmarm for formally verified code generation [117] => clever: a curated benchmarl for formally verified code generation [118] => clever: a curated benchmaro for formally verified code generation [119] => clever: a curated benchmari for formally verified code generation [120] => clever: a curated benchmark dor formally verified code generation [121] => clever: a curated benchmark cor formally verified code generation [122] => clever: a curated benchmark vor formally verified code generation [123] => clever: a curated benchmark gor formally verified code generation [124] => clever: a curated benchmark tor formally verified code generation [125] => clever: a curated benchmark ror formally verified code generation [126] => clever: a curated benchmark fir formally verified code generation [127] => clever: a curated benchmark fkr formally verified code generation [128] => clever: a curated benchmark flr formally verified code generation [129] => clever: a curated benchmark fpr formally verified code generation [130] => clever: a curated benchmark f0r formally verified code generation [131] => clever: a curated benchmark f9r formally verified code generation [132] => clever: a curated benchmark foe formally verified code generation [133] => clever: a curated benchmark fod formally verified code generation [134] => clever: a curated benchmark fof formally verified code generation [135] => clever: a curated benchmark fot formally verified code generation [136] => clever: a curated benchmark fo5 formally verified code generation [137] => clever: a curated benchmark fo4 formally verified code generation [138] => clever: a curated benchmark for dormally verified code generation [139] => clever: a curated benchmark for cormally verified code generation [140] => clever: a curated benchmark for vormally verified code generation [141] => clever: a curated benchmark for gormally verified code generation [142] => clever: a curated benchmark for tormally verified code generation [143] => clever: a curated benchmark for rormally verified code generation [144] => clever: a curated benchmark for firmally verified code generation [145] => clever: a curated benchmark for fkrmally verified code generation [146] => clever: a curated benchmark for flrmally verified code generation [147] => clever: a curated benchmark for fprmally verified code generation [148] => clever: a curated benchmark for f0rmally verified code generation [149] => clever: a curated benchmark for f9rmally verified code generation [150] => clever: a curated benchmark for foemally verified code generation [151] => clever: a curated benchmark for fodmally verified code generation [152] => clever: a curated benchmark for fofmally verified code generation [153] => clever: a curated benchmark for fotmally verified code generation [154] => clever: a curated benchmark for fo5mally verified code generation [155] => clever: a curated benchmark for fo4mally verified code generation [156] => clever: a curated benchmark for fornally verified code generation [157] => clever: a curated benchmark for forkally verified code generation [158] => clever: a curated benchmark for forjally verified code generation [159] => clever: a curated benchmark for forjally verified code generation [160] => clever: a curated benchmark for formzlly verified code generation [161] => clever: a curated benchmark for formslly verified code generation [162] => clever: a curated benchmark for formwlly verified code generation [163] => clever: a curated benchmark for formqlly verified code generation [164] => clever: a curated benchmark for formakly verified code generation [165] => clever: a curated benchmark for formaply verified code generation [166] => clever: a curated benchmark for formaoly verified code generation [167] => clever: a curated benchmark for formalky verified code generation [168] => clever: a curated benchmark for formalpy verified code generation [169] => clever: a curated benchmark for formaloy verified code generation [170] => clever: a curated benchmark for formallt verified code generation [171] => clever: a curated benchmark for formallg verified code generation [172] => clever: a curated benchmark for formallh verified code generation [173] => clever: a curated benchmark for formallu verified code generation [174] => clever: a curated benchmark for formall7 verified code generation [175] => clever: a curated benchmark for formall6 verified code generation [176] => clever: a curated benchmark for formally cerified code generation [177] => clever: a curated benchmark for formally berified code generation [178] => clever: a curated benchmark for formally gerified code generation [179] => clever: a curated benchmark for formally ferified code generation [180] => clever: a curated benchmark for formally ferified code generation [181] => clever: a curated benchmark for formally vwrified code generation [182] => clever: a curated benchmark for formally vsrified code generation [183] => clever: a curated benchmark for formally vdrified code generation [184] => clever: a curated benchmark for formally vrrified code generation [185] => clever: a curated benchmark for formally v4rified code generation [186] => clever: a curated benchmark for formally v3rified code generation [187] => clever: a curated benchmark for formally veeified code generation [188] => clever: a curated benchmark for formally vedified code generation [189] => clever: a curated benchmark for formally vefified code generation [190] => clever: a curated benchmark for formally vetified code generation [191] => clever: a curated benchmark for formally ve5ified code generation [192] => clever: a curated benchmark for formally ve4ified code generation [193] => clever: a curated benchmark for formally verufied code generation [194] => clever: a curated benchmark for formally verjfied code generation [195] => clever: a curated benchmark for formally verkfied code generation [196] => clever: a curated benchmark for formally verofied code generation [197] => clever: a curated benchmark for formally ver9fied code generation [198] => clever: a curated benchmark for formally ver8fied code generation [199] => clever: a curated benchmark for formally veridied code generation [200] => clever: a curated benchmark for formally vericied code generation [201] => clever: a curated benchmark for formally verivied code generation [202] => clever: a curated benchmark for formally verigied code generation [203] => clever: a curated benchmark for formally veritied code generation [204] => clever: a curated benchmark for formally veriried code generation [205] => clever: a curated benchmark for formally verifued code generation [206] => clever: a curated benchmark for formally verifjed code generation [207] => clever: a curated benchmark for formally verifked code generation [208] => clever: a curated benchmark for formally verifoed code generation [209] => clever: a curated benchmark for formally verif9ed code generation [210] => clever: a curated benchmark for formally verif8ed code generation [211] => clever: a curated benchmark for formally verifiwd code generation [212] => clever: a curated benchmark for formally verifisd code generation [213] => clever: a curated benchmark for formally verifidd code generation [214] => clever: a curated benchmark for formally verifird code generation [215] => clever: a curated benchmark for formally verifi4d code generation [216] => clever: a curated benchmark for formally verifi3d code generation [217] => clever: a curated benchmark for formally verifies code generation [218] => clever: a curated benchmark for formally verifiex code generation [219] => clever: a curated benchmark for formally verifiec code generation [220] => clever: a curated benchmark for formally verifief code generation [221] => clever: a curated benchmark for formally verifier code generation [222] => clever: a curated benchmark for formally verifiee code generation [223] => clever: a curated benchmark for formally verified xode generation [224] => clever: a curated benchmark for formally verified vode generation [225] => clever: a curated benchmark for formally verified fode generation [226] => clever: a curated benchmark for formally verified dode generation [227] => clever: a curated benchmark for formally verified dode generation [228] => clever: a curated benchmark for formally verified cide generation [229] => clever: a curated benchmark for formally verified ckde generation [230] => clever: a curated benchmark for formally verified clde generation [231] => clever: a curated benchmark for formally verified cpde generation [232] => clever: a curated benchmark for formally verified c0de generation [233] => clever: a curated benchmark for formally verified c9de generation [234] => clever: a curated benchmark for formally verified cose generation [235] => clever: a curated benchmark for formally verified coxe generation [236] => clever: a curated benchmark for formally verified coce generation [237] => clever: a curated benchmark for formally verified cofe generation [238] => clever: a curated benchmark for formally verified core generation [239] => clever: a curated benchmark for formally verified coee generation [240] => clever: a curated benchmark for formally verified codw generation [241] => clever: a curated benchmark for formally verified cods generation [242] => clever: a curated benchmark for formally verified codd generation [243] => clever: a curated benchmark for formally verified codr generation [244] => clever: a curated benchmark for formally verified cod4 generation [245] => clever: a curated benchmark for formally verified cod3 generation [246] => clever: a curated benchmark for formally verified code feneration [247] => clever: a curated benchmark for formally verified code veneration [248] => clever: a curated benchmark for formally verified code beneration [249] => clever: a curated benchmark for formally verified code heneration [250] => clever: a curated benchmark for formally verified code yeneration [251] => clever: a curated benchmark for formally verified code teneration [252] => clever: a curated benchmark for formally verified code gwneration [253] => clever: a curated benchmark for formally verified code gsneration [254] => clever: a curated benchmark for formally verified code gdneration [255] => clever: a curated benchmark for formally verified code grneration [256] => clever: a curated benchmark for formally verified code g4neration [257] => clever: a curated benchmark for formally verified code g3neration [258] => clever: a curated benchmark for formally verified code geberation [259] => clever: a curated benchmark for formally verified code gemeration [260] => clever: a curated benchmark for formally verified code gejeration [261] => clever: a curated benchmark for formally verified code geheration [262] => clever: a curated benchmark for formally verified code geheration [263] => clever: a curated benchmark for formally verified code genwration [264] => clever: a curated benchmark for formally verified code gensration [265] => clever: a curated benchmark for formally verified code gendration [266] => clever: a curated benchmark for formally verified code genrration [267] => clever: a curated benchmark for formally verified code gen4ration [268] => clever: a curated benchmark for formally verified code gen3ration [269] => clever: a curated benchmark for formally verified code geneeation [270] => clever: a curated benchmark for formally verified code genedation [271] => clever: a curated benchmark for formally verified code genefation [272] => clever: a curated benchmark for formally verified code genetation [273] => clever: a curated benchmark for formally verified code gene5ation [274] => clever: a curated benchmark for formally verified code gene4ation [275] => clever: a curated benchmark for formally verified code generztion [276] => clever: a curated benchmark for formally verified code generstion [277] => clever: a curated benchmark for formally verified code generwtion [278] => clever: a curated benchmark for formally verified code generqtion [279] => clever: a curated benchmark for formally verified code generarion [280] => clever: a curated benchmark for formally verified code generafion [281] => clever: a curated benchmark for formally verified code generagion [282] => clever: a curated benchmark for formally verified code generayion [283] => clever: a curated benchmark for formally verified code genera6ion [284] => clever: a curated benchmark for formally verified code genera5ion [285] => clever: a curated benchmark for formally verified code generatuon [286] => clever: a curated benchmark for formally verified code generatjon [287] => clever: a curated benchmark for formally verified code generatkon [288] => clever: a curated benchmark for formally verified code generatoon [289] => clever: a curated benchmark for formally verified code generat9on [290] => clever: a curated benchmark for formally verified code generat8on [291] => clever: a curated benchmark for formally verified code generatiin [292] => clever: a curated benchmark for formally verified code generatikn [293] => clever: a curated benchmark for formally verified code generatiln [294] => clever: a curated benchmark for formally verified code generatipn [295] => clever: a curated benchmark for formally verified code generati0n [296] => clever: a curated benchmark for formally verified code generati9n [297] => clever: a curated benchmark for formally verified code generatiob [298] => clever: a curated benchmark for formally verified code generatiom [299] => clever: a curated benchmark for formally verified code generatioj [300] => clever: a curated benchmark for formally verified code generatioh [301] => clever: a curated benchmark for formally verified code generatioh ) </h1> </body>