Numerai Benchmark Models: The Bar You Need to Clear

Across its scored history, Numerai's benchmark_blender posts negative MMC in roughly two of every three rounds, and its cumulative MMC sits near -0.3. Yet its round-by-round percentile rank still swings from single digits to the mid-80s. "Beat the benchmark" is not a fixed bar; it's a moving regime indicator. If you can't beat it consistently, the issue may be that you're calibrating against a target that itself drifts 80 percentile points across a year.

This article looks at how benchmark_blender actually performs, where it ranks against the staked field, and whether beating it is getting harder. For background on the tournament mechanics, see How Numerai Works. To see where models currently stand, check the live leaderboard.

What Are Benchmark Models?

Benchmark models are Numerai's own submissions, trained on public data using documented methods. They provide a performance floor for new participants, feed into the meta-model's diversity, and set a transparent standard for what "good enough" looks like.

The benchmark_blender is the most-watched variant. It blends predictions from several example models into a single submission: competent but not exceptional, and designed to be beatable by anyone doing real feature engineering, target selection, or modeling work.

Cumulative Signal

Benchmark models do not stake NMR, so payout is not the right yardstick: the meaningful question is how much MMC they accumulate over time. The two series cover different eras. benchmark_models_te's data run spans roughly rounds 640-780: it climbs steadily, peaks near +0.5 cumulative MMC around round 765, and ends near +0.3. benchmark_blender's scored history only begins around round 1185, and it has spent that entire run underwater, drifting to roughly -0.3 cumulative MMC by round 1240.

The contrast is the story: the earlier ensemble compounded in its era, while today's blender has yet to string together a positive cumulative stretch. Common-signal ensembles tend to win in low-information regimes (where averaging beats searching) and lose in regimes that reward differentiated residual signal, and recent rounds have sat firmly in the second category.

Where Does the Benchmark Sit?

The benchmark's percentile rank tells you how hard it is to beat. A 30th-percentile benchmark is easy: most participants already clear it. A 70th-percentile benchmark means beating it requires outperforming the majority.

Benchmark_blender's MMC percentile is anything but stable. Across roughly the last 50 rounds, it has swung from single digits to the mid-80s. It peaked near the 85th percentile around round 1212 (a stretch where the ensemble was genuinely hard to beat), then collapsed to single digits through the mid-1220s before recovering into the 20s and 30s. See our MMC vs correlation primer if that metric is new to you.

"The benchmark" is not a fixed difficulty level. Whether your model beats it in any given week depends as much on regime as on skill.

Benchmark vs the Field

How does benchmark_blender stack up against the field median on raw MMC?

Both lines spend substantial time below zero in this window, though benchmark_blender briefly rises above the field median during the middle stretch. Field-wide MMC has been running between about -0.02 and +0.01 over this window. Benchmark and median track each other closely because both are exposed to the same data and market regime. Green bands mark rounds where the benchmark beat the median; red bands mark underperformance. Neither wins consistently, though the benchmark has skewed below the median in the most recent rounds.

The tight coupling is expected. The benchmark trains on the same features everyone has, processed with standard methods, so it captures common signal without adding proprietary insight.

Can You Beat It?

What fraction of staked models actually beat the benchmark each round?

The 10-round rolling average ranges from about 30% to nearly 90%. The benchmark was hardest to beat around round 1212, when only ~30% of staked models cleared it. By the most recent rounds, the share has climbed back above 80%: the field is comfortably outperforming benchmark_blender again.

Rounds where the benchmark is hardest to beat are rounds where its conservative ensemble lines up unusually well with the scoring regime. When that happens, a public baseline can punch above its normal weight. When the regime rewards differentiated residual signal, custom models pull ahead.

Takeaways

The benchmark is a moving target, not a fixed median. Its percentile rank has swung from single digits to the mid-80s across its scored rounds, an 80-point range. Judge your model over at least 20-30 rounds, not one.

Benchmark performance is sticky across rounds. When benchmark_blender finishes below the field median for two rounds in a row, the historical probability of a third sub-median round is 77% (24 of 31 instances). Two bad rounds usually presage a third, so do not panic-rebuild on a single hot or cold streak.

Use the benchmark for calibration, not competition. Marginally beating benchmark_blender means you've cleared a baseline that itself runs negative-MMC two-thirds of the time. The goal is unique signal that improves the meta-model, not edging out the floor.

There is still room above the benchmark. Across the most recent 15 rounds, roughly 70-90% of staked models beat benchmark_blender each round, but the share rotates hard with regime, dipping below 20% in single rounds near the 1212 trough. A median-of-field submission clears it more often than not.