By Evan Vega
A new workshop paper argues that the training method behind today’s reasoning models, reinforcement learning with
verifiable rewards, does not teach a model new reasoning. “RLVR Sharpens the Pencil, Not the Mind” by Shreeyam Bangera
and Manya Gupta of IIT Roorkee (arXiv:2611.02574, posted 2 November 2026,
accepted at the MOSS workshop at COLM 2026) tracked Ai2’s open OLMo 2 1B model across 13 training snapshots on
grade-school maths. First-try accuracy rose from 58.0% to 61.3%, while a simple detector for planted arithmetic mistakes
stayed flat; the only internal change the authors found was at the final answer.
Novel Cognition checked the paper against its own tables and released code, re-ran its accuracy measurements on a
MacBook Pro with the authors’ own prompt and answer checker, and added the one model the paper leaves out: the one from
before training began. The full analysis is at rlvr.novcog.us.com.
A MOSS @ COLM 2026 workshop paper (arXiv:2611.02574) argues that reinforcement learning with verifiable rewards only sharpens what a model already knows: tracking OLMo 2 1B across 13 training snapshot
For teams that train, tune or buy reasoning models, the practical reading is to measure reach as well as first-try
accuracy, before and after training, and to price a training run against asking the existing model several times and
checking the answers. Checkers built from a model’s internal activity have held up in other studies for triage: stopping
early (about 24% fewer tokens), filtering candidates before an expensive check (about half the verification cost) and
flagging answers to hold back. They have failed at picking the best of several answers and at finding the step that went
wrong, and one study found a planted-mistake detector said nothing about whether a final answer was right
(what to do with it). The experiments the paper sets up, starting
with freezing a base-model checker and scoring every later snapshot, are at
what to test next.
The paper’s tables hold up. Its abstract says more than they do. The rise in “solved at least once in 256 tries”
(94.5% to 96.9%) is about three problems out of 128, measured from training step 200, so it cannot show what the model
knew before training, though the abstract calls it proof. The detector range quoted “across all 13 checkpoints and 3
error types” covers one error type at 13 snapshots; the other two were measured at five. The paper’s most useful practical
claim, that a mistake-checker built on the base model should keep working after training and save verification compute,
was never tested: every detector was retrained at each snapshot. Its own introduction scopes the work as “a single-model,
single-task (arithmetic) result” (the paper, checked).
The study joins a live argument. Yue et al. (2025) found trained models win at one try while their untrained versions
win at hundreds; Akgül et al. (2026) found reward training changes only 1 to 3 percent of the words a model writes;
NVIDIA’s ProRL found longer training reached new strategies. Bansal, Kakade and colleagues (June 2026) reported that “the
sharpening effect reported in recent work arises only when RL follows SFT”, which is the set-up this paper studies
(the debate).
The paper’s first measurement is reward-training step 200. Ai2 publishes the model from before that training began,
so Novel Cognition ran it alongside the paper’s first and last snapshots, using the paper’s own prompt, answer checker and
300-token limit on all 1,319 GSM8K test problems, one answer each, on a MacBook Pro. Before training the model answered
52.2% correctly; at step 200, 62.5%; at step 2,600, 62.4%. With one answer each, the whole gain lands before
the paper starts measuring: between steps 200 and 2,600, 95 problems went from wrong to right and 97 from right to
wrong. It is not a formatting effect: before training the model already finished 99.7% of its answers inside the
limit, and accuracy among finished answers rose from 52.3% to 63.7%. Sampled the paper’s way (32 answers each on its
200 problems), averaged one-try accuracy went 44.4%, 57.7% and 60.7%, close to the paper’s 58.0% and 61.3% for the last two, so most of the gain again arrives before step 200; problems solved at least once went 176, 183 and
181 of 200 (our re-run,
method). It is a spot check on one machine, not a replication.
Novel Cognition’s reading: a careful small study whose title says more than it found. Reward training after the usual
supervised training does look a lot like sharpening on easy maths, and the paper adds a view from inside the model across
a whole run. It does not show that reward training cannot teach anything, and its clock starts after training has begun.
The analysis is also an episode of the Frontier Watch
podcast.
Novel Cognition tests research and vendor claims like these for companies as part-time AI leadership
(novcog.com/contact). The checks behind every published number are at
hiddenstatedrift.com/verify. Novel Cognition is not affiliated with the
authors, IIT Roorkee, Ai2 or COLM.
Part of the Frontier Watch Series: Read the previous investigation
More Coverage:
→ Read this investigation on North Denver Tribune
→ Coverage from Daily Colorado News
