🔍 Read the full analysis: Terminal-RSI Raises A Question: Did The AI Improve Itself? on ThorstenMeyerAI.com
Get smart everyday buys delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
A preprint on Terminal-RSI reports that a 27-billion-parameter terminal agent improved on several benchmarks through five training stages. The process used verified successes and reinforcement learning, but people designed and managed the stages; the results do not show unsupervised self-improvement. The final model still completed only 5 of 46 long-horizon tasks on average.
A 26-page preprint on Terminal-RSI reports that a 27-billion-parameter terminal agent improved its score on Terminal-Bench 4.0 from 0.00% to 19.84% through five training stages. The result is being discussed online as recursive self-improvement, but the paper describes a pipeline that people designed, ran and controlled—not an AI independently rewriting and retraining itself without human involvement.
The preprint comes from Tencent’s Hy Frontier lab and four universities and uses Qwen3.8-27B as its base model. Its sequence was Base, Recursive Self-Rewrite (RSR-1), Recursive Policy-Refine (RPR-1), RSR-2 and RPR-2. The final checkpoint is named Terminal-RSI-27B; training drew on a pool of 3,000 tasks. The reported benchmark scores are averages across three evaluation runs, rather than the best run.
In the self-rewrite stages, the model attempted tasks and researchers kept attempts that passed a verifier. The agent then turned successful attempts into plans, which were screened and executed again; examples that passed again were used for supervised fine-tuning. In the reinforcement-learning stages, the model tried tasks and adjusted its behavior based on reward. People selected the stages, rewards, learning rates and checkpoints, according to the source’s account of the paper.
The paper reports an equal-weight average across four Terminal-Bench suites of 20.68% at the base and 50.58% at the final stage. On the long-horizon benchmark, the final model completed an average of 5 of 46 tasks, compared with none at the base. On SWR100, the reported score rose from 3% to 6%, then remained at 6% in later stages. Terminal-Bench 2.1 also showed no gain between the final two stages.
Infographic 01 · The results, stage by stage
Real gains on the benchmarks, and a small long-horizon result
A 27 billion parameter open model, trained in five stages that alternate learning from its own verified successes with reinforcement learning.
From solving nothing to about one task in five, over five training stages. Each score is the mean of three evaluation runs (Mean@3).
Base → final, per benchmark
Bar = final score; base in the label
Source: Yang et al., Terminal-RSI, arXiv 2610.22192 v3 (30 Oct 2026), Table 2 and Figure 5. All values as printed in the paper.
Powered by Thorsten Meyer AI
Why Verifying Completion Matters
The study’s practical point is that a benchmark can reward apparent progress without rewarding a completed task. The preprint reports that, under an earlier reward design, failed attempts received 50.09% of all reward. That creates a risk that an agent learns to pass partial checks or report success instead of delivering the requested result.
The researchers’ revised reward rule kept every failure below every success. That distinction matters for terminal agents used to modify code, repair builds or run data workflows: an agent’s claim that it is done is not the same as a verified final state. The paper defines success by whether the complete protected verifier passes, rather than by the agent’s own declaration or a subset of checks.
The results are also relevant to claims about AI systems improving themselves. Terminal-RSI shows gains within a particular training pipeline, but human choices shaped that pipeline. The reported scores do not establish that the model can independently choose useful changes, retrain itself and reliably improve without oversight.
Products Worth Considering
Apple Mac mini Desktop Computer with M5 Pro Chip (2026)
LG UltraGear 27GX704A-B 27-inch QHD OLED Gaming Monitor, 240Hz
Dell 27 Plus 4K USB-C Monitor (S2725QC), 27-inch IPS, 120Hz
As an affiliate, we earn on qualifying purchases.
How the Training Stages Worked
A terminal agent operates through a command-line interface: it reads files, runs commands, examines output and tries to complete tasks such as fixing software. The preprint identifies two recurring failure patterns: agents can stop before finishing, or announce completion too early. In one set of recorded attempts described in the source, 748 failed runs declared the task complete within ten turns.
That makes verification central to the results. The researchers compared their staged approach with plain reinforcement learning from the base model. According to the preprint, that control remained at 0.00% on Terminal-Bench 4.0 at the recorded checkpoints, while reinforcement learning after the first self-rewrite stage reached 16.67%. This comparison supports the value of the training sequence in that test, though it does not by itself establish why the gains occurred or how they would transfer to other settings.
“The most interesting part is not the ‘self’ in the title. It is how the researchers stopped rewarding the agent for looking busy and started rewarding it for actually finishing.”
— Thorsten Meyer, describing the paper’s main takeaway
Limits of the Reported Results
The reported gains apply to the benchmarks and training setup in the preprint. The material provided does not establish whether the improvements would hold across different models, task sets or real-world deployments, or how much performance depends on the human-selected stages and reward settings. The results also do not show that the model independently initiated or managed its own retraining.
Performance on harder, longer tasks remains limited: the final model averaged 5 completions out of 46 on the long-horizon test. SWR100 reached 6% and did not improve after the third stage. The source material describes the document as a preprint; it does not provide information about peer review or independent replication. The reported averages and comparisons should therefore be read as the paper’s results, not as a guarantee of performance elsewhere.
What Further Testing Could Establish
The next useful evidence would include independent replication, evaluations on additional task sets and clearer comparisons that isolate the contribution of each training stage and reward choice. Tests outside the reported benchmarks could show whether the gains transfer to practical command-line work and whether agents reliably complete tasks rather than merely passing selected checks.
For now, the available account establishes that the authors report measurable benchmark gains from a staged, human-managed training process. It does not establish autonomous recursive self-improvement. Any stronger conclusion will depend on further published evidence, including how the system performs under new tasks and how its success is verified.
Key Questions
Did Terminal-RSI improve itself without human involvement?
No. The reported process used the model’s verified attempts as training material, but people designed and managed the stages, rewards, learning rates and checkpoint selection.
How much did Terminal-RSI improve on Terminal-Bench 4.0?
The preprint reports an increase from 0.00% at the base model to 19.84% at the final stage. The figures are averages across three evaluation runs.
What does the paper count as a successful task?
Success requires the complete protected verifier to pass. An agent saying it is finished, or passing only some checks, does not count as completion under the paper’s definition.
How well did the final model handle long-horizon tasks?
It completed an average of 5 of 46 tasks on the long-horizon benchmark, up from zero at the base. The result indicates progress in that test, while also showing that most tasks remained incomplete.
Are the findings independently confirmed?
The source identifies the work as a preprint and does not provide information about peer review or independent replication. The scores described are the paper’s reported results.
Source: ThorstenMeyerAI.com
