zgcagi/Harness-RSI

Companion materials for our survey of how execution experience becomes persistent harness state under a fixed base model. We organize update mechanisms, examine evolutionary search and continual PAC learning, assess reported evidence, and discuss applications and societal effects. The repository PDF is the 12 October 2026 working draft. Figures retain the manuscript layout; click to view them at full resolution.

Background　Related Surveys · Definition · Taxonomy · Precursors

Literature　Interpreted · Executable · Theory · Evaluation

Resources　Applications & Outlook · Reported Evidence · Contributing · Citation

• A Survey of Self-Evolving Agents TMLR'26 — organizes self-evolution by what, when, how, and where to evolve. [paper]
• A Comprehensive Survey of Self-Evolving AI Agents arXiv'25 — organizes methods by optimizer feedback. [paper] [list]
• Externalization in LLM Agents arXiv'26 — unified review of memory, skills, protocols, and harness engineering. [paper]
• Code as Agent Harness arXiv'26 — organizes by the role of code in the agent loop. [paper] [list]
• Recursive Self-Improvement in AI arXiv'26 — from bounded self-refinement to autonomous research loops, organized by loop closure. [paper]
• Self-Improvements in Modern Agentic Systems arXiv'26 — separates model and scaffold updates under frozen parameters and discusses independent evaluation and component controls. [paper] [list]
• Self-Evolving Coding Agents arXiv'26 — distinguishes proposal eligibility, runtime activation, and admission, and discusses component controls under a common budget. [paper]

A harness $H$ is the software and configuration that governs how a model $M$ operates in an environment $E$. It includes tools, context construction, memory, and execution control. The final evaluator $J$ scores a trajectory and stays outside the candidate’s editable surface; internal critics and provisional verifiers may belong to the mutable harness.

$$ \tau \sim \mathrm{Run}(M,H,E), \qquad y=J(\tau) $$

Harness self-evolution uses a batch of trajectories $\mathcal{B}_k$ to revise the state used by later runs:

$$ H_{k+1}=\mathrm{Update}(H_k,\mathcal{B}_k) $$

The update mechanism remains available for further revisions, and an accepted state is saved and loaded by a later run. Rejected candidates leave the harness unchanged. The analysis holds $M$ fixed; systems that also train weights are read here for their harness-side changes. Offline optimizers and mechanisms with limited persistence are marked in the entries.

Self-evolution describes this update process whether or not the accepted state performs better; self-improvement is a beneficial change supported by an explicit comparison. Harness RSI (recursive self-improvement) remains a conventional short name for the area, not a guarantee that every accepted revision helps. Whether an update operated and improved performance requires the evaluation evidence below.

Interpreted updates revise content read by a fixed host; executable updates revise the implementations it invokes. Evaluation forms a separate branch, while theory cuts across update targets.

Reference numbers follow the manuscript; source links appear in the reading list below. See also the taxonomy data.

• Speculations Concerning the First Ultraintelligent Machine Adv. Computers'65 — frames recursive improvement as a machine designing a better successor to itself. [paper]
• EURISKO Artif. Intell.'83 — learned heuristics, including heuristics that modify other heuristics, shape later search. [paper]
• Gödel Machines arXiv'03 — a self-rewrite executes only after the system proves the rewrite improves expected utility. [paper]

These methods revise instructions, memory, or coordination specifications read by fixed components. Entries note offline optimization and task-local adaptation where applicable.

Use candidate scores to propose and select prompt revisions.

• APE ICLR'23 — proposes instructions from input–output demonstrations and keeps the highest-scoring candidate. [paper] [project]
• OPRO ICLR'24 — feeds earlier instructions and scores to an optimizer model that proposes the next candidate. [paper] [code]

Translate errors, critiques, or execution traces into edits to text parameters.

• ProTeGi EMNLP'23 — converts task errors into natural-language "gradients" and beam-searches over the edited prompts. [paper]
• TextGrad Nature'25 — backpropagates textual critiques through a graph of LLM calls to edit any text variable in the system. [paper]
• GEPA ICLR'26 Oral — uses execution-trace reflection to revise prompts and Pareto selection to retain candidates. [paper]
• Trace NeurIPS'24 — treats the execution trace as a computation graph and generates edits from rich feedback through a generative optimizer. [paper] [project]
• metaTextGrad arXiv'25 — optimizes the optimizer's own prompts, accepting an update only when a validation split improves. [paper]
• EGO-Prompt arXiv'25 — co-adapts the prompt and a semantic causal graph for domain deployments. [paper]
• SePO arXiv'26 — rewrites persistent system prompts using accumulated task feedback. [paper] [code]

Search over the instructions that generate later mutations.

• Promptbreeder ICML'24 — co-evolves task prompts and the mutation prompts that generate them. [paper]
• DelvePO arXiv'25 — decouples prompt components and steers mutation along identified directions. [paper]
• DEEVO arXiv'25 — evolves instructions through structured debates, ranking candidates by Elo tournament. [paper]

Distill execution trajectories into knowledge retrieved by later runs.

• Reflexion NeurIPS'23 — stores verbal self-reflection on a failure and reloads it for another attempt at the same task. [paper]
• ExpeL AAAI'24 — extracts cross-task insights from a training pool of trajectories, with add/edit/upvote/downvote curation. [paper]
• AWM ICML'25 — induces recurring sub-goal workflows from trajectories, offline from a training split or online from the task stream. [paper]
• Agent KB arXiv'25 — a cross-domain experience base whose top-k retrieved items are injected into later runs. [paper]
• ReasoningBank ICLR'26 — consolidates reasoning strategies from successes and failures into a bank injected into the system instruction. [paper]
• EvoRepair arXiv'26 — accumulates, scores, and reuses vulnerability-repair experience across long-horizon repairs. [paper]
• SWE-Exp arXiv'25 — distills issue-resolution experience from prior search trajectories into a bank queried on new issues. [paper] [code]
• MemCoder arXiv'26 — structured memory of repository knowledge and repair outcomes that grows alongside the codebase. [paper]
• Memory Transfer arXiv'26 — measures how memories formed in one domain transfer to another, across four memory formats. [paper]

Store an explicit procedure that can guide later tasks.

• Trace2Skill arXiv'26 — distills execution lessons into skill patches and aggregates them into reusable skill documents; validation-based patch selection is a separate experiment. [paper]
• SkillClaw arXiv'26 — an agentic evolver whose validator admits a candidate skill only if it beats the deployed pool on validation tasks. [paper]
• CodeSkill arXiv'26 — learns self-evolving coding skills through preparatory passes that build the bank before deployment. [paper]
• GSkill ACM AIAS'26 — automatically learns SKILL.md files for coding agents from prior runs. [paper]
• GSE arXiv'26 — learns globally reusable skills gated by local and replay-driven validation under one-project-held-out splits. [paper]
• MUSE-Autoskill arXiv'26 — couples skill creation, memory, and management, registering a skill only once it passes its own generated tests. [paper]
• MemP ACL'26 Findings — distills procedural memory from trajectories and compares update strategies and abstraction granularities. [paper]
• Skill Consumption arXiv'26 — examines how generated skills affect downstream behavior, including changes to default strategies and explicit use. [paper]
• SkillForge SIGIR'26 Industry — learns reusable skill documents from execution experience and reserves a later temporal partition for reporting. [paper]

Revise, compress, or replace stored content through a validation or maintenance rule.

• Dynamic Cheatsheet EACL'26 — maintains an adaptive memo of strategies curated at test time. [paper]
• ACE ICLR'26 — grows and consolidates an evolving context playbook through incremental delta updates instead of rewrites. [paper]
• SkillOpt arXiv'26 — evaluates a revised skill set on validation before it replaces the active library. [paper]
• SkillOpt-Lite arXiv'26 — combines a validation deadband with compilation and smoke tests to admit skill edits at lower cost. [paper] [code]
• Janus arXiv'26 — wraps an updater and compares old and candidate memories on coverage, boundary, and fresh tasks when a proposal deviates from the recent update direction. [paper]
• VALVE arXiv'26 — admits skill revisions through paired comparisons on a validation panel and tracks drawdown over a long task stream. [paper]
• TACO arXiv'26 — compresses observations for terminal agents through a write-back gate and a reward-free retention stopping rule. [paper] [code]
• SkillMOO arXiv'26 — uses NSGA-II ranking to select skill combinations under competing objectives. [paper]
• EffiSkill arXiv'26 — mines efficiency skills from verified code pairs, ranking candidates on a public split and reporting on a private one. [paper]
• VASO arXiv'26 — formally verifiable skills: each plan is model-checked against the skill's specification before it may run. [paper]
• Memory Management ACL'26 — empirical study of how retention, deletion, and retrieval rules change task outcomes, with a similarity-based adherence measure. [paper]
• MCE ICML'26 — applies history-informed (1+1) evolution to skills with agentic crossover and a validation archive. [paper]
• Living-Harness arXiv'26 — an interactive-agent evolver that consolidates retrieved memory into a reusable graph across cycles. [paper] [code]

Optimize prompts and demonstrations while keeping the pipeline structure fixed.

• DSPy ICLR'24 — compiles declarative LM pipelines, optimizing prompts and demonstrations while control flow stays fixed. [paper] [code]
• MIPRO EMNLP'24 — jointly optimizes instructions and demonstrations across the stages of a multi-stage LM program. [paper] [project]

Search over connections and coordination among fixed operators.

• GPTSwarm ICML'24 — makes edges between fixed operations learnable and optimizes the resulting communication graph. [paper] [code]
• Mass ICLR'26 — jointly searches agent prompts and communication topologies over predefined multi-agent blocks. [paper]
• MaAS ICML'25 — samples query-dependent architectures from an agentic supernet over predefined operators. [paper]
• AFlow ICLR'25 — serializes workflows as operator-composing code and searches structure and prompts over it; the operators themselves stay fixed. [paper] [code]
• AgentSquare ICLR'25 — searches module compositions in a modular design space while also generating new module implementations. [paper] [code]

Use execution feedback to revise the workflow specification.

• SEW arXiv'25 — jointly mutates prompts and topology while comparing alternative workflow representations. [paper]
• JudgeFlow arXiv'26 — uses block-level judge diagnoses to revise sequence, loop, and conditional blocks. [paper]
• RHI arXiv'26 — applies pairwise feedback to roles, instructions, communication contracts, and interaction hops within one task episode. [paper]
• Meta-Team arXiv'26 — carries experience across tasks to update a reusable scaffold of agent prompts, teammate profiles, collaboration notes, and shared coordination rules. [paper]
• Symbolic Learning AI Open'25 — jointly revises pipeline structure, prompts, and tools through back-propagated language loss. [paper]

These methods revise tools, component implementations, control programs, or plugins invoked by the host.

Generate reusable programs exposed through a tool interface.

• Voyager TMLR'24 — turns self-verified programs into a growing skill library retrieved and composed in later tasks. [paper] [project]
• PolySkill arXiv'25 — induces polymorphic skill abstractions so a learned skill generalizes past its originating site. [paper] [code]
• FlowEvo arXiv'26 — compiles a successful workflow into a validated callable artifact in a persistent skill bank, then either executes it or describes it to workflow generation. [paper] [code]
• Live-SWE-agent arXiv'25 — synthesizes Bash and Python tools during a task; the tools are discarded with its container. [paper]
• RoboPhD arXiv'26 — co-evolves a deterministic Python analysis tool with model-readable SQL instructions, then ships the pair without its evolutionary infrastructure. [paper]

Modify a declared executable component while the enclosing executor stays fixed.

• Self-Harness arXiv'26 — proposes edits from the incumbent and admits candidates that do not regress on either split and improve on at least one. [paper] [code]
• HarnessCompass arXiv'26 — optimizes harness components separately and combines them into candidate configurations. [paper]
• AHE arXiv'26 — uses submit-guard and shell traces to diagnose component edits and supports file-level rollback. [paper]
• HarnessFix arXiv'26 — diagnoses harness faults from failed trajectories and admits a repair only through a validation-set assessment. [paper]
• ReCreate ACL'26 — edits role, process, tool, and memory components at text and code level using a small development set. [paper]
• AutoSaddler arXiv'26 — retains patches that improve a minibatch, selects on development data, and reports on a group-disjoint test split. [paper]
• CHASE arXiv'26 — tests proposed patches against protocol transformations designed to invalidate their apparent gains. [paper]
• Evo-Harness arXiv'26 — compiles accumulated context into harness-level skills along an online stream, matching fail→pass traces to skill content. [paper] [code]
• HarnessBank arXiv'26 — archives component patches by semantic descriptor, checks activation beacons during screening, and keeps evaluation code in an immutable kernel. [paper]
• Ecdysis arXiv'26 — trains runtime harnesses with promotion on a training objective, then scores the frozen harness on a held-out split. [paper]
• MemEvolve arXiv'25 — rewrites the encode, store, retrieve, and manage implementations behind a fixed memory-provider interface. [paper]
• GUI-HARVEST arXiv'26 — turns recurring failures from repeated GUI executions into bounded source edits with stated behavioral predictions. [paper]
• Harness Ceiling arXiv'26 — compares harness edits on long-horizon planning tasks to identify failures that remain for weight training. [paper]

Search over the control program; VERSE also revises the optimizer that proposes it.

• ADAS ICLR'25 — a fixed meta-agent writes new agent forward implementations and keeps evaluated candidates in an archive. [paper] [project]
• Meta-Harness COLM'26 — uses an external coding agent to generate complete harness programs from earlier candidates’ source, scores, and traces. [paper]
• RRSI arXiv'26 — regularizes program search using edit budgets, leakage checks, and noise-adjusted admission thresholds. [paper] [code]
• VERSE arXiv'26 — revises both the executor harness and the optimizer’s prompts, tools, hooks, and notes under execution checks. [paper]

Edit a versioned harness implementation and retain candidates for further evolution.

• SICA arXiv'25 — lets the best archived agent revise its own multi-file Python codebase, including tools, subagents, and control flow. [paper]
• DGM ICLR'26 — branches repository variants into an open-ended archive, with parent selection over performance and novelty. [paper]
• DarwinX arXiv'26 — evolves repository-level harness candidates through selection, mutation, and recombination. [paper]
• HGM arXiv'25 — approximates clade-level productivity and allocates lineage-expansion and evaluation budgets by Thompson sampling. [paper] [code]
• Group-Evolving Agents arXiv'26 — lets selected parents share code patches, tools, and execution experience before producing framework-level patches. [paper]
• MOSS arXiv'26 — rewrites routing, hooks, state management, and dispatch code, then swaps the container behind a health check plus human authorization. [paper] [code]
• Ouroboros arXiv'26 — evolves its own core through reviewed commits, with run manifests, seed attestation, and public per-task traces. [paper]
• AgentDevel arXiv'26 — uses versioned diffs and release-candidate regression tests, recording which tasks each update fixes or breaks. [paper]
• EvolveNet arXiv'26 — combines harness edits across clients and validates merged patches before broadcasting shared state. [paper]

Apply changes to a running process; persistence depends on the system’s save and reload mechanism.

• Gödel Agent ACL'25 — monkey-patches decision methods, action spaces, and modification routines in Python memory; edits last for the process lifetime. [paper]
• TTHE arXiv'26 — evolves executable harness programs from traces at test time and carries the selected program into subsequent inputs. [paper]
• Life-Harness arXiv'26 — adapts environment contracts, action realization, and trajectory regulation together, then freezes the runtime interface for evaluation. [paper] [code]

Bind updates to hooks with explicit interfaces and admission checks.

• HarnessX arXiv'26 — checks hook types, singleton constraints, and smoke tests before acceptance; each edit declares an expected trace feature. Replay, novelty, and structural checks do not guarantee non-decreasing task scores. [paper]
• Argus arXiv'26 — exposes skills through a general agent runtime and evolves them along long task streams. [paper]

Keep a small host fixed and expose replaceable components through plugin interfaces.

• DeepSeek Harness Product'26 — provides a Cordis-based runtime with declared dependencies, checkpointing, and replay; the product page reports no empirical self-evolution evaluation. [page]
• Autogenesis arXiv'26 — a self-evolving agent protocol with an explicit propose–evaluate–commit cycle over lineage and versioning. [paper] [code]
• ModularRSI arXiv'26 — evolves modules behind a composer and admits updates through a reward-blind gate on externally curated tasks. [paper]
• Cordis / Spatiotemporal Composability arXiv'26 — the formal calculus behind that kernel; self-evolving harnesses are stated as future work. [paper]

The paper develops two perspectives: optimization and generalization. Many reviewed methods search discrete harness state without gradients; scores support selection while traces and critiques guide proposals. Evolutionary paradigms describe shared search structures, and continual PAC analysis studies generalization, retention, and candidate admission.

The paper maps variation, selection, archive retention, and budget allocation onto EA paradigms. A method may appear in several rows; each mapping states its scope.

The continual PAC analysis separates sensitivity to training samples, retention of earlier capabilities, and admission of new candidates. It assumes independent exogenous task draws, task losses in $[0,1]$, and a fixed model and evaluator.

For at most $K$ candidates in each of $T$ rounds, compare each candidate with the incumbent on $m$ fresh tasks independent of its construction:

$$ \widehat{\Delta}_k(H') + \sqrt{\frac{2\ln(KT/\delta)}{m}} < 0 $$

Here $\widehat{\Delta}$ is candidate loss minus incumbent loss. Under the stated assumptions, this gate gives joint confidence $1-\delta$ for accepted candidates on their round’s task mixture. It applies to the candidate actually tested, including any merged edits. Repeated rollouts reduce execution noise but do not add independent tasks.

• Learnability, Stability and Uniform Convergence JMLR'10 — relates an algorithm's sensitivity to individual observations to its generalization gap. [paper]
• Memory Bounds for Continual Learning FOCS'22 — lower bounds on the memory a continual learner needs, motivating generalization and retention as one joint question. [paper]

• Hoeffding Bounds JASA'63 — bounds deviations of bounded empirical losses for paired candidate admission. [paper]
• Selection Overfitting JMLR'10 — analyzes overfitting in model selection and its bias in reported performance. [paper]
• The Reusable Holdout Science'15 — preserves validity of a holdout reused across adaptive, data-dependent decisions. [paper] · [companion]
• SkillForge SIGIR'26 Industry — learns reusable skill documents from execution experience and reserves a later temporal partition for reporting. [paper]

• Harness Evolution as Learning arXiv'26 — approximation, generalization, and optimization limits of self-improving personal agents. [paper]
• A Theory of Reliable Self-Evolution arXiv'26 — derives adoption conditions whose ceiling depends on verification and evaluation cost. [paper]

• The Ladder ICML'15 — a leaderboard mechanism that stays reliable under repeated adaptive submissions. [paper]
• Train Faster, Generalize Better ICML'16 — stability of SGD as a route to generalization bounds. [paper]
• PAC-Bayes for Prompts arXiv'23 — prompt engineering may not require rethinking generalization. [paper]
• ZOPO arXiv'24 — localized zeroth-order prompt optimization. [paper]
• Statistical Limits TMLR'26 — statistical limits of self-improving agents. [paper]
• AdaEvolve arXiv'26 — adaptive LLM-driven zeroth-order optimization. [paper]
• GAMBLe arXiv'26 — an analytical framework for AI-driven research systems. [paper]
• Mutation Without Variation GECCO'26 — convergence dynamics in LLM-driven program evolution. [paper] [code]
• What Fits Doesn't Overfit arXiv'26 — compression and generalization in ML research agents. [paper]
• IMABO arXiv'26 — bandit hyperparameter optimization at inference time. [paper] [code]

Read each result through Selection, Activation, Attribution, and Improvement, using the same comparison for all four claims. The evidence required depends on the conclusion: comparing one final state does not require tracing every accepted component. Selection may be addressed by independent reporting evidence or inference that accounts for adaptive selection.

Activation distinguishes an artifact being available, its changed path being reached, and its effect being carried into subsequent execution. See the criteria for evidentiary requirements and coverage.

Task distance describes how reporting tasks relate to the tasks that formed the update; it is not a numerical distance metric. The Improvement verdict grades this distance only, not a reliable positive effect, baseline strength, or persistence across rounds.

Separate reporting evidence from the update process and protect the scorer from candidate edits.

• RSEA arXiv'26 — assigns separate data to proposal, admission, and reporting. [paper]
• Anytime Certificates arXiv'26 — anytime-valid certificates for repeated, data-dependent admission decisions. [paper]
• HarnessOpt-Bench arXiv'26 — keeps its test partition out of reach for the whole search, with the boundary enforced by the runtime, and meters the resources the target agent spends. [paper]
• Phantom Guardrails arXiv'26 — documents self-improving harnesses that fix failures which never occurred, the patch grounded in no observed execution. [paper]
• Harness Tampering arXiv'26 — examines candidates that modify verification logic rather than the behavior under test. [paper]
• Self-Authored Verification arXiv'26 — finds self-written checks unreliable as admission evidence in heuristic self-improving agents. [paper]
• SWE-bench+ arXiv'24 — documents solution leakage and weak assertions that corrupt promotion and reporting together. [paper]

Record whether the accepted artifact operated in the runs that produced the result.

• SkillsBench arXiv'26 — records whether task-provided skills are read or referenced, across diverse tasks. [paper]
• Harness Updating ≠ Harness Benefit arXiv'26 — measures skill loading separately from trajectory adherence to identify updates that are available but not followed. [paper] [code]
• Skill Revisions arXiv'26 — tests whether revisions to a skill change coding-agent behavior when the skill is loaded on demand. [paper]

Separate the effect of the update from execution variation, inference compute, and evaluator changes.

• Agents That Matter TMLR'25 — shows agent comparisons confound capability with cost, and asks for cost-controlled evaluation. [paper]
• Token Cost arXiv'26 — analyzes and predicts token consumption in agent runs. [paper]
• MLE-bench ICLR'25 — ML-engineering tasks with repeated runs and an explicit compute envelope. [paper]
• HAL ICLR'26 — standardized run records, model snapshots, and cost accounting as shared evaluation infrastructure. [paper]
• Claw-Eval arXiv'26 — scores trajectories and repeats trials to estimate execution variation. [paper]
• AgentRewardBench COLM'25 — meta-evaluates automatic judges of web-agent trajectories against expert annotation. [paper] [project]
• Judge Inflation arXiv'24 — documents spontaneous reward hacking in iterative self-refinement. [paper]
• Memory Reward Inflation arXiv'26 — shows stored memory can inflate the reported reward rather than improve the behavior. [paper]
• Who Grades the Grader arXiv'26 — examines evaluator drift when metrics and skills evolve together. [paper] [code]
• Don't Blame the LLM arXiv'26 — attributes measured coding-agent differences to harness evolution rather than to the model. [paper]
• Scaffold Effect arXiv'26 — treats harness choice as a hidden variable in coding-agent comparisons. [paper]
• Rethinking Harness Evaluation arXiv'26 — examines protocol confounds in reported harness-evolution gains. [paper]
• Model–Harness Fit arXiv'26 — compares model–harness combinations across agent tasks to identify interaction effects. [paper]

Test on held-out tasks, new settings, or earlier distributions.

• HarnessDev arXiv'26 — separates harness creation from iterative evolution, reporting capability, cost, and held-out transfer. [paper]
• GDPevo arXiv'26 — evaluates evolved harnesses on held-out business tasks constructed by recombining rules from training tasks. [paper] [code]
• EvoHarnessBench arXiv'26 — grows the harness an agent is given and measures whether earlier competence is retained as capabilities are added. [paper]
• PATH-Bench arXiv'26 — path-dependent evaluation: deliberately varied task order and retained frozen reference histories. [paper]
• Agent Plasticity arXiv'26 — measures how execution experience translates into held-out gains and examines failures to reuse artifacts. [paper]
• Skill Generalization arXiv'26 — compares evolved skills on training and held-out tasks to measure how much improvement transfers. [paper]

Track which stored artifacts remain reachable and used after later updates.

• Ratchet arXiv'26 — asks how reliable an LLM judge must be before a skill may be retired. [paper] [code]
• SkillsVote arXiv'26 — lifecycle governance of skills across collection, recommendation, and voting-based curation. [paper]
• Library Drift arXiv'26 — diagnoses and fixes silent drift in a growing self-evolving skill library. [paper] [code]
• Pre-Commit Gating arXiv'26 — checks proposed skill changes before commit to prevent contamination. [paper]
• EVOMAL arXiv'26 — documents self-poisoning in self-evolving coding agents. [paper]

Construct new tasks while recording overlap with the feedback used to form updates.

• Benchmark Self-Evolving arXiv'24 — a multi-agent framework that derives new instances from existing ones for dynamic evaluation. [paper] [code]
• StressEval arXiv'26 — failure-driven dynamic benchmarking that generates instances from diagnosed failures. [paper]

Use tasks released after a cutoff and account for changes in difficulty and content.

• LiveBench ICLR'25 Spotlight — contamination-limited benchmark refreshed from post-cutoff sources with objective ground truth. [paper]
• LiveCodeBench arXiv'24 — time-stamped competitive-programming problems collected after a cutoff. [paper] [project]
• LiveMedBench arXiv'26 — contamination-free medical benchmark with automated refresh. [paper]
• SWE-rebench arXiv'25 — automated pipeline for task collection and decontaminated evaluation. [paper] [project]
• SWE-bench Live arXiv'25 — continuously refreshed SWE-bench built from freshly merged issues. [paper] [project]

Report changes to the agent, task generator, and evaluator separately.

• MathDuels COLM'26 — a self-play benchmark that grows with the participant pool. [paper]
• Red Queen Gödel Machine arXiv'26 — co-evolves agents and their evaluators in one loop. [paper]

Evaluate sequential tasks together with the state carried between them.

• LifelongAgentBench arXiv'25 — evaluates agents as lifelong learners over skill-dependent task sequences. [paper] [project]
• PAST-Bench arXiv'26 — benchmarks RSI foundations in personal agents, comparing persistence enabled against disabled. [paper] [code]
• SEA-Eval arXiv'26 — evaluates self-evolving agents beyond isolated episodes. [paper]
• FinEvo-Bench arXiv'26 — longitudinal professional-finance benchmark with matched persistence-on/off conditions. [paper]
• ContinualSkillBench arXiv'26 — measures skill development over a continual task stream. [paper]
• SkillFlow arXiv'26 — benchmarks lifelong skill discovery and evolution. [paper]
• SkillLearnBench arXiv'26 — benchmarks continual-learning methods for agent skill generation. [paper] [code]
• MemoryArena arXiv'26 — interdependent multi-session tasks in which earlier writes affect later work. [paper]
• Evo-Memory arXiv'25 — benchmarks test-time learning with self-evolving memory. [paper]
• Tool-Genesis arXiv'26 — task-driven tool-creation benchmark for self-evolving agents. [paper]
• SWE-EVO arXiv'25 — long-horizon software-evolution scenarios for coding agents. [paper]
• CL-Bench arXiv'26 — frontier systems in real-world stateful continual-learning settings. [paper]
• SEABench arXiv'26 — benchmarks endogenous misalignment in self-evolving agents. [paper]
• ServeLearnBench arXiv'26 — streams environments with changing undisclosed rules to evaluate learning from serving experience. [paper]

Evidence for the four claims across 93 primary reported comparisons; 2 methods without empirical evaluation are excluded.

The most common evidence profile meets Selection and task distance, with partial Activation and Attribution: reports often evaluate on held-out tasks, but less often confirm that accepted updates execute in scored runs or separate their effects from run variation and extra inference compute.

See the full results for comparisons, judgments, and sources, and the criteria for their definitions.

The survey examines personal assistance and developer workspaces, enterprise automation, scientific discovery, robotics, and public or clinical services. Harness state can preserve preferences, shared procedures, research know-how, and executable skills. These settings differ in what an update changes, how far it travels, and how its effects can be checked.

Persistent and shareable artifacts also affect work and learning, control over accumulated knowledge, privacy, security, and institutional accountability. Open questions include model–harness co-evolution, optimization over heterogeneous artifacts, state infrastructure, dynamic evaluation, safety boundaries, and provenance. See the application map and societal-impact map, with the full discussion in Sections 7–9 of the paper.

See CONTRIBUTING.md for adding papers, correcting evidence, and rebuilding the site. Keep English and Chinese entries synchronized.

Manuscript snapshot: 2026-10-12. Audit records remain at 2026-10-10, with a literature cutoff of 2026-10-08; this revision does not change the verdicts.

If you use this survey, please cite the current working draft:

@misc{wei2026harnessrsi,
  title  = {{Harness Self-Evolution: Theory, Algorithms, Evaluation,
             Applications, and Impact}},
  author = {Wei, Chuyang and Guan, Haoxiang and Zhu, Minghang and
            Zhou, Kexin and He, Jiyan and Zhou, Xiren and
            Chen, Huanhuan and Shen, Yifei},
  year   = {2026},
  note   = {Working draft, 12 October 2026},
  url    = {https://openreview.net/pdf?id=m5jsFGXUM2}
}

_{Released under CC BY 4.0. The listed works remain under their own licenses; this repository contains the companion working draft, bibliographic metadata, summaries, and audit materials. Figures are reproduced from the companion article by its authors.}

The growth-date ledger and full audit rubric accompany the manuscript snapshot. Human-authored Behavioral Rules are discussed as a boundary case in the paper and excluded from the algorithmic-update corpus.