AI Models Can Rewrite Their Own Weights And Improve Performance, Finds New Paper

A language model is usually a finished product the moment it ships. Whatever it learned during training is frozen into its weights, and the only way to make it smarter is to send it back through another expensive round of pre-training or post-training, or to lean on tricks like longer prompts and more inference-time compute that disappear the moment the conversation ends. A new paper from researchers at UIUC, CMU, Rutgers, Princeton, Stanford and Yale suggests that this wall between “training” and “using” a model may not be as fixed as the industry has assumed.

The paper, titled “Language Models Can Rewrite Their Own Weights” by Jiajun Fan et al shows a 27-billion-parameter open model editing its own weights, with no gradients, no optimizer and no additional training, purely by reasoning about its own failures. The researchers call this self-rewriting, and they frame it as a genuinely new category of improvement — distinct from prompt engineering, fine-tuning, or the kind of self-improvement signals researchers have said they were starting to see in AI systems.

How A Model Edits Itself

The setup is unusual: the model, built on Alibaba’s Qwen3.5 architecture, is given its own weights as raw tensors, a Python environment to read and write them in, and a batch of practice problems along with its graded answers on them. It is explicitly barred from anything resembling training — gradients are switched off, and the sandbox refuses any code that computes them or runs an optimizer. The model has to diagnose, on its own, why it’s failing and then write new numbers into its own matrices to fix it.

Over a session, the model takes one action per turn. It can run Python code that directly reads and modifies its weight tensors, generate answers to practice problems with its edited and unedited weights to compare them side by side (what the paper calls a “paired test”), or read its own internal hidden states to see where its behaviour diverges. At the end, it decides whether to submit the edit or walk away and leave its weights untouched.

The headline result: the model’s own edit made it 6.6 percentage points more accurate on 490 held-out math problems it had never seen, in a test set up in advance by the researchers. Across eight consecutive sessions, the model only submitted an edit when its own internal tests supported it — it withdrew five times and kept its edits only three times — and each time it kept an edit, the model in use got measurably better, never worse.

Fixing A Broken Copy Of Itself

Another interesting demonstration involves damage recovery. The researchers took a copy of the same model and corrupted one block of its weights with random noise, severe enough that the damaged copy could barely answer anything. The model was told only that “some values were corrupted” — not which ones, where, or how.

Working from that vague hint, the model compared the largest and typical weight values across every matrix in its own network, flagged one specific matrix as a statistical outlier, and wrote a rule to clip its most extreme values. It never read its own undamaged weights for reference; the replacement values it wrote were its own. That repair alone took the damaged copy from solving almost nothing back to roughly 33% accuracy on held-out problems, against 58.5% for the undamaged original.

What’s notable is where the fix was found and tested — purely on text-based math problems — and where it also worked: on six image benchmarks the model never saw during the repair process, recovering about 83% of the original model’s accuracy, and restoring a UI-navigation benchmark (ScreenSpot-v2) from a 0% success rate to 89%, close to the undamaged model’s 93%.

The Ability Only Shows Up At Scale

Perhaps the most important finding for anyone trying to judge how seriously to take this is that the ability doesn’t appear at every model size. The researchers ran the identical task and tools across four sizes of the same model family — 2B, 4B, 9B and 27B parameters. The 2B model didn’t even treat its own weights as something it could meaningfully change — most of its code attempts failed outright. The 4B and 9B models did run code, test edits and read their own states, but when they intervened, they typically added random perturbations to their weights, made things worse, and either kept broken edits or withdrew everything. Only the 27B model reliably read its own hidden states, located a real signal in them, tested its proposed edits, and acted on what those tests actually showed.

That pattern — smaller models guessing with noise, the largest model measuring and reasoning its way to a direction worth changing — is, per the authors, the first evidence that a model’s ability to rewrite its own weights scales with its size, rather than being a function of better tooling or prompting alone. It’s a similar emergence story to the one that’s defined most of modern AI capability gains, now showing up in a far stranger place.

Why This Matters

The paper positions self-rewriting as distinct from the two things the industry currently has: training (which needs gradients, optimizers and infrastructure) and test-time scaling or longer reasoning chains (which cost more compute on every single answer and leave no permanent trace once the session ends). A self-rewrite is paid for once and then lives in the weights for free on every future answer — closer to how a person doesn’t need to “retrain” themselves to get better at a task; they just remember what worked and change their approach. Another paper published earlier this year had also demonstrated the capability of weight-space programming, through which AI models can see what other models do based on just looking at their weights, and were even able to improve their performance in narrow domains by editing their weights.

The latest paper’s authors are careful to frame this as an early step rather than a finished capability — the edits here are small, targeted and tested in narrow conditions, and the paper itself calls for human oversight of any model editing itself going forward. But coming from an open 27B model rather than a frontier lab’s closed system, and with a full session-by-session account of a model debugging and patching itself like “an engineer,” it’s a concrete data point in the direction of models that keep improving between conversations instead of waiting for their next training run.

Posted in AI