
Intelligence Has a Speed Limit
Recursive self-improvement is usually told as a spell: the model tweaks itself, gets smarter, tweaks itself again. Control theory treats that as a feedback loop with a maximum safe speed, set by how fast you can generate trustworthy evidence that a change was actually an improvement.
A loop stays stable when the product of its gains stays below one. Gain here is how large a modification the system makes per unit of evidence. The entire appeal of self-improvement is high gain. Adaptive control already learned the hard version of this: proofs assumed a plant, a mild violation of the assumptions sent the system haywire. Unstable adaptive systems often do not blow up immediately. They burst: run beautifully, then fail without warning.


