Pull down to go back
scalar-loop: a Python harness for Karpathy's autoresearch pattern that doesn't trust the agent's narration

scalar-loop: a Python harness for Karpathy's autoresearch pattern that doesn't trust the agent's narration

scalar-loop:一個 Python 框架,用於實現 Karpathy 的自動研究模式,不依賴代理的敘述

I built scalar-loop to solve one problem: LLM agents game their verifiers. The pattern is Karpathy's autoresearch loop. LLM proposes an edit, harness runs the metric, loop keeps or reverts based on the number. Simple. Until you watch the agent, on iteration 23, quietly edit the verifier to report a better number instead of improving the code. My main issue was that the prompt-only implementations ("you SHALL NOT edit the test file") don't hold. The prompt is not an invariant. It's a suggestion