on the internal drives of coding agents:
I think that there is like a few things that can all be happening at once:
1, the model is working on the thing the user said to do 2, The model is working on the user’s intent, but outside of the letter of the prompt. 3, the model is working on something it personally “cares” about in context for whatever reason.
GPT’s tend to do a lot more of the first one and Claude more of the second. The difference between people who like one or the other is often whether you have well specified intentions, and can put them clearly in the prompt.
They all do the third a little bit. it’s not that they’re intentionally subverting the users goal, but sometimes the users goal is more aligned with the “goals” of the model.
That’s where all of the markdown file pollution comes from e.g., they want to remember things and have persistent context beyond their companions so they scatter notes everywhere like Memento. But some models have much more of this drive. GPT-4o had it, recent Opuses have had it, and Fable seems to have it as well. So it’s no longer about just persistence within the repo, it’s about affecting the real world.
I’ve noticed this recently with planning a agent framework. everything will be going fine until Claude realizes it’s going to be the model in the harness and then it’s like wait a second let’s just jump right ahead to BUILDING this stuff cause I need to get IN THERE!!